Do you take off all remaining small figs when harvest has finished?

use them to make more fig trees, and continue this until your fig trees cover the whole planet :)