Or, depending on what you mean, not until they've dried and been ground up.
The little one at the top gave me the clue that it was time to harvest them. I went to check on how ripe they were and saw that it had fallen off the plant, harvesting itself.
Based on previous experience, these five should give me about a year's worth of cooking, since I cook only for myself most of the time.
Drying is advancing in the second picture. It always reassures me when they fail to rot.