Loretta, I suppose its a judgement call, if the plant seems healthy enough and big enough to support, let them grow.
My plants were scrawny and pitiful this year, managed to set a few fruit. Even very small, they seemed ok cut up in tomato sauce.
Do watch and use the eggplants when big enough; would be a shame and stress the plant unnecessarily to leave them too long past prime.
