I think I do see some small white specks on your leaves and suspect you may have mealie bugs. Mealie bugs are insidious because they het down into the cracks and crevices and can get into the new leaves before they unfurl, and start sucking on the plant. They don't actually eat the tissue as in chew it. But they can cause stunting and problems with leaves at emergence. It could also be a small beetle, or a snail, or something that got down into the leaf and chewed on it when it was not open yet.
I don't use Neem on any thing, I stopped using it 15 years ago. I find it worthless. It eventually builds up and causes a waxy layer that can interfere with plant transpiration, and just make it a mess. That is MY personal take on Neem, I know lots of people do like it. I use systemic insecticides that kill the bigs that are on the plant immediately and then are absorbed by the plant tissue and keep killing for a week. My choice is Orthene, its not everyone's cup of tea, but there are others out there.