think its frass - excrement from wood-eating pests like - caterpillars, termites or carpenter ants
try to catch or see what is doing it (may only happen at night)....what it is, will help to determine what to do next ....
from your 2nd picture ...inclined to think carpenter ants may be the culprit ...because of the 2 holes in the leaf
