Perhaps your attackers are the same as the tiny flies plaguing us now. Some people call them midges. Ordinary insect repellents do not seem to impress them, but vanilla or peppermint oil do seem to discourage them. You might try some with your flies.
