JoeBrug, It appears the older leaves with will show signs first of insect damage or have you had high hot winds? Try to remove and just keep the newer healthy leaves.
An old trick is lay your garden hose out so the water in it warms then use that water to hit your brugs first when you water. Mites hate warm water!! Another hint is new stalks growth is much better than the old trees trunks. So when you trim back in the fall that should help you cutting back for new plants next spring.