Sounds like your semps were doing well when the weather wasn't so dry. With the higher temps and drier conditions, you may have to water more often. Sounds like you have good drainage with the sand and gravel mix so a thorough watering when the conditions are dry will help. Not all semps do well in full sun; some do better with just morning sun. Can the semps that are suffering be moved under the rock overhang too?
Some people fertilize their semps; some don't. I think it may depend on one's soil conditions. I find that a lot of semps I've purchased have these little time-released fertilizer pellets already in the soil.