You might want to try a really fast draining mixture for the semps. Morning sun, or sun that comes from a strong angle (not vertical) is likely the best. Also I've heard of people growing them outside, but under a rain shield. Think a carport with a roof, or an awning attached to the side of a building (but do not use those paper umbrellas that people put in drinks!).
I've grown some semps for two years now in just a pure mixture of chicken grit. Small gravel, mixed with course sand, would be what I would try there. Over time, the mixture will gradually pick up bits of dirt, leaves, etc that will provide some nutrients. I'm assuming that you are growing them in pots. If the pot is sitting on the ground, make sure water does not pool around it. I've also had a few plants sit in or under water for a few days without serious problems. They were, however, healthy plants when the problem occurred and they had a chance to dry out after.
Once the plants have established in their pot, they should do much better for you. Since your conditions are somewhat extreme, try to limit moving the plants, transplanting and propagation to the cooler times. During the heat and rainy seasons, just leave them alone, out of the rain if you can. Any injury when under that kind of stress could lead to problems. I'd find a way to get a rain shield going. Have you looked into hoop houses with plastic or a rain shedding shade cloth? Might be the easiest way to go.
Those of us in the colder climates also go through periods when the plants don't look too well, with dried up leaves or poor color, etc. Sometimes we are lucky and have snow cover, so we don't get to see and fret over them at that time.
