Add lots of compost and then mulch it heavily after you plant in it. Plant shallow rooted things that like sun. Zinnias, marrigolds, petunias. After a few years of adding compost and mulch, the soil should improve alot. Do you see any earthworms in it at all when you dig?
You could also consider adding some interesting rocks, just pile them up and then plant a rock garden on top. Like sedums, hens and chicks, rock moss, agave, yucca maybe mini or border iris would do ok.
