I use poultry grit without the oyster as a topper in my rock garden. It's basically just small rock / clay? It does help the texture of our sandy SW soil and is much less obnoxious than perlite. I don't use perlite out in the garden as it works it way to the top in my sand based soil.
