I consider some of the plants I grow 'rare' because you can't buy them in a nursery or garden center (usually, that is). I'm always on the lookout for something odd or unusual.
Then again things like Plumeria or other tropicals are considered rare in this area because they're not grown here.