There are many Ficus trees that grow in ground here in Florida:
Ficus benjamina (Weeping Fig)
Ficus lyrata (Fiddleleaf Fig)
Ficus elastica (Rubber Fig)
These two are listed on the Florida EPPC's (Exotic Pest Plant Council) list:
Category I Non-native Invasive:
Ficus microcarpa (Laurel Fig):
Category II Non-native Invasive:
Ficus altissima (False Banyan)
edited to add: And they all get HUGE!