I will recommend to check the root system, I have 1 fairy castle that grown something similar to that, those brown stuff only grown on the outer layer of the cacti, the plant also stop growing and somehow shrink. Below is the current situation, last month I get rid of more than 10 branches that seriously damaged.
This morning I repot the plant, the roots still exist, but I have a strong feeling that it should at least have a root ball that is triple in size, I checked the soil, there are some very small white bugs moving around, very small and difficult to see them without black color background, not very sure whether they are the cause of the problem, anyway I just change the soil with a new media.
And last month when I get rid of the damaged offsets, I kept and root 1 pcs that had less damage, and today it grows very well, that's why I suspect there may be some damage on the root system of the mother plant
Another parameter to consider, normally I water all my succulents with ground / rain water, just a couple of weeks before I had this problem, I watered the plant with water from the koi pond ( assuming that the water from pond will have more nitrate for the plant), this pond water may have brought in the parasites or something that damaged the roots of this plant.