When I've removed old one cane wonders, they have most often not had the fibrous feeder root system you expect a vigorous, vital plant to possess. Something has prevented the plant from growing and maintaining it.
Sometimes, it's been due to mulch or erosion burying the bud union and the plant usually hasn't gone "own root" to form the new feeder root system in the elevated soil. Sometimes it has, but doesn't produce the quality of feeder root system necessary to support a vigorous growing plant. Raising the soil too far can actually suffocate a plant. Sometimes I've found galls either at the crown or on the roots which strangle the upward flow of water and nutrients from the roots into the canes. Occasionally it's been due to the organics in the soil being depleted over the years, leaving mainly silt and clay. That inhibits those fibrous roots required to support the plant because of compaction; reduced water percolation which exchanges the carbon dioxide the roots 'exhale' and drawing in oxygen from the air as it drains downward through the soil. The remaining soil could be too dry or too wet. Both will prevent growth of the fibrous feeder roots.
After investigating to the best of your ability and determining which, if any, of these is the case, you can more easily decide on a treatment. If the plant is now set too deeply, excavating the soil from around the bud union, applying organics and replenishing the mulch to help to gradually increase aeration of the soil can help. As they break down, Humic Acid is released which helps to break down clay and improve soil friability. Earth worms and other soil organisms help to burrow through and churn up the soil, also improving drainage and oxygen exchange.
If gall is the culprit, there is often very little which will help. You can try cutting out as much of the tissue as possible, then improving the soil as described above, feeding and watering well in hopes of an improvement.
If all seems OK and the plant still isn't responding, you sometimes have to resort to the "kill or cure" tactics. I hate removing growth buds from a plant which doesn't have many left, but sometimes you have to in order to shock the plant into doing something. I prefer doing as much soil improvement as possible in combination with the rejuvenation prune, as I feel it provides the best chances possible.
I also prefer not removing more than a third of the plant at a time. There are stored nutrients in the cane. There many not be much, depending upon how long it's been ailing, but there are some. Over time, the capillaries in that cane become restricted, just as our arteries do, restricting sap flow. Combine that with reduced water intake due to reduced feeder roots (perhaps even reduced anchor roots, the thicker, woodier roots) and you can see why there isn't much growth. Hopefully, improving the soil, fertilizing and drainage conditions, combined with the pruning will result in the plant putting out new growth. It won't always, but it sometimes does. If removing about a third of the cane doesn't improve the situation, you might try a bit more, if there is more to try.
It's also entirely possible the problem is viral. Many commercial roses have only been available infected with a number of plant viruses. Some of them haven't been shown to inhibit growth, a few have. Depending upon how resistant the plant's immune system is to the particular virus, or combination of viruses, and the condition of the plant and its growing conditions, you may be able to bring it back from the brink.
