potting soil into the garden is ALWAYS good
Garden soil into a pot is NEVER good.
Potting soil is almost dirt free- its high drainage, fluffy. Good quality potting soil is moisture neutral, typically its mostly pearlite and peatmoss.
Dirt has things like clay- that holds water and rots roots or sand that drains so fast plants cant uptake the moisture they need, which you can get away with a a large surface area- like a 200+ sq ft garden, but contain it in a gallon pot, eghh- death sentence.