My theory is that, if you build up visible fungus, the surface is staying too wet. If you think of those coffee grounds as a mulch, they are much too fine. They hold water instead of letting it pass through and drying out.
Coffee grounds are very fine and tend to hold water. If you must use them in pots, I'd suggest a thinner layer of grounds, and mix them with something gritty or chunky, like pine bark shreds in sizes from 1/10" (2.5 mm) to 1/8" up to BB size (0.177" or 4.5 mm).
If the surface dries out quickly after watering, you may have less fungus.
But the main problem is that organic additions to soil work better in the ground than in containers. If you're thinking of the coffee grounds as spot-composting or sheet-composting, it would work better outdoors on the soil or in a compost heap. The ground, outdoors, can support diverse life that will compost things like coffee grounds.
Maybe top-dressing each pot with a little finished compost would meet your goals? Or aerobic compost tea?
Organic additions mainly benefit soil as they decompose, and pots usually are not big enough to maintain a soil ecosystem or a compost-heap-ecosystem.
That fungus comes from the grounds "trying to decompose", but instead of being inside a compost heap with an entire "community" of microorganisms, worms, insects etc. working together, it's on the surface of a small pot and just one kind of fungus took over.
Small pots just aren't "the Earth". Trying to spot-compost in a small pot is going to be difficult.
Many or most people rely on a soilless mix in pots to maintain drainage and aeration, then supply nutrients with either soluble fertilizer or something like liquid seaweed. Maybe if you use big pots and replace the soil every year with fresh soil, you could do more in-pot composting.
