Yes, but they're so young I'm not sure that's the problem, they should still be living on the stored food in the seed. Also it wouldn't cause wilting, although I'm not sure that they are wilting as opposed to the outer leaves declining (based on the picture it looks like there is an emerging new leaf that looks normal).
Usually peat moss is just part of the mix for seed starting, along with other components such as perlite or vermiculite to add more structure and maybe improve drainage/aeration depending on the texture of the peat.
Peat moss also has problems being wetted (which is why commercial peat-based media often contains a "wetting agent" ). It can happen that the top of the media dries out after wetting but at root depth it is still soggy, so you could check that out.
If it's Canadian sphagnum peat moss it will be quite acidic, normally dolomitic limestone would be added but I'm not at all sure that would be a problem for plants this young and in containers).
Peat moss does have little fertility of its own so you will ultimately have to start fertilizing but my guess is the yellowing has more to do with water or aeration. Can you check that pot with the unhappy seedling to see if it is dry or wet lower down?
