My headphones aren't working on one side

I've bought new headphones and for some reason they won't work on one side when I plug them into my desktop. I've used them on another device, like my laptop, and they work fine. But for some reason when I plug them into my PC, they stop working on the right side. I even ordered a replacement, in case it may be the headphones at fault, but they came with the same issue.

I tried troubleshooting the issues but it detected no problem, so I have a feeling that the headphone jack may be the problem, though I don't remember it being damaged.