I'm encouraged to hear that you have a wife and fam. Sometimes I worry that marriage won't be possible for me because of my problem; but I see that many folks on this site are in relationships/marriages.
At the same time, I'm surprised to hear that your wife is unaware of your difficulty! Wouldn't it only make things easier to get it out in the open? I could be wrong. But it might make it possible to excuse yourself somewhere if you needed alone time, which would make for more better days and more quality time spent with fam (if that's what you want).
I began being open about my issue with friends/girlfriends a few years ago. Only once did it backfire. Of course, every situation is different and you would know best
PS the bit about shaking your confidence mentally . . . man. On a good day, sometimes, my cup runneth over with confidence. Then the next day I'm bloated and crampy and cranky, just a recluse who wouldn't want a girl to touch him. I feel you, man.