What do you mean by jitters ?
Whats your relationship with your wife ? Long term marriage with good relationship ?
I have been through this......see my 'nervous' posting.
In my own case I can shut the experience in a box without guilt or remorse. I dont crave any relationship with a TS/Shemale, for me it is an intimate massage, no love.
Also, I thought what my reaction might be like if my wife announced she was of to see a woman who was dressed like a guy for some lesbian activities.......phew, I would probably pay to watch that !!
Only you truly know how you handle these things. I used to know a guy that was always after women for what he described as 'just sex'. He always ended up in some sort of relationship with them and started to get love sick. If you are like that, the sort of total romantic that falls for every woman, then you might find it difficult to seperate the physical from the emotional.