Hate to say it, but a lot of wives want the fantasy... but without the bf/husband. I've played with wives who said it made them feel uncomfortable to be watched, or to be watched by HIM specifically. Reasoning was mostly out of shame or didn't want to be seen as better pleasured by another man, or bigger dicked man. Women are far more self-conscious then men when it comes to sex (from what I've seen). Some even seem to want to hide it, don't want their man clinging to them later that night or after the date, etc. Lots of reasons, and sadly you as the cuck aren't part of the equation.
So you have to ask yourself, are you good with her playing around and keeping you completely in the dark, or do you need to be there?