I haven't faced this when married but something came up in between marriages.

My ex wife (from my first marriage) told me that one of her friends - who I had always found extremely attractive - was also attracted to me.

After the marriage was over, I got together with this friend, and we hit it off spectacularly. She was both beautiful and brilliant, I loved hanging out with her - but I also knew i wasn't interested in a serious relationship with her. I had the sense, but was not sure, she DID want a relationship.

I talked this over with several friends and they all agreed, if I slept with her she would end up being very hurt.

We only got together one more time. She invited me to her place, and made a LOT of wine available. I realized not long after I got there that I had made a huge mistake in going. There was no question she wanted to sleep together.

What kept me from sleeping with her? I kept repeating the words of my friends - both of whom I trusted deeply - and kept thinking about how hurt she would be when it was clear I only wanted a causal relationship.

Did I look back and wonder some time later if I made a mistake?

sure.

Did I make a mistake?

I don't think so. If it helps you to think of the pain you might cause someone else, that may be useful at some point in the future for you.