Why should I want someone that doesn’t want me anymore

have been married for 21years a lovely son and daughter. Was extremely happy in the past and never looked at anyone else. 18 months ago I found myself flirting with a man at work then found myself saying yes to a drink one Saturday lunchtime. At first I felt guilty as I made an excuse to my husband about where I was going, the date turned out well and before I knew it we kissed and had sex. I didn’t feel guilty I just knew I needed more. We arranged to see each other in a hotel next time and the sec was fantastic. I also discovered that he had finished with his partner and was now single which really attracted me. He told me he lived me and was texting every day wanting more but I couldn’t give it to him I had to be careful after his can you get to know someone when you can’t spend a great deal of time together. We also went into lockdown which made life difficult as he was not local. He waited for me for 18 months but I couldn’t give him what he wanted I felt cruel like I was holding him back. The amount of weekends I dreamt about seeing him instead spending time with my husband who I wasn’t getting on with purely because I was devoted to my AP. I cut emotional ties with my husband and we have not had sex for 2 years he knows something is wrong but I have not told him the truth. I needed time to process everything and was not going to give up on my family and my lovely home. I often thought that if the AP was off the scene I would start to live my husband again. Without any warning my AP ran out of patience and wanted to take a break I stayed to not get those living messages they got less and less and I suddenly found myself heartbroken. I have discovered since he had been out on a date on Facebook and it has cut me to the core, in someways I don’t blame him I didn’t give him what he wanted but I sometime think I went through 18 months of being disturbed and feeling guilty and wondering what my next lie would be so that I could see him and in someways I z as m relieved that I don’t have to do that anymore. I have had closure from him as we did fall out and he had now said he is glad to be shot of me which hurts but I just cannot get him out of my head. I have been waking up at 3.30 then I lie there and cannot stop thinking of it my heart sinks to the out of my stomach. I know it’s over but why do I feel so bad, why should I want someone that doesn’t want me anymore. My feelings for my husband are not there and the thoughts of sex with him throws me off before all this my life was good. Please advise me what to do as I feel so disturbed by it.