My Dad left my mother for another woman when I was 14. We weren't close before then, because he was always out sneaking around on her. Don't get me wrong, my mother can be quite insufferable at times (i.e. most times). Looking back on them now from an adult point of view, the fact that they were ever married is so fucking illogical given their respective personalities.
I would fight him if it came to it, but I wouldn't fuck him up too bad...after all he is my one and only father. He is 5'8" and around 185# and I go 6'2" and around 230#.
So, STDiva, give it serious thought about consequences with the relationships with everyone in your family, and do what you must.