I have to agree that this is a ridiculous question. Now if, on the other hand, you had seen him in the stall, wandered in and given him head? Well, yeah, we gotta call you gay on that one.
However, you turned away, apologized, and don't find yourself aroused by men. You don't sound gay...