Joe Simpson -- I'M NOT GAY!!!

Rumors be damned ... a frustrated Joe Simpson is telling friends and colleagues the stories that he's switched teams are absurd.There have been several reports out recently that Joe has told his family he's gay and that's why his marriage fell…