Simon Cowell -- Targeted in 'X Factor' Sabotage Plot

There is a war brewing behind the scenes at "X Factor" ... with two judges vowing to destroy each other ... and TMZ has learned it's the contestants who could feel the repercussions.

Sources connected to the show tell us ... ever since judges Simon Cowell and L.A. Reid were assigned to mentor competing groups of contestants on the show (Reid with the boys, Cowell the girls) ... the two have been ruthless with each other ... even after the cameras turned off.

Our spies overheard Reid telling people he plans to play "mind games" and "dirty tricks" with the members of Cowell's team ... explaining, "Even if I have to give really bad advice to the other contestants when their mentors are not around ... I will!"

We're told Cowell seemed to be wise to the scheme ... and even called Reid out on it after a rehearsal ... but sources say L.A. is adamant he'll do "whatever it takes" to win and doesn't plan on letting up anytime soon.

Dirty.