Simon Cowell -- Britney Spears Almost Quit 'X Factor' on Day 1

Britney Spears tried to back out of her $15 million "X Factor" deal on the VERY FIRST day of production ... after experiencing a panic attack on the set ... this according to Simon Cowell.

The "X Factor" creator just phoned in to "TMZ Live" ... and told us his people "half-lied" to the media back in May when they said Britney only walked off the set because she had to go to the bathroom.

"No one is going to admit that on the first day your highest paid star after 20 minutes has walked off set," Cowell told us.

So what really happened? Cowell admits Britney "had a moment on the first day where she said to me 'I'm not sure I can do this.'"

Of course, Cowell says he talked Brit back into the gig ... and now she's not only loving it, but she's developed into a kick-ass judge.

Cowell also blasted his rivals at "The Voice" ... claiming "Voice" honcho Mark Burnett was full of crap when he told TMZ he had no idea NBC moved his show to Wednesday night, where it will go head-to-head with "X Factor."