Multiple sources connected to the negotiations tell TMZ ... "X Factor" producers cannot sign Khloe to the show unless E! gives permission -- exclusivity is part of the Kardashian's contract with the network.
We're told the problem isn't so much that E! doesn't want Khloe to do Simon Cowell's show ... it's that Khloe would have to promote the program by spending a lot of time doing interviews, social media and making appearances, and E! is worried it might dilute Khloe's impact on their reality shows and also create time conflicts.
One source directly connected with the negotiations tells TMZ ... it looks like both sides will reach an agreement and Khloe will be able to host "X Factor," but the source added, "I wouldn't bet my house on it."
We are told, in the event that Khloe's deal falls apart, they would probably pick Mario Lopez as the second choice, but "X Factor" has its sights set on Khloe as a single host.