Kim Kardashian, Kris Jenner Sue Ray J for Defamation

Kim Kardashian and Kris Jenner are fed up with Ray J, claiming they're the subject of a federal criminal racketeering investigation, so they're suing him for defamation.

Kim and Kris say it's just the latest in a decades-long campaign Ray J has waged against them. The lawsuit, filed by Alex Spiro, says Ray is "unable to accept the end of his fleeting relationship with Ms. Kardashian over 20 years ago."

They say he has repeatedly exploited their prominence for personal gain.

The lawsuit specifically mentions TMZ's Tubi doc, "United States vs. Sean Combs," where Ray said RICO charges against the Kardashians would be appropriate. He says, "If you told me the Kardashians was being charged for racketeering, I might believe it."

That comment in the documentary "was designed to plant the seed" ... according to Kim and Kris, but they say he ratcheted things up during a Sept. 24 live stream -- with Chrisean Rock -- where they say Ray falsely declared the feds had launched an investigation into the Kardashians.

Ray claimed, "The feds is coming, there's nothing I can do about it. It's worse than Diddy."

BTW, Kim and Kris also say this isn't Ray J's first time telling lies about other celebs ... for example, RJ made claims months ago about having intimate relations with Sexyy Red.

She ended up hopping in his DMs after his comments took off, and she shared a screenshot of their convo ... in which Ray J says, in part, "You know I playin I'm always trolling and you know I love you ... sorry if you mad at me."

He also told her he'd make things right by making an on-camera apology ... which he did not long after.

K&K say Ray's attacks have caused them "substantial and ongoing harm to their personal and professional reputation."

Spiro tells TMZ ... “Kris Jenner and Kim Kardashian have never brought a defamation claim before nor have they been distracted by noise -- but this false and serious allegation left no choice."