Floyd Mayweather says Andre Berto is CRAZY if he thinks he's wasting his time prank calling the boxer -- with a rep for The Money Team telling TMZ Sports, "He's not even on Floyd's speed dial."
Berto appeared on "TMZ Hollywood Sports" on Wednesday ... and told us he believed Mayweather has been terrorizing him on the phone over beef that has to do with a woman.
"I know it's him on the other line. I can hear him breathing," Berto told us.
But a rep for TMT tells TMZ Sports ... Berto is "delusional" -- saying, "If they fight, we will see who has the last laugh."