The actor was visiting an L.A. County building Monday -- where it looked like he was handling some biz -- when a photog asked how his Father's Day went. Djimon didn't mince words ... he says it would've been nice to have seen his and Kimora's kid, Kenzo, let alone a convo.
The camera guy asks point blank how long it's been since he's seen the boy? Djimon holds up a five, but doesn't get specific about time frames. He then says he can't even recall.
Waiting for your permission to load the Instagram Media.
Oddly enough, Djimon posted a photo of him and Kenzo together on Sunday ... but it's unclear if it's just an old photograph he threw up for the holiday.
As Djimon walks back to his car, someone on the street asks about fighting for custody -- but he quickly makes it clear that he's not engaging in a custody battle with his baby mama.
Kimora and Djimon dated for 5 years between 2007 and 2012 -- they had one child together, who was born in 2009.
When the photog asked if Djimon had any message for Kimora herself ... he scoffs and closes the door in frustration. Something tells us trouble could be brewing here ...