James Woods 'Family Guy' Candy Joke Made NO SENSE to Me

4/10/2013 3:30 AM PDT

It's almost incomprehensible ... James Woods' infamous "OOH A PIECE OF CANDY" joke on "Family Guy" was nearly a bust ... because he tells TMZ, he didn't think it was all that funny when he did it ... at first.

Woods -- who said he loved playing himself on FG "more than anything you can imagine" -- tells us, the joke just didn't make any sense ... and then it occurred to him ... that was kinda the point.

And the candy joke is just the tip of the iceberg ... Woods says some of the stuff that went on backstage with Seth MacFarlane should've landed him in jail.

Check out the -- OOH A PIECE OF CANDY!