Ben Affleck -- I Didn't Kick the Paps ... They Were Scaring My Daughter!

Ben Affleck didn't kick anyone yesterday and was merely trying to get the paparazzi to back off ... because they were scaring his daughter ... this according to sources close to the Oscar winner.

Ben was holding 4-year-old Seraphina yesterday when they encountered a large mass of paps. According to our sources, the photogs got too close for comfort and little Seraphina started crying because she was afraid.

According to our sources, Ben asked the paps (none of whom work for TMZ) twice to back off and when that didn't work ... he said "You need to be this far away" and swung his leg to demonstrate.

Our source says Ben never had any intention of kicking anyone and he was merely trying to protect his daughter from the paps ... who he feels have been getting more and more aggressive.