Why such a big difference? There are a lot of possible factors. Arm angles have something to do with it. The break of the ball also does. Is it harder for a hitter to hit a breaking ball that's coming at him and breaks away from him than one that starts away from him and breaks in? Some say yes, some say no.
We put together the charts below to see how often pitchers throw their breaking ball against righty and lefty batters. What we found is this: pitchers who throw sliders will use it against same side hitters, but will substitute the change-up when facing a hitter from the opposite side.
The surprising part was the curveball. We expected the same trend for the curve, but pitchers throw curves, in aggregate, the same amount regardless of which side of the plate the hitter stands.