For ground-based exoplanet imaging, atmosphere turbulence is changing from time to time, which induces speckle noise and hampers the high-contrast imaging of the faint exoplanets. We propose a differential-imaging polarimeter dedicated for exoplanet high-contrast imaging. The system contains a zero-order half-wave plate (HWP) located on the optical pupil plane, which can rotate to modulate the incoming light, and a Wollaston prism (WP) is used to gen-erate two polarized images, which is used for simultaneously polarization differential imaging and thus our system is fundamentally immune to the atmospheric turbulence induced temporally-variable wavefront aberration. For the star image is much brighter than that of the exoplanet, the exoplanet image cannot be seen before the polarimeter. Since starlight is in generally unpolarized, while exoplanet light is somehow polarized, we focus our discussions on the subtraction of the on-axis starlight, which will automatically result in an extraction of the polarized exoplanet light. However, the direction subtraction of the left side and right side image cannot yield a good result. For example, the intensity difference in both side beams limits the performance of such subtraction. In addition, the image distor-tion, which will result in a difference for the star point spread functions (PSFs) on both sides, will also seriously limit the subtraction. Therefore, in order to achieve best differential-imaging performance, we also propose an eight-variable optimization algorithm, which is proven to be able to effectively correct the distortion and the inten-sity unbalance between the two differential images. Laboratory experiments indicate that the proposed polarimeter combined with the optimization algorithm successfully achieves an extra contrast of about 30~50 times in a close angular distance in the region of 3λ/D~5λ/D, and the contrast improvement throughout the region of small angular distance by the polarimeter is apparent. The experimental results demonstrate that our polarimeter works well for both perpendicular polarization components. In conclusion, our polarimeter has the following advantages: 1) The system is simple and compact, which is different from the traditional polarimeter that employs the mechanical modulation approach; 2) Image distortion and intensity unbalance are considered and optimized for best contrast performance; 3) Most important, each polarization component is measurement simultaneously, which is fundamen-tally immune to the rapidly-changed atmospheric turbulence induced speckle noise. The work is demonstrated to be a promising technique, since our polarimeter has the potential to achieve an overall contrast better than 10-8 when used with current extreme adaptive optics and coronagraph systems, and such work is critical to achieve the scien-tific goal toward the direct imaging of giant Jupiter-like exoplanets for a ground-based telescope.