Raman spectrum, as a kind of scattering spectrum, has been widely used in many fields because it can characterize the special properties of materials. However, Raman signal is so weak that the noise distorts the real signals seriously. Polynomial fitting has been proved to be the most convenient and simplest method for baseline correction. It is hard to choose the order of polynomial because it may be so high that Runge phenomenon appears or so low that inaccuracy fitting happens. This paper proposes an improved approach for baseline correction, namely the piecewise polynomial fitting (PPF). The spectral data are segmented, and then the proper orders are fitted, respectively. The iterative optimization method is used to eliminate discontinuities between piecewise points. The experimental results demonstrate that this approach improves the fitting accuracy.