Based on the traditional directional coupler, we proposed a scheme to design on-chip polarization beam splitters using an inverse design method. In our scheme, the coupling area of the designed devices are only 0.48 μm×6.4 μm. By manipulating the refractive index of the coupling region, the devices can work in C-band, L-band, O-band, or any other communication band. Different from conventional design methods, which need to adjust the design parameters artificially, if the initial conditions are determined, the proposed scheme can automatically adjust the design parameters of devices according to specific requirements. The simulation results show that the insertion losses of the designed polarization beam splitters can be less than 0.4 dB (0.35 dB) for TE (TM) mode at the wavelengths of 1310, 1550, and 1600 nm, and the extinction ratios are larger than 19.9 dB for the TE and TM modes at all three wavelengths. Besides, the extinction ratios of both polarization states are more than 14.5 dB within the wavelength range of 1286–1364 nm, 1497–1568 nm, and 1553–1634 nm. At the same time, the insertion losses are smaller than 0.46 dB.