Broadband transverse displacement sensing by exploiting the interaction of a focused radially polarized beam with a silicon hollow nanodisk is proposed. The multipolar decomposition analysis indicates that the interference between a longitudinal total electric dipole (TED) moment and a lateral magnetic dipole (MD) moment is dominant in the far-field transverse scattering in the near-infrared region. Within a broadband wavelength range with the width of 155 nm, the longitudinal TED is almost in phase with the lateral MD, and then broadband position sensing based on the sensitivity of scattering directivity to transverse displacement can be achieved.