A new image registration algorithm with robust cost function and progressive refinement estimation is developed on the basis of direct method (DM). The robustness lies in M-estimation to avert larger local noise and outliers. Moreover, the progressive refinement model estimation under the multi-resolution framework, where the initial parameter values of coarse level are estimated by Mellin transformation, is adopted so as to get global optimization and reduce search space. Experiments show that the proposed algorithm greatly extends the scope of the conventional DM of image registration.