X-ray drive asymmetry is one of the main seeds of low-mode implosion asymmetry that blocks further improvement of the nuclear performance of “high-foot” experiments on the National Ignition Facility [Miller et al., Nucl. Fusion 44, S228 (2004)]. More particularly, the P2 asymmetry of Au's M-band flux can also severely influence the implosion performance of ignition capsules [Li et al., Phys. Plasmas 23, 072705 (2016)]. Here we study the smoothing effect of mid- and/or high-Z dopants in ablator on Au's M-band flux asymmetries, by modeling and comparing the implosion processes of a Ge-doped ignition capsule and a Si-doped one driven by X-ray sources with P2 M-band flux asymmetry. As the results, (1) mid- or high-Z dopants absorb hard X-rays (M-band flux) and re-emit isotropically, which helps to smooth the asymmetric Mband flux arriving at the ablation front, therefore reducing the P2 asymmetries of the imploding shell and hot spot; (2) the smoothing effect of Ge-dopant is more remarkable than Si-dopant because its opacity in Au's M-band is higher than the latter's; and (3) placing the doped layer at a larger radius in ablator is more efficient. Applying this effect may not be a main measure to reduce the low-mode implosion asymmetry, but might be of significance in some critical situations such as inertial confinement fusion (ICF) experiments very near the performance cliffs of asymmetric X-ray drives.