Abstract: To further improve the accuracy and efficiency of multi-source remote sensing image matching, an algorithm based on contourlet transform, Hausdorff distance and improved particle swarm optimization was proposed in this paper. Firstly, the target image and reference image were decomposed to the low resolution image using contourlet transform. Then, wavelet modulus maxima algorithm was employed to extract the edges of the low-frequency subbands, and Least-trimmed-squares Hausdorff Distance(LTS-HD) was used as similarity measure for image matching. Meanwhile, the extremum disturbed and simple particle swarm optimization was introduced to get the rough matching results. The position of rough matching results were corresponded to the original image and then the matching between the higher resolution images could be implemented stepwise up to the full resolution images. The experimental results show that, compared with those of other existing sensing image matching methods, the proposed algorithm has the high accuracy, efficiency and strong robustness.