A Multimodal Image Registration Method for UAV Visual Navigation Based on Feature Fusion and Transformers

  1. He, Ruofei 1
  2. Long, Shuangxing 3
  3. Sun, Wei 3
  4. Liu, Hongjuan 4
  5. González-Aguilera, Diego ed. lit. 2
  1. 1 365th Research Institute, Northwestern Polytechnical University, Xi’an 710072, China
  2. 2 Universidad de Salamanca
    info

    Universidad de Salamanca

    Salamanca, España

    ROR https://ror.org/02f40zc51

  3. 3 Xidian University
    info

    Xidian University

    Xi'an, China

    ROR https://ror.org/05s92vm98

  4. 4 Xi’an ASN Technology Group Co., Ltd., Xi’an 710065, China
Show affiliations +
Journal:
Drones

ISSN: 2504-446X

Year of publication: 2024

Volume: 8

Issue: 11

Pages: 651

Type: Article

DOI: 10.3390/DRONES8110651 GOOGLE SCHOLAR lock_openOpen access editor

More publications in: Drones

€ View funding

Funding information

Funders

  • National Natural Science Foundation of China
    • 62173330
    • 62371375
  • Shaanxi Key R&D Plan Key Industry Innovation Chain Project
    • 2022ZDLGY03-01
  • China College Innovation Fund of Production, Education, and Research
    • 2021ZYAO8004
  • Xi’an Science and Technology Plan Project
    • 2022JH-RGZN-0039

Bibliographic References

  • Flusser, (2003), Image Vis. Comput., 21, pp. 977, 10.1016/S0262-8856(03)00137-9
  • Le Moigne, J., Netanyahu, N.S., and Eastman, R.D. (2011). Image Registration for Remote Sensing: Survey of Image Registration Methods, Cambridge University Press.
  • Sui, (2015), IEEE Trans. Geosci. Remote Sens., 53, pp. 6058, 10.1109/TGRS.2015.2431498
  • Chen, (2022), Med. Image Anal., 82, pp. 102615, 10.1016/j.media.2022.102615
  • Zhang, X., Leng, C., Hong, Y., Pei, Z., Cheng, I., and Basu, A. (2021). Multimodal Remote Sensing Image Registration Methods and Advancements: A Survey. Remote Sens., 13.
  • Detone, D., Malisiewicz, T., and Rabinovich, A. (2018, January 18–23). SuperPoint: Self-Supervised Interest Point Detection and Description. Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), Salt Lake City, UT, USA.
  • Canny, J. (1987). A computational approach to edge detection. Read. Comput. Vis., 184–203.
  • Lehureau, G., Tupin, F., Tison, C., Oller, G., and Petit, D. (2008, January 2–5). Registration of metric resolution SAR and Optical images in urban areas; proceedings of the Synthetic Aperture Radar (EUSAR). Proceedings of the 2008 7th European Conference on Synthetic Aperture Radar, Friedrichshafen, Germany.
  • Beymer, D. (1996, January 18–20). Feature correspondence by interleaving shape and texture computations. Proceedings of the CVPR IEEE Computer Society Conference on Computer Vision and Pattern Recognition, San Francisco, CA, USA.
  • Vaswani, A. (2017, January 4–9). Attention is all you need. Advances in Neural Information Processing Systems. Proceedings of the 31st Conference on Neural Information Processing Systems (NIPS 2017), Long Beach, CA, USA.
  • He, K., Zhang, X., Ren, S., and Sun, J. (2016, January 27–30). Deep Residual Learning for Image Recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA.
  • Lowe, (2004), Int. J. Comput. Vis., 60, pp. 91, 10.1023/B:VISI.0000029664.99615.94
  • Bay, H., Tuytelaars, T., and Gool, L.V. (2006, January 7–13). SURF: Speeded up robust features. Proceedings of the 9th European Conference on Computer Vision—Volume Part I, Graz, Austria.
  • Ma, (2016), IEEE Geosci. Remote Sens. Lett., 14, pp. 3, 10.1109/LGRS.2016.2600858
  • Wu, (2014), IEEE Geosci. Remote Sens. Letters, 12, pp. 43, 10.1109/LGRS.2014.2325970
  • Ye, (2017), IEEE Trans. Geosci. Remote Sens., 55, pp. 2941, 10.1109/TGRS.2017.2656380
  • Li, (2019), IEEE Trans. Image Process., 29, pp. 3296, 10.1109/TIP.2019.2959244
  • Zhu, (2023), IEEE Trans. Geosci. Remote. Sens., 61, pp. 1
  • Yi, K.M., Trulls, E., Lepetit, V., and Fua, P. (2016). LIFT: Learned Invariant Feature Transform, Springer International Publishing.
  • Ono, Y., Trulls, E., Fua, P., and Yi, K.M. (2018). LF-Net: Learning Local Features from Images, Curran Associates Inc.
  • Revaud, J., De Souza, C., Humenberger, M., and Weinzaepfel, P. (2019, January 8–14). R2D2: Repeatable and Reliable Detector and Descriptor. Proceedings of the Neural Information Processing Systems, Vancouver, BC, Canada.
  • Luo, Z., Zhou, L., Bai, X., Chen, H., Zhang, J., Yao, Y., Li, S., Fang, T., and Quan, L. (2020, January 13–19). ASLFeat: Learning Local Features of Accurate Shape and Localization. Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA.
  • Li, (2022), IEEE Trans. Geosci. Remote Sens., 60, pp. 1
  • Deng, (2023), IEEE Trans. Image Process., 32, pp. 591, 10.1109/TIP.2022.3231135
  • Gao, (2023), IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens., 16, pp. 10132, 10.1109/JSTARS.2023.3326959
  • Wang, X., Jabri, A., and Efros, A.A. (2019, January 16–17). Learning Correspondence from the Cycle-Consistency of Time. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, USA.
  • Ma, J., Zhao, J., Jiang, J., Zhou, H., and Guo, X. (2017, January 19–25). Locality Preserving Matching. Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence, Melbourne, Australia.
  • Li, Z., and Snavely, N. (2018, January 18–23). MegaDepth: Learning Single-View Depth Prediction from Internet Photos. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Salt Lake City, UT, USA.
  • Yao, (2022), IEEE Trans. Image Process., 31, pp. 2584, 10.1109/TIP.2022.3157450
  • Balntas, V., Lenc, K., Vedaldi, A., and Mikolajczyk, K. (2017, January 21–26). HPatches: A benchmark and evaluation of handcrafted andlearned local descriptors. Proceedings of the IEEE Conference on Computer Vision and PatternRecognition, Honolulu, HI, USA.
  • Fan, Z., Liu, Y., Liu, Y., Zhang, L., Zhang, J., Sun, Y., and Ai, H. (2022). 3MRS: An Effective Coarse-to-Fine Matching Method for Multimodal Remote Sensing Imagery. Remote Sens., 14.