Drone Control in AR: An Intuitive System for Single-Handed Gesture Control, Drone Tracking, and Contextualized Camera Feed Visualization in Augmented Reality

  1. Konstantoudakis, Konstantinos
  2. Christaki, Kyriaki
  3. Tsiakmakis, Dimitrios
  4. Sainidis, Dimitrios
  5. Albanis, Georgios
  6. Dimou, Anastasios
  7. Daras, Petros
  8. González Aguilera, Diego 1
  9. Rodríguez-Gonzálvez, Pablo
  1. 1 Universidad de Salamanca
    info

    Universidad de Salamanca

    Salamanca, España

    ROR https://ror.org/02f40zc51

Revista:
Drones

ISSN: 2504-446X

Año de publicación: 2022

Volumen: 6

Número: 2

Páginas: 43

Tipo: Artículo

DOI: 10.3390/DRONES6020043 GOOGLE SCHOLAR lock_openAcceso abierto editor

Otras publicaciones en: Drones

Información de financiación

Financiadores

Referencias bibliográficas

  • Loianno, (2017), IEEE Robot. Autom. Lett., 2, pp. 1740, 10.1109/LRA.2017.2699790
  • Shihavuddin, A., Chen, X., Fedorov, V., Nymark Christensen, A., Andre Brogaard Riis, N., Branner, K., Bjorholm Dahl, A., and Reinhold Paulsen, R. (2019). Wind turbine surface damage detection by deep learning aided drone inspection analysis. Energies, 12.
  • Wu, (2019), Remote Sens. Environ., 235, pp. 111456, 10.1016/j.rse.2019.111456
  • Hill, (2019), J. Archaeol. Sci. Rep., 24, pp. 80
  • Joyce, (2019), Mar. Freshw. Res., 70, pp. 952, 10.1071/MF17380
  • Karjalainen, K.D., Romell, A.E.S., Ratsamee, P., Yantac, A.E., Fjeld, M., and Obaid, M. (2017, January 17–20). Social drone companion for the home environment: A user-centric exploration. Proceedings of the 5th International Conference on Human Agent Interaction, Bielefeld, Germany.
  • Mishra, (2020), Comput. Commun., 156, pp. 1, 10.1016/j.comcom.2020.03.012
  • Burke, C., McWhirter, P.R., Veitch-Michaelis, J., McAree, O., Pointon, H.A., Wich, S., and Longmore, S. (2019). Requirements and Limitations of Thermal Drones for Effective Search and Rescue in Marine and Coastal Areas. Drones, 3.
  • Tezza, (2019), IEEE Access, 7, pp. 167438, 10.1109/ACCESS.2019.2953900
  • Suárez Fernández, R.A., Sanchez-Lopez, J.L., Sampedro, C., Bavle, H., Molina, M., and Campoy, P. (2016, January 7–10). Natural user interfaces for human-drone multi-modal interaction. Proceedings of the 2016 International Conference on Unmanned Aircraft Systems (ICUAS), Arlington, VA, USA.
  • Herrmann, (2018), i-com, 17, pp. 15, 10.1515/icom-2018-0001
  • Kleinschmidt, S.P., Wieghardt, C.S., and Wagner, B. (2017, January 26–28). Tracking Solutions for Mobile Robots: Evaluating Positional Tracking using Dual-axis Rotating Laser Sweeps. Proceedings of the ICINCO 2017, Madrid, Spain.
  • Islam, S., Ionescu, B., Gadea, C., and Ionescu, D. (, January 23–26). Indoor positional tracking using dual-axis rotating laser sweeps. Proceedings of the 2016 IEEE International Instrumentation and Measurement Technology Conference Proceedings, Taipei, Taiwan.
  • Arreola, L., De Oca, A.M., Flores, A., Sanchez, J., and Flores, G. (, January 12–15). Improvement in the UAV position estimation with low-cost GPS, INS and vision-based system: Application to a quadrotor UAV. Proceedings of the 2018 International Conference on Unmanned Aircraft Systems (ICUAS), Dallas, TX, USA.
  • Tsai, S.E., and Zhuang, S.H. (, January 27–29). Optical flow sensor integrated navigation system for quadrotor in GPS-denied environment. Proceedings of the 2016 International Conference on Robotics and Automation Engineering (ICRAE), Jeju, Korea.
  • Hong, Y., Lin, X., Zhuang, Y., and Zhao, Y. (July, January 29). Real-time pose estimation and motion control for a quadrotor uav. Proceedings of the 11th World Congress on Intelligent Control and Automation, Shenyang, China.
  • Hoff, (1996), Intelligent Robots and Computer Vision XV: Algorithms, Techniques, Active Vision, and Materials Handling, Volume 2904, pp. 538
  • Deshmukh, (2018), Int. Res. J. Eng. Technol., 5, pp. 287
  • Shreyas, E., Sheth, M.H. (2021, January 27–28). 3D Object Detection and Tracking Methods using Deep Learning for Computer Vision Applications. Proceedings of the 2021 International Conference on Recent Trends on Electronics, Information, Communication Technology (RTEICT), Bangalore, India.
  • Rambach, J., Deng, C., Pagani, A., and Stricker, D. (2018, January 16–20). Learning 6DoF Object Poses from Synthetic Single Channel Images. Proceedings of the 2018 IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct), Munich, Germany.
  • Li, (2020), Int. J. Digit. Earth, 13, pp. 727, 10.1080/17538947.2018.1564379
  • Yuan, (2019), IEEE Robot. Autom. Lett., 4, pp. 1343, 10.1109/LRA.2019.2895419
  • Albanis, G., Zioulis, N., Dimou, A., Zarpalas, D., and Daras, P. (2020). European Conference on Computer Vision, Springer.
  • Endsley, (1995), Hum. Factors, 37, pp. 32, 10.1518/001872095779049543
  • Silvagni, (2017), Geomat. Nat. Hazards Risk, 8, pp. 18, 10.1080/19475705.2016.1238852
  • Volckaert, (2018), Proceedings of the Knowledge and Systems Sciences: 19th International Symposium, KSS 2018, Volume 949, pp. 87, 10.1007/978-981-13-3149-7_7
  • Hong, (2015), Proc. Hum. Factors Ergon. Soc. Annu. Meet., 59, pp. 1722, 10.1177/1541931215591372
  • Rowen, (2019), IEEE Trans. Hum.-Mach. Syst., 49, pp. 652, 10.1109/THMS.2019.2944384
  • Lukosch, (2015), Comput. Support. Coop. Work (CSCW), 24, pp. 613, 10.1007/s10606-015-9235-4
  • Brejcha, J., Lukác, M., Chen, Z., DiVerdi, S., and Cadík, M. (2018, January 14–17). Immersive trip reports. Proceedings of the 31st Annual ACM Symposium on User Interface Software and Technology, Berlin, Germany.
  • Wang, (2007), IEEE Trans. Vis. Comput. Graph., 13, pp. 1568, 10.1109/TVCG.2007.70544
  • Konstantoudakis, K., Albanis, G., Christakis, E., Zioulis, N., Dimou, A., Zarpalas, D., and Daras, P. (2020, January 24–27). Single-Handed Gesture UAV Control for First Responders—A Usability and Performance User Study. Proceedings of the 17th International Conference on Information Systems for Crisis Response and Management (ISCRAM 2020), Blacksburg, VA, USA.
  • Peshkova, (2017), IEEE Pervasive Comput., 16, pp. 34, 10.1109/MPRV.2017.3
  • Camurri, A., and Volpe, G. (2004). Gesture-Based Communication in Human-Computer Interaction, Springer.
  • Sainidis, D., Tsiakmakis, D., Konstantoudakis, K., Albanis, G., Dimou, A., and Daras, P. (2021, January 23–26). Single-handed gesture UAV control and video feed AR visualization for first responders. Proceedings of the 18th International Conference on Information Systems for Crisis Response and Management (ISCRAM 2021), Blacksburg, VA, USA.
  • Zhang, F., Bazarevsky, V., Vakunov, A., Tkachenka, A., Sung, G., Chang, C.L., and Grundmann, M. (2020). Mediapipe hands: On-device real-time hand tracking. arXiv.
  • Anton, H., and Rorres, C. (2013). Elementary Linear Algebra: Applications Version, John Wiley & Sons.
  • Sun, K., Xiao, B., Liu, D., and Wang, J. (2019, January 15–20). Deep high-resolution representation learning for human pose estimation. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA.
  • Chen, B., Parra, A., Cao, J., Li, N., and Chin, T.J. (2020, January 14–19). End-to-end learnable geometric vision by backpropagating PnP optimization. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA.
  • Albanis, G.N., Zioulis, N., Chatzitofis, A., Dimou, A., Zarpalas, D., and Daras, P. (2021, December 20). On End-to-End 6DOF Object Pose Estimation and Robustness to Object Scale. ML Reproducibility Challenge 2020. Available online: https://openreview.net/forum?id=PCpGvUrwfQB.