An ETA-Based Tactical Conflict Resolution Method for Air Logistics Transportation

  1. Li, Chenglong 1
  2. Gu, Wenyong 1
  3. Zheng, Yuan 2
  4. Huang, Longyang 1
  5. Zhang, Xuejun 3
  6. Tavares Calafate, Carlos
  7. González Aguilera, Diego 4
  1. 1 College of Air Traffic Management, Civil Aviation Flight University of China, Guanghan 618307, China
  2. 2 School of Computer Science, Civil Aviation Flight University of China, Guanghan 618307, China
  3. 3 School of Electronic and Information Engineering, Beihang University, Beijing 100191, China
  4. 4 Universidad de Salamanca
    info

    Universidad de Salamanca

    Salamanca, España

    ROR https://ror.org/02f40zc51

Mostrar afiliaciones +
Revista:
Drones

ISSN: 2504-446X

Año de publicación: 2023

Volumen: 7

Número: 5

Páginas: 334

Tipo: Artículo

DOI: 10.3390/DRONES7050334 GOOGLE SCHOLAR lock_openAcceso abierto editor

Otras publicaciones en: Drones

Resumen

Air logistics transportation has become one of the most promising markets for the civil drone industry. However, the large flow, high density, and complex environmental characteristics of urban scenes make tactical conflict resolution very challenging. Existing conflict resolution methods are limited by insufficient collision avoidance success rates when considering non-cooperative targets and fail to take the temporal constraints of the pre-defined 4D trajectory into consideration. In this paper, a novel reinforcement learning-based tactical conflict resolution method for air logistics transportation is designed by reconstructing the state space following the risk sectors concept and through the use of a novel Estimated Time of Arrival (ETA)-based temporal reward setting. Our contributions allow a drone to integrate the temporal constraints of the 4D trajectory pre-defined in the strategic phase. As a consequence, the drone can successfully avoid non-cooperative targets while greatly reducing the occurrence of secondary conflicts, as demonstrated by the numerical simulation results.

€ Ver financiación

Información de financiación

Financiadores

Referencias bibliográficas

  • (2022, November 08). Global Drone Delivery Market—Analysis and Forecast, 2023 to 2030. Available online: https://www.asdreports.com/market-research-report-575426/global-drone-delivery-market-analysis-forecast.
  • Dahle, O.H., Rydberg, J., Dullweber, M., Peinecke, N., and Bechina, A.A.A. (2022, January 21–24). A proposal for a common metric for drone traffic density. Proceedings of the 2022 International Conference on Unmanned Aircraft Systems (ICUAS), Dubrovnik, Croatia.
  • Bradford, S., and Kopardekar, P. (2021). FAA/NASA UAS Traffic Management Pilot Program (UPP) UPP Phase 2 Final Report, FAA/NASA Unmanned Aerial Systems Traffic Management Pilot Program Industry Workshop.
  • Mohamed Salleh, M.F.B., and Low, K.H. (2017). AIAA Information Systems-AIAA Infotech@ Aerospace, American Institute of Aeronautics and Astronautics, Inc.
  • Arafat, (2022), IEEE Internet Things J., 9, pp. 21751, 10.1109/JIOT.2022.3182750
  • Huang, (2020), IEEE Trans. Intell. Transp. Syst., 22, pp. 4941, 10.1109/TITS.2020.2983491
  • Khan, A. (2021). Risk Assessment, Prediction, and Avoidance of Collision in Autonomous Drones. arXiv.
  • Siqi, (2018), Chin. J. Aeronaut., 31, pp. 1579, 10.1016/j.cja.2018.04.017
  • Peinecke, N., and Kuenz, A. (2017, January 17–21). Deconflicting the urban drone airspace. Proceedings of the 2017 IEEE/AIAA 36th Digital Avionics Systems Conference (DASC), St. Petersburg, FL, USA.
  • Chen, T., Zhang, G., Hu, X., and Xiao, J. (June, January 31). Unmanned aerial vehicle route planning method based on a star algorithm. Proceedings of the 2018 13th IEEE conference on industrial electronics and applications (ICIEA), Wuhan, China.
  • Maini, P., and Sujit, P.B. (2016, January 7–10). Path planning for a uav with kinematic constraints in the presence of polygonal obstacles. Proceedings of the 2016 international conference on unmanned aircraft systems (ICUAS), Arlington, VA, USA.
  • Abhishek, (2020), SN Appl. Sci., 2, pp. 1805, 10.1007/s42452-020-03498-0
  • Khatib, (1986), Int. J. Robot. Res., 5, pp. 90, 10.1177/027836498600500106
  • Wang, (2022), IEEE Trans. Robot., 38, pp. 3259, 10.1109/TRO.2022.3160022
  • Howard, (2008), J. Field Robot., 25, pp. 325, 10.1002/rob.20244
  • Mankiewicz, R.H. (2005). Global Air Traffic Management Operational Concept, ICAO.
  • Florence, H.O., and HO, F. (2020). Scalable Conflict Detection and Resolution Methods for Safe Unmanned Aircraft Systems Traffic Management. [Ph.D. Thesis, The Graduate University for Advanced Studies].
  • Gardi, A., Lim, Y., Kistan, T., and Sabatini, R. (2016, January 25–30). Planning and negotiation of optimised 4D trajectories in strategic and tactical re-routing operations. Proceedings of the 30th Congress of the International Council of the Aeronautical Sciences, ICAS, Daejeon, Republic of Korea.
  • Qian, (2017), Transp. Res. Part C Emerg. Technol., 81, pp. 18, 10.1016/j.trc.2017.05.008
  • Chaimatanan, S., Delahaye, D., and Mongeau, M. (2015, January 7–10). Aircraft 4D trajectories planning under uncertainties. Proceedings of the 2015 IEEE Symposium Series on Computational Intelligence, Cape Town, South Africa.
  • (2022, November 15). FAA, Eurocontrol Pursue Initial Trajectory-Based Operations Now, Full Implementation Later. Available online: https://interactive.aviationtoday.com/avionicsmagazine/july-august-2022/faa-eurocontrol-pursue-initial-trajectory-based-operations-now-full-implementation-later/.
  • (2022, November 15). 4D Skyways Improving Trajectory Management for European Air Transport. Available online: https://www.eurocontrol.int/project/4d-skyways.
  • Park, J.W., Oh, H.D., and Tahk, M.J. (2008, January 20–22). UAV collision avoidance based on geometric approach. Proceedings of the 2008 SICE Annual Conference, Chofu, Japan.
  • Strobel, A., and Schwarzbach, M. (2014, January 27–30). Cooperative sense and avoid: Implementation in simulation and real world for small unmanned aerial vehicles. Proceedings of the 2014 International Conference on Unmanned Aircraft Systems (ICUAS), Orlando, FL, USA.
  • Marchidan, (2020), J. Guid. Control. Dyn., 43, pp. 96, 10.2514/1.G004446
  • Roadmap, A.I. (2020). A Human-Centric Approach to AI in Aviation.
  • Brat, G. (2021, January 18). Are we ready for the first easa guidance on the use of ml in aviation. Proceedings of the SAE G34 Meeting, Online.
  • Yang, X., and Wei, P. (2018, January 26–29). Autonomous on-demand free flight operations in urban air mobility using Monte Carlo tree search. Proceedings of the International Conference on Research in Air Transportation (ICRAT), Barcelona, Spain.
  • Chen, Y., González-Prelcic, N., and Heath, R.W. (2020, January 21–24). Collision-free UAV navigation with a monocular camera using deep reinforcement learning. Proceedings of the 2020 IEEE 30th International Workshop on Machine Learning for Signal Processing (MLSP), Espoo, Finland.
  • Cetin, E., Barrado, C., Munoz, G., Macias, M., and Pastor, E. (2019, January 8–12). Drone navigation and avoidance of obstacles through deep reinforcement learning. Proceedings of the 2019 IEEE/AIAA 38th Digital Avionics Systems Conference (DASC), San Diego, CA, USA.
  • Wan, K., Gao, X., Hu, Z., and Wu, G. (2020). Robust motion control for UAV in dynamic uncertain environments using deep reinforcement learning. Remote Sens., 12.
  • Monk, K.J., Rorie, C., Smith, C., Keeler, J., Sadler, G., and Brandt, S.L. (2020, January 10–13). Unmanned Aircraft Systems (UAS) Integration in the National Airspace System (NAS) Project: ACAS-Xu Run 5 Human-In-The-Loop Sim SC-147 Results Outbrief. Proceedings of the RTCA Special Committee 147 Face-to-Face Meeting, Phoenix, AZ, USA. No. ARC-E-DAA-TN73281.
  • (2022). Standard Specification for UAS Traffic Management (UTM) UAS Service Supplier (USS) Interoperability (Standard No. ASTM F3548-21).
  • (2023, May 05). Fully Automated Instant Delivery Network. Antwork Technology. Available online: https://www.antwork.link.
  • (2023, May 05). TCAS Event Recorder. Honeywell. Available online: https://aerospace.honeywell.com/us/en/about-us/news/2021/09/tcas-event-recorder.
  • Kopardekar, P., Rios, J., Prevot, T., Johnson, M., Jung, J., and Robinson, J.E. (2016). Unmanned aircraft system traffic management (UTM) concept of operations, AIAA Aviation and Aeronautics Forum (Aviation 2016), No. ARC-E-DAA-TN32838.
  • Johnson, M. (2022, June 05). Unmanned Aircraft Systems (UAS) Traffic Management (UTM) Project, Available online: https://nari.arc.nasa.gov/sites/default/files/attachments/UTM%20TIM-Marcus%20Johnson.pdf.
  • ACSL (2023, May 05). Made-in-Japan Drone for Logistics AirTruck. ACSL. 22 December 2022. Available online: https://product.acsl.co.jp/en/wp-content/uploads/2022/12/220627_AirTruck_en_trim.pdf.
  • Lu, P. (2023, May 05). Overview of China’s Logistics UAV Industry in 2020. LeadLeo. April 2020. Available online: https://pdf.dfcfw.com/pdf/H3_AP202101071448279174_1.pdf.
  • 36 Kr Venture Capital Research Institute (2023, May 05). Unmanned Distribution Field Research Report. 36 Kr. 26 February 2020. Available online: http://pdf.dfcfw.com/pdf/H3_AP202003041375814837_1.pdf.
  • Huang, (2022), J. Civ. Aviat., 6, pp. 50
  • Minsky, (1961), Proc. IRE, 49, pp. 8, 10.1109/JRPROC.1961.287775
  • Mnih, (2015), Nature, 518, pp. 529, 10.1038/nature14236
  • Van Hasselt, H., Guez, A., and Silver, D. (2016, January 12–17). Deep reinforcement learning with double q-learning. Proceedings of the AAAI Conference on Artificial Intelligence, Phoenix, AZ, USA.
  • Wang, Z., Schaul, T., Hessel, M., Hasselt, H., Lanctot, M., and Freitas, N. (2016, January 20–22). Dueling network architectures for deep reinforcement learning. Proceedings of the International Conference on Machine Learning, PMLR, New York, NY, USA.
  • (2022, November 18). ICAO Model UAS Regulations. Available online: https://www.icao.int/safety/UA/Pages/ICAO-Model-UAS-Regulations.aspx.
  • Mo, S., Pei, X., and Chen, Z. (2019, January 21–22). Decision-making for oncoming traffic overtaking scenario using double DQN. Proceedings of the 2019 3rd Conference on Vehicle Control and Intelligence (CVCI), Hefei, China.
  • Fang, (2019), IOP Conf. Ser. Mater. Sci. Eng., 612, pp. 052039, 10.1088/1757-899X/612/5/052039
  • Han, (2020), IEEE Access, 8, pp. 186474, 10.1109/ACCESS.2020.3029868
  • Sui, Z., Pu, Z., Yi, J., and Xiong, T. (2019, January 14–19). Formation control with collision avoidance through deep reinforcement learning. Proceedings of the 2019 International Joint Conference on Neural Networks (IJCNN), Budapest, Hungary.
  • Radio Technical Commission for Aeronautics (US) (1983). Minimum Operational Performance Standards for Traffic Alert and Collision Avoidance System (TCAS) Airborne Equipment, Radio Technical Commission for Aeronautics.
  • (2023, January 18). Indian Defence Review. Aviation: The Future Is Unmanned. Available online: http://www.indiandefencereview.com/news/aviation-the-future-is-unmanned/2/.