A Fault-Tolerant Multi-Agent Reinforcement Learning Framework for Unmanned Aerial Vehicles–Unmanned Ground Vehicle Coverage Path Planning

Bibliographic References

• Zubrin, R. (2008). How to Live on Mars: A Trusty Guidebook to Surviving and Thriving on the Red Planet, Crown.
• Islam, M.R., Chowdhury, F.H., Rezwan, S., Ishaque, M.J., Akanda, J.U., Tuhel, A.S., and Riddhe, B.B. (2017, January 3–5). Novel design and performance analysis of a Mars exploration robot: Mars rover mongol pothik. Proceedings of the 2017 Third International Conference on Research in Computational Intelligence and Communication Networks (ICRCICN), Kolkata, India.
• (2010), Icarus, 208, pp. 165, 10.1016/j.icarus.2010.01.006
• AlandiHallaj, (2017), Acta Astronaut., 140, pp. 225, 10.1016/j.actaastro.2017.08.019
• AlandiHallaj, (2019), Acta Astronaut., 161, pp. 531, 10.1016/j.actaastro.2019.04.009
• Alandihallaj, (2023), Adv. Space Res., 71, pp. 645, 10.1016/j.asr.2022.06.067
• Dinelli, C., Racette, J., Escarcega, M., Lotero, S., Gordon, J., Montoya, J., Dunaway, C., Androulakis, V., Khaniani, H., and Shao, S. (2023). Configurations and applications of multi-agent hybrid drone/unmanned ground vehicle for underground environments: A review. Drones, 7.
• Crisp, (2003), J. Geophys. Res. Planets, 108, pp. 8061, 10.1029/2002JE002038
• Lamarre, O., and Kelly, J. (2018). Overcoming the challenges of solar rover autonomy: Enabling long-duration planetary navigation. arXiv.
• Arvidson, R.E., Bell III, J.F., Bellutta, P., Cabrol, N.A., Catalano, J., Cohen, J., Crumpler, L.S., Des Marais, D., Estlin, T., and Farrand, W. (2010). Spirit Mars Rover Mission: Overview and selected results from the northern Home Plate Winter Haven to the side of Scamander crater. J. Geophys. Res. Planets, 115.
• Farley, (2020), Space Sci. Rev., 216, pp. 1, 10.1007/s11214-020-00762-y
• Ramezani, M., Atashgah, M., Sanchez-Lopez, J.L., and Voos, H. (2024, January 4–7). Human-Centric Aware UAV Trajectory Planning in Search and Rescue Missions Employing Multi-Objective Reinforcement Learning with AHP and Similarity-Based Experience Replay. Proceedings of the 2024 International Conference on Unmanned Aircraft Systems (ICUAS), Chania, Crete.
• Mohsan, (2023), Intell. Serv. Robot., 16, pp. 109
• Muchiri, G., and Kimathi, S. (2016, January 4–6). A review of applications and potential applications of UAV. Proceedings of the Annual Conference on Sustainable Research and Innovation Conference, Nairobi, Kenya.
• Serna, J.G., Vanegas, F., Gonzalez, F., and Flannery, D. (2020). A review of current approaches for UAV autonomous mission planning for Mars biosignatures detection. 2020 IEEE Aerospace Conference, IEEE.
• Rizk, (2019), ACM Comput. Surv. (CSUR), 52, pp. 1, 10.1145/3303848
• Xing, (2022), IEEE Internet Things J., 10, pp. 3548, 10.1109/JIOT.2022.3218491
• Asadi, (2020), Autom. Constr., 112, pp. 103068, 10.1016/j.autcon.2019.103068
• Emami, M.R., and Alandihallaj, M.A. (2022, January 16–24). Performance Enhancement of Fractionated Spacecraft for Earth Observation. Proceedings of the 4th COSPAR Scientific Assembly, Athens, Greece.
• Ohradzansky, M.T., Rush, E.R., Riley, D.G., Mills, A.B., Ahmad, S., McGuire, S., Biggie, H., Harlow, K., Miles, M.J., and Frew, E.W. (2021). Multi-agent autonomy: Advancements and challenges in subterranean exploration. arXiv.
• Bartolomei, (2023), IEEE Robot. Autom. Lett., 8, pp. 5576, 10.1109/LRA.2023.3296037
• Ropero, F., Muñoz, P., and R-Moreno, M.D. (2024, January 16–18). A Strategical Path Planner for UGV-UAV Cooperation in Mars Terrains. Proceedings of the International Conference on Innovative Techniques and Applications of Artificial Intelligence, Tokyo, Japan.
• Nowak, (2013), Int. Arch. Photogramm. Remote Sens. Spat. Inf. Sci., 40, pp. 299, 10.5194/isprsarchives-XL-1-W2-299-2013
• Alejo, (2023), IEEE Robot. Autom. Lett., 8, pp. 6475, 10.1109/LRA.2023.3301292
• Hu, (2022), Build. Environ., 221, pp. 109349, 10.1016/j.buildenv.2022.109349
• Cabreira, T.M., Brisolara, L.B., and Ferreira, P.R. (2019). Survey on coverage path planning with unmanned aerial vehicles. Drones, 3.
• Koktas, (2024), IEEE Aerosp. Electron. Syst. Mag., 39, pp. 216, 10.1109/MAES.2024.3396117
• Sasaki, (2020), IEEE Robot. Autom. Lett., 5, pp. 2123, 10.1109/LRA.2020.2970650
• Szklany, M., Cohen, A., and Boubin, J. (2024, January 4–7). Tsunami: Scalable, Fault Tolerant Coverage Path Planning for UAV Swarms. Proceedings of the 2024 International Conference on Unmanned Aircraft Systems (ICUAS), Chania, Crete.
• Tan, (2021), IEEE Access, 9, pp. 119310, 10.1109/ACCESS.2021.3108177
• Ramezani, M., Habibi, H., Sanchez-Lopez, J.L., and Voos, H. (2023, January 6–9). UAV path planning employing MPC-reinforcement learning method considering collision avoidance. Proceedings of the 2023 International Conference on Unmanned Aircraft Systems (ICUAS), Warsaw, Poland.
• Ramezani, M., and Amiri Atashgah, M. (2024). Energy-Aware Hierarchical Reinforcement Learning Based on the Predictive Energy Consumption Algorithm for Search and Rescue Aerial Robots in Unknown Environments. Drones, 8.
• Ramezani, M., Alandihallaj, M.A., and Hein, A.M. (2024, January 13–17). PPO-Based Dynamic Control of Uncertain Floating Platforms in Zero-G Environment. Proceedings of the 2024 IEEE International Conference on Robotics and Automation (ICRA), Yokohama, Janpan.
• Alandihallaj, M.A., Ramezani, M., and Hein, A.M. (2024, January 21–23). MBSE-Enhanced LSTM Framework for Satellite System Reliability and Failure Prediction. Proceedings of the 12th International Conference on Model-Based Software and Systems Engineering (MODELSWARD 2024), Rome, Italy.
• Kong, (2023), IEEE Access, 11, pp. 44051, 10.1109/ACCESS.2023.3273164
• Alandihallaj, (2022), Acta Astronaut., 191, pp. 451, 10.1016/j.actaastro.2021.11.026
• Zanol, R., Chiariotti, F., and Zanella, A. (2019, January 15–18). Drone mapping through multi-agent reinforcement learning. Proceedings of the 2019 IEEE Wireless Communications and Networking Conference (WCNC), Marrakech, Morocco.
• Zhao, Y., Zhou, C., Cao, J., Zhao, Y., Liu, S., Cheng, C., and Li, X. (2024). Multi-Scenario Combination Based on Multi-Agent Reinforcement Learning to Optimize the Advertising Recommendation System. arXiv.
• Alandihallaj, (2024), Acta Astronaut., 223, pp. 58, 10.1016/j.actaastro.2024.06.050
• Alandihallaj, (2022), Acta Astronaut., 196, pp. 157, 10.1016/j.actaastro.2022.04.014
• Sharma, P.K., Fernandez, R., Zaroukian, E., Dorothy, M., Basak, A., and Asher, D.E. (2021). Survey of recent multi-agent reinforcement learning algorithms utilizing centralized training. Artificial Intelligence and Machine Learning for Multi-Domain Operations Applications III, SPIE.
• Leottau, (2018), Artif. Intell., 256, pp. 130, 10.1016/j.artint.2017.12.001
• Zhou, Y., Liu, S., Qing, Y., Chen, K., Zheng, T., Huang, Y., Song, J., and Song, M. (2023). Is centralized training with decentralized execution framework centralized enough for MARL?. arXiv.
• Oroojlooy, (2020), Adv. Neural Inf. Process. Syst., 33, pp. 4079
• Ramezani, M., Alandihallaj, M.A., Sanchez-Lopez, J.L., and Hein, A. (2023). Safe Hierarchical Reinforcement Learning for CubeSat Task Scheduling Based on Energy Consumption. arXiv.
• Ramezani, M., Atashgah, M., Alandihallaj, M., and Hein, A. (2023, January 2–6). Reinforcement Learning for Planning and Task Coordination in a Swarm of CubeSats: Overcoming Processor Limitation Challenges. Proceedings of the International Astronautical Congress, Baku, Azerbaijan.
• Chen, (2020), Comput. Commun., 151, pp. 485, 10.1016/j.comcom.2019.12.054
• Stojicic, (2012), Int. Endod. J., 45, pp. 363, 10.1111/j.1365-2591.2011.01985.x
• Sutton, R.S. (2018). Reinforcement Learning: An Introduction, The MIT Press. A Bradford Book.
• Lv, (2019), IEEE Access, 7, pp. 67319, 10.1109/ACCESS.2019.2918703
• Littman, M.L. (1994). Markov games as a framework for multi-agent reinforcement learning. Machine Learning Proceedings 1994, Elsevier.
• Puterman, (1990), Handb. Oper. Res. Manag. Sci., 2, pp. 331
• Ge, (2021), IEEE Trans. Intell. Transp. Syst., 23, pp. 12572, 10.1109/TITS.2021.3115240
• Wang, (2024), Swarm Evol. Comput., 85, pp. 101475, 10.1016/j.swevo.2024.101475
• Taud, H., and Mas, J.-F. (2017). Multilayer perceptron (MLP). Geomatic Approaches for Modeling Land Change Scenarios, Springer.
• Ketkar, N., and Ketkar, N. (2017). Stochastic gradient descent. Deep Learning with Python, Apress.
• Dann, C., Mansour, Y., Mohri, M., Sekhari, A., and Sridharan, K. (2022, January 17–23). Guarantees for epsilon-greedy reinforcement learning with function approximation. Proceedings of the International Conference on Machine Learning, Baltimore, MD, USA.
• Dong, (2024), Appl. Soft Comput., 151, pp. 111108, 10.1016/j.asoc.2023.111108