Multiagent Systems
- [1] arXiv:2405.19570 [pdf, ps, html, other]
-
Title: Distributed Online Planning for Min-Max Problems in Networked Markov GamesComments: Accepted to appear in the IEEE Robotics and Automation LettersSubjects: Multiagent Systems (cs.MA); Robotics (cs.RO)
Min-max problems are important in multi-agent sequential decision-making because they improve the performance of the worst-performing agent in the network. However, solving the multi-agent min-max problem is challenging. We propose a modular, distributed, online planning-based algorithm that is able to approximate the solution of the min-max objective in networked Markov games, assuming that the agents communicate within a network topology and the transition and reward functions are neighborhood-dependent. This set-up is encountered in the multi-robot setting. Our method consists of two phases at every planning step. In the first phase, each agent obtains sample returns based on its local reward function, by performing online planning. Using the samples from online planning, each agent constructs a concave approximation of its underlying local return as a function of only the action of its neighborhood at the next planning step. In the second phase, the agents deploy a distributed optimization framework that converges to the optimal immediate next action for each agent, based on the function approximations of the first phase. We demonstrate our algorithm's performance through formation control simulations.
- [2] arXiv:2405.19895 [pdf, ps, html, other]
-
Title: Dispersion of personal spacesComments: This is the preprint of the paper presented at the 6th International Conference on the Dynamics of Information Systems (DIS 2023), September 3-6, 2023, Prague, Czech Republic. The paper was accepted for publication in conference proceedings in Lecture Notes in Computer ScienceSubjects: Multiagent Systems (cs.MA)
There are many entities that disseminate in the physical space - information, gossip, mood, innovation etc. Personal spaces are also entities that disperse and interplay. In this work we study the emergence of configurations formed by participants when choosing a place to sit in a rectangular auditorium. Based on experimental questionnaire data we design several models and assess their relevancy to a real time-lapse footage of lecture hall being filled up. The main focus is to compare the evolution of entropy of occupied seat configurations in time. Even though the process of choosing a seat is complex and could depend on various properties of participants or environment, some of the developed models can capture at least basic essence of the real processes. After introducing the problem of seat selection and related results in close research areas, we introduce preliminary collected data and build models of seat selection based on them. We compare the resulting models to the real observational data and discuss areas of future research directions.
- [3] arXiv:2405.19998 [pdf, ps, html, other]
-
Title: LAGMA: LAtent Goal-guided Multi-Agent Reinforcement LearningComments: Accepted at ICML 2024Subjects: Multiagent Systems (cs.MA)
In cooperative multi-agent reinforcement learning (MARL), agents collaborate to achieve common goals, such as defeating enemies and scoring a goal. However, learning goal-reaching paths toward such a semantic goal takes a considerable amount of time in complex tasks and the trained model often fails to find such paths. To address this, we present LAtent Goal-guided Multi-Agent reinforcement learning (LAGMA), which generates a goal-reaching trajectory in latent space and provides a latent goal-guided incentive to transitions toward this reference trajectory. LAGMA consists of three major components: (a) quantized latent space constructed via a modified VQ-VAE for efficient sample utilization, (b) goal-reaching trajectory generation via extended VQ codebook, and (c) latent goal-guided intrinsic reward generation to encourage transitions towards the sampled goal-reaching path. The proposed method is evaluated by StarCraft II with both dense and sparse reward settings and Google Research Football. Empirical results show further performance improvement over state-of-the-art baselines.
- [4] arXiv:2405.20018 [pdf, ps, other]
-
Title: Safe Multi-agent Reinforcement Learning with Natural Language ConstraintsComments: 23 pages, 6 figuresSubjects: Multiagent Systems (cs.MA); Computation and Language (cs.CL); Machine Learning (cs.LG)
The role of natural language constraints in Safe Multi-agent Reinforcement Learning (MARL) is crucial, yet often overlooked. While Safe MARL has vast potential, especially in fields like robotics and autonomous vehicles, its full potential is limited by the need to define constraints in pre-designed mathematical terms, which requires extensive domain expertise and reinforcement learning knowledge, hindering its broader adoption. To address this limitation and make Safe MARL more accessible and adaptable, we propose a novel approach named Safe Multi-agent Reinforcement Learning with Natural Language constraints (SMALL). Our method leverages fine-tuned language models to interpret and process free-form textual constraints, converting them into semantic embeddings that capture the essence of prohibited states and behaviours. These embeddings are then integrated into the multi-agent policy learning process, enabling agents to learn policies that minimize constraint violations while optimizing rewards. To evaluate the effectiveness of SMALL, we introduce the LaMaSafe, a multi-task benchmark designed to assess the performance of multiple agents in adhering to natural language constraints. Empirical evaluations across various environments demonstrate that SMALL achieves comparable rewards and significantly fewer constraint violations, highlighting its effectiveness in understanding and enforcing natural language constraints.
- [5] arXiv:2405.20232 [pdf, ps, html, other]
-
Title: Distributed maze exploration using multiple agents and optimal goal assignmentComments: 11 pages, 9 figuresSubjects: Multiagent Systems (cs.MA)
Robotic exploration has long captivated researchers aiming to map complex environments efficiently. Techniques such as potential fields and frontier exploration have traditionally been employed in this pursuit, primarily focusing on solitary agents. Recent advancements have shifted towards optimizing exploration efficiency through multiagent systems. However, many existing approaches overlook critical real-world factors, such as broadcast range limitations, communication costs, and coverage overlap. This paper addresses these gaps by proposing a distributed maze exploration strategy (CU-LVP) that assumes constrained broadcast ranges and utilizes Voronoi diagrams for better area partitioning. By adapting traditional multiagent methods to distributed environments with limited broadcast ranges, this study evaluates their performance across diverse maze topologies, demonstrating the efficacy and practical applicability of the proposed method. The code and experimental results supporting this study are available in the following repository: this https URL.
New submissions for Friday, 31 May 2024 (showing 5 of 5 entries )
- [6] arXiv:2405.20085 (cross-list from cs.LG) [pdf, ps, html, other]
-
Title: Soft Partitioning of Latent Space for Semantic Channel EqualizationSubjects: Machine Learning (cs.LG); Information Theory (cs.IT); Multiagent Systems (cs.MA)
Semantic channel equalization has emerged as a solution to address language mismatch in multi-user semantic communications. This approach aims to align the latent spaces of an encoder and a decoder which were not jointly trained and it relies on a partition of the semantic (latent) space into atoms based on the the semantic meaning. In this work we explore the role of the semantic space partition in scenarios where the task structure involves a one-to-many mapping between the semantic space and the action space. In such scenarios, partitioning based on hard inference results results in loss of information which degrades the equalization performance. We propose a soft criterion to derive the atoms of the partition which leverages the soft decoder's output and offers a more comprehensive understanding of the semantic space's structure. Through empirical validation, we demonstrate that soft partitioning yields a more descriptive and regular partition of the space, consequently enhancing the performance of the equalization algorithm.
Cross submissions for Friday, 31 May 2024 (showing 1 of 1 entries )
- [7] arXiv:2205.12880 (replaced) [pdf, ps, html, other]
-
Title: Trust-based Consensus in Multi-Agent Reinforcement Learning SystemsComments: Accepted for publication in proceedings of the first Reinforcement Learning Conference (RLC 2024)Subjects: Multiagent Systems (cs.MA); Artificial Intelligence (cs.AI); Machine Learning (cs.LG)
An often neglected issue in multi-agent reinforcement learning (MARL) is the potential presence of unreliable agents in the environment whose deviations from expected behavior can prevent a system from accomplishing its intended tasks. In particular, consensus is a fundamental underpinning problem of cooperative distributed multi-agent systems. Consensus requires different agents, situated in a decentralized communication network, to reach an agreement out of a set of initial proposals that they put forward. Learning-based agents should adopt a protocol that allows them to reach consensus despite having one or more unreliable agents in the system. This paper investigates the problem of unreliable agents in MARL, considering consensus as a case study. Echoing established results in the distributed systems literature, our experiments show that even a moderate fraction of such agents can greatly impact the ability of reaching consensus in a networked environment. We propose Reinforcement Learning-based Trusted Consensus (RLTC), a decentralized trust mechanism, in which agents can independently decide which neighbors to communicate with. We empirically demonstrate that our trust mechanism is able to handle unreliable agents effectively, as evidenced by higher consensus success rates.
- [8] arXiv:2310.01945 (replaced) [pdf, ps, html, other]
-
Title: Homotopy-Aware Multi-Agent Path Planning in PlaneComments: 12 pages with 4 pages of references and appendices, 9 figuresSubjects: Multiagent Systems (cs.MA); Computational Geometry (cs.CG)
We propose an efficient framework using the Dynnikov coordinates for homotopy-aware multi-agent path planning in the plane. We developed a method to generate multiple homotopically distinct solutions of multi-agent path planning problem in the plane by combining our framework with revised prioritized planning and proved its completeness in the grid world under specific assumptions. Experimentally, we demonstrated the scalability of our method for the number of agents. We also confirmed experimentally that homotopy-aware planning contributes to avoiding locally optimal solutions when searching for low-cost trajectories for a swarm of agents in a continuous environment.