[Multi-agent][feature request] Acme multi-agent support

[DanielWicz]

I'm preparing a project which uses RL and trying to choose an appropriate framework for this. In the near future, my project will require a multi-agent approach and there is a question:

Does the ACME support a multi-agent approach?
If not (which based on my research is the case), it is going to be supported in near future, e.g. this year?

[fastturtle]

Hi @DanielWicz, there is nothing about Acme that would stop you from doing multi-agent research. We don't currently have any examples (or utilities) for doing so, however would be interested in seeing anything you come up with :)

[DanielWicz]

Hi @DanielWicz, there is nothing about Acme that would stop you from doing multi-agent research.
Thanks for an answer.
So the ACME's env does not restrict you to a single-agent approach?

We don't currently have any examples (or utilities) for doing so, however would be interested in seeing anything you come up with :)
If I'll set up a notebook, then I'll notify.

[lanctot]

Hi @DanielWicz, you might be interested in this thread too: #80 (tagging @jhtschultz as well).

To my understanding, the EnvironmentLoop we'd need to write to support OpenSpiel games / environments would look similar to one for general multiagent experiments, or maybe we would write a more general one and special-case it for OpenSpiel games.

[IanRDavies]

@DanielWicz There is a recently released multi-agent RL library built upon Acme called Mava which may be of interest.

[nikolamomchev]

I feel this was answered sufficiently well, so I'm closing it. I'd just add that we've recently added support for distributed setups with LaunchPad in Acme which would open some interesting possibilities for developing multi-agent algorithms. We have also successfully used Acme for building MARL algorithms. We are trying to open-source some of those in the near future.

[BharathRajM]

Hi team

Checking in if you can share the examples of acme for benchmarking MARL algorithms like MAPPO, HATRPO, QMIX etc.,?