What are the differences between centralized learning (in monolithic systems) and decentralized learning (in multi-agent systems)?

Looking at the advantages of collective learning.

A blog from GoodAI CEO Marek Rosa.

Summary

• Question: Why are we studying social learning in multi-agent systems?
• Answer: Multi-agent systems are made up of agents where each has its own objective. We believe that this leads to learning dynamics that are impossible in centralized systems.

This blog post is one of my personal takeaways from the Badger Seminar 2021.

Definitions:

• Monolithic system (centralized): has one objective that is shared by all parts of the system. Let’s assume a homogeneous network and learning via a back-prop algorithm. This is what most deep learning is about.
• Multi-agent system (decentralized): each part of the system has its own objective, therefore its learning resembles social learning in an evolving population of agents. We are not assuming back-prop learning.

Example of collective learning in a multi-agent system

An ant colony simulation where ants are stateless policies, moving randomly until they discover food, then start laying down pheromones (memories). If other ants discover pheromone trails, they start following them and adding more pheromones.

Summary: We can see that individual ants don’t know where the food is, but the collective of ants know where the food is and how to gather it.

Monolithic vs Multi-agent

The following table breaks down my intuitions. The hypotheses need to be verified experimentally.

Conclusion

Emergent learning is a type of learning that can only happen on the collective level when multiple individuals are interacting and qualitatively new behaviors can emerge.

The main difference between monolithic and multi-agent systems: the latter is made from agents where each has local objectives (not a global objective) and this leads to interesting evolutionary dynamics.

Questions for reflection

Key points of social learning:

• External information storage — Is it the key for better collective learning? The storage can be cumulative and bigger than the memory of an individual agent.
• Multiple feedback mechanisms — A social system can have many adaptive feedback mechanisms, will they scale better than a centralized one in monolithic systems?
• Efficiency threshold — Is there a threshold at which social systems become more efficient than monolithic systems?

Identified benefits of social systems:

• Better scaling — does a modular / hierarchical system with mostly local communication scale better than a monolithic system?
• Replication of skills — discovered skills can be replicated to other parts of the society, whereas in a monolithic system it needs to be rediscovered. Are there some counterexamples?
• Open-ended learning — due to not having a single fixed feedback mechanism and the learning diverges, the social systems are more suitable for open-ended learning.

General:

• Does a monolithic system learn faster than a multi-agent system?
• Is there a limit where a monolithic system won’t be sufficient anymore and you need to switch to multi-agent learning? Can we get open-ended learning inside a monolithic system?
• Can we simulate a multi-agent system on a monolithic system ? For example, a multi-agent system being simulated by a monolithic interpreter.

Thank you for reading this blog!

Best,
Marek Rosa
CEO, CTO, Founder at GoodAI

Originally published at https://www.goodai.com on January 18, 2022.