Not everyone knows - but @LocalAI_API has two ways of distributing load across nodes (if you are building a cluster of GPUs) 1) P2P Fedaration: this uses @libp2p behind the scenes - has a ledger and an in-memory state storage which is distributed across nodes. It uses Gossip protocol for co-ordination, suited for community use (very simple to setup) 2) full-fledged distributed mode: LocalAI uses workers that are connected via NATS and to the frontend. This allows to scale horizontally multiple frontends and to multiple worker machines. LocalAI orchestrates building, maintenance, of models and backends. LocalAI has an extensible backend system that allows to support ANY backend for inferencing. With 2) you get control, with 1) you get decentralization.

@mudler_it @LocalAI_API @libp2p Thank you for using @libp2p at @LocalAI_API. If you need any help, support, or have feature requests that could enable your project further, please reach out, great efforts here 🙌 Great to see the P2P federation using @libp2p + gossip for coordination-first systems. Neat :)

@manusheel @mudler_it @libp2p 🫶 @libp2p