A new large language model (LLM) named Collective-1 has been built using a novel approach: distributed AI training powered by GPUs scattered across the globe. This project, spearheaded by startups Flower AI and Vana, could significantly disrupt how artificial intelligence is typically developed.
Traditional LLMs, like those behind ChatGPT and Claude, require immense compute power centralized in elite data centers. These models are trained using proprietary datasets and high-performance GPUs connected via fiber-optic networks. This limits AI development to tech giants with access to vast financial and computing resources. In contrast, Collective-1’s development broke this mold, harnessing a decentralized network of computers and combining both private and public data.
Flower AI developed the distributed training technology, enabling model training to span hundreds of internet-connected machines. Vana contributed private datasets from platforms like X (formerly Twitter), Reddit, and Telegram—sources usually out of reach for traditional training methods.
Though smaller than today’s industry leaders, Collective-1’s 7 billion parameters represent a key milestone. Flower AI cofounder Nic Lane, also a Cambridge computer scientist, says the approach is scalable. He reveals that Flower AI is already training a 30-billion-parameter model and plans a 100-billion-parameter model later this year. Multimodal features, such as audio and image inputs, are also in development.
Democratizing AI: A New Power Shift
Distributed AI training could shift the balance of power in the AI sector. Today, only well-funded corporations or nations with high-end data centers can build top-tier models. This centralization sidelines smaller organizations, research labs, and countries without state-of-the-art infrastructure.
However, the distributed model offers an alternative. It allows multiple smaller players to pool their resources, effectively bypassing the need for single-location data centers. Universities or startups could team up and use their collective computing power to train competitive models. Lane highlights this benefit, noting that distributed training scales more elegantly and flexibly than centralized methods.
Helen Toner, an AI governance expert at the Center for Security and Emerging Technology, sees potential in this method. While she doubts it will surpass frontier models soon, she believes it could offer a fast-follower route that reshapes how AI models are built.
A New Tool and New Data Paradigms
Central to the success of Collective-1 is Photon, a tool developed by Lane and researchers from the UK and China. Photon optimizes distributed training by making data representation and model synchronization more efficient than existing systems like Google’s DiPaCo. Though slower than traditional training, Photon allows hardware to be added flexibly, enhancing scalability.
In March, Photon was released as open source, giving developers worldwide access to a powerful tool for distributed AI training.
Vana’s role also introduces a major shift in how AI models access training data. Instead of scraping publicly available content—which is often copyrighted—Vana’s platform lets users directly contribute private data. Users can even control how their data is used and potentially profit from it. This marks the first time user-contributed data is being used to build a foundation model, according to Vana cofounder Anna Kazlauskas.
Mirco Musolesi, a computer scientist at University College London, sees enormous potential in using decentralized, privacy-sensitive data—especially in fields like healthcare and finance. These sectors often cannot centralize data due to legal or ethical constraints, but distributed AI training offers a secure alternative.
In summary, distributed AI training not only redefines how models are built but also who can build them. It opens doors for broader participation, more ethical data sourcing, and innovative collaboration. While the frontier of AI may still belong to big players for now, distributed methods like those used to build Collective-1 are set to change the rules of the game.
