We’re in the early stages of an AI arms race.
The biggest, most influential and well resourced companies in the world have, more-or-less, all pivoted their business models to focus primarily on progressing their own development of Artificial Intelligence.
Google, Microsoft, Meta (previously Facebook), Amazon, X.
In a somewhat surprising plot twist, Meta have emerged as the leading flag bearers of ‘Open Source AI’.
In a recent conversation and blog post, Zuck (from Meta), shared the motivations behind his Open Sorcery.
Big ideas from the conversation:
1. All in on Open Source
Zuck has repeatedly shared that he believes the pros of open sourcing the models outweigh the cons. I think it’s a good sign that he shares both self-interested motivations (i.e. why it’s in the best interests of himself and Meta in the race to AGI), how he has been screwed in the past by closed source platforms like Apple, and the broader ethical interests for everyone to pursue Open Source AI.
Here, he shares some of those motivations.
2. Creator AI, AI studio and Boutique Experiences
The new set of tools being developed by META will allow users to create personalized AI agents through AI Studio. “a set of tools that eventually is going to make it so that every creator can build sort of an AI version of themselves, as sort of an agent or an assistant that their community can interact with.”
The creator studio will be deeply integrated with Instagram, Facebook etc – but the bigger implication here, is that with an open source model like Llama, anyone will have the ability to build their own similar service – retraining the model on different data sets.
You can get a feel for how that looks with Huang explaining how they’re already using Llama to build their own custom AI agents.
If successful, this will probably force other companies toward some version of open source and greater API access. Or to at least make more flexibly accessible variations of their models.
3. AI agents and Social Simulation findings
Super interesting observation which I think is going to become more pronounced in coming months and years – AI is already being used to simulate unique social situations. A finding which is obviously driving the push toward these new personalised AI agents and tools.
What is ‘Open Source AI’, and why does it matter?
Models are trained on ginormous amounts of internet data.
The training produces ‘model weights’ which you can think of as a digital cookbook, storing all the learned knowledge.
Only a small handful of companies have the resources required to create competing cook books from scratch – a process requiring disturbing amounts of money and energy.
Unsurprisingly, most of these companies don’t share (Open Source) their entire cookbook.
When you use ChatGPT, you’re basically getting gated access to a tiny sliver of a finished dish. The entire book, the recipes, methods, ingredients are all hidden.
The fundamental difference between open-source models like LLaMA and closed-source models like ChatGPT, is that with LLaMA, the entire cook-book, the whole shebang – the source code, including the architecture, training scripts and pre-trained weights are all shared openly.
There is a heated debate contesting the pros and cons of open sourcing the entire cook-book. We won’t go into those now – suffice to say there are big pros and big cons.
Here’s a quote directly from Zuck’s most recent spiel on Open Source and the newly released Open Source Llama model.
“Today we’re taking the next steps towards open source AI becoming the industry standard. We’re releasing Llama 3.1 405B, the first frontier-level open source AI model, as well as new and improved Llama 3.1 70B and 8B models. In addition to having significantly better cost/performance relative to closed models, the fact that the 405B model is open will make it the best choice for fine-tuning and distilling smaller models.”