Last week Meta held an event in which it told us about its Orion augmented reality glasses or the Meta Quest 3S. But he also took the opportunity to present Llama 3.2, the new version of his LLM that is now also very focused on conquering our mobile phones. In an interview commenting on these launches, Mark Zuckerberg talked about an important topic: why they are investing money in AI as if there were no tomorrow.
AI had limits. In a recent interview with YouTuber Cleo Abram, Mark Zuckerberg explained how in previously explored AI architectures you could feed the system with a certain amount of data but at some point it didn’t matter if you fed it with more data or used more resources to train it: you reached a stalemate and it didn’t move any further.
But now he doesn’t seem to have it. However, with systems based on the “Transformer” architecture, that deadlock never seems to be reached. In recent years these systems have not stopped being fed with more and more data and the training and inference infrastructures do not stop growing, but it doesn’t matter: the models continue to improve. As Zuckerberg says, “we haven’t found the limit yet.”
Flame will continue to grow. The CEO of Meta highlighted that because of this “that leads us to that dynamic in which we could train Llama 3 with between 10 and 20,000 GPUs, then train Llama 4 with more than 100,000 and with Llama 5 we could scale even more, and there It raises an interesting question, which is how far we can go.”
But they may be wrong. Zuckerberg, however, is aware that they could be wrong. “It’s entirely possible that at some point we will reach the limit” but he believes we won’t reach that limit anytime soon. They are clearly betting on it, but he admits that he is not sure what can happen. Of course, for the moment it is clear: they will continue to invest money without stopping.
It’s what they all do. Zuckerberg’s speech supports the current situation. Not just that of Meta, but that of all the big technology companies, which have invested absolutely enormous sums of money trusting that AI models are indeed the future.
Qualifications. Yann LeCun, head of AI at Meta, also shared this comment on ” if we are looking for human-level AI, but “they are still very useful in the short term.” For him, all this infrastructure can go beyond simply squeezing current generative AI models.
Zuck says this because he has to say it.. Mark Zuckerberg’s words are not at all surprising: he has to defend his bet on AI as he previously did with the metaverse bet. With that one he lost and continues to lose a real fortune, but on that occasion Meta was the only one who trusted the concept. Now they have pivoted and left the metaverse in the background—but they are still working on the concept—but the difference is that their clear commitment to AI is not just theirs: it is an entire industry. That doesn’t mean the entire industry couldn’t be wrong, of course: Zuckerberg himself points this out in those comments.
In WorldOfSoftware | OpenAI is burning money like there is no tomorrow. The question is how long can he last like this?
