Zuck's Candid Take on AI's Radical Impact on Humanity

Imagine a world where AI isn't just a tool but a transformative force on par with the invention of computers. In a recent interview on the Dwarkesh Podcast, Mark Zuckerberg shares his insights, hinting at a future where the very fabric of our existence is reshaped by intelligent machines.

Exciting Path Ahead for Meta AI

• The models have become smarter without increasing in size, showcasing improved compression. This means that more intelligence has been packed into the same set of numbers while maintaining the same energy cost for inference.

• In 2022, Meta bought more GPUs than needed to create a video recommender similar to TikTok's For You page for Instagram. Zuckerberg increased his investments as a precaution making them well positioned to train their LLMs.

• Llama 3 has been trained on 15T of data using Meta’s new 24K GPU clusters, which is 7 times larger than the dataset used for Llama 2. This model has 4 times more code and now supports an 8K context length, doubling the capacity of Llama 2. Llama 3 is now the most capable open source LLM.

• The future is parasocial. Zuck envisions 200 million creators on Meta platforms running bots to interact with their fans and followers.

Energy Bottlenecks As AI models become increasingly sophisticated

Zuckerberg anticipates that energy constraints could become a significant bottleneck in their development and deployment. He explained, "There is a capital question. At what point does it stop being worth it to put the capital in? I actually think before we hit that, you're going to run into energy constraints." 

This highlights the need for innovative solutions to power the growing demands of AI systems, such as more efficient data centres and advanced energy management techniques.

Dangers of Open Source 

While Meta is a proponent of open-sourcing AI models, Zuckerberg acknowledges the potential risks associated with this approach. He emphasised the importance of responsible development and monitoring: "We want to see what other people are observing, what we're observing, what we can mitigate, and then we'll make our assessment on whether we can make it open source." 

However, he also noted that the concentration of AI power in the hands of a few companies could be equally dangerous.

"I stay up at night worrying more about an untrustworthy actor having the super strong AI, whether it's an adversarial government or an untrustworthy company or whatever."

Intelligence Is Separable from Consciousness

Zuckerberg challenged the common belief that intelligence is fundamentally tied to consciousness or life. He stated, "I think another bias that people tend to have is thinking that intelligence is somehow fundamentally connected to life. It's not actually clear that it is." 

This separation of intelligence from consciousness has significant implications for the development of AI systems. It suggests that highly capable AI models may not necessarily exhibit human-like behaviours or emotions, but rather serve as powerful tools to enhance human capabilities.

Unchartered territories ahead 

As AI continues to advance at an unprecedented pace, it is crucial for tech leaders and policymakers to carefully consider its potential impact on society. Zuckerberg's insights provide a glimpse into the challenges and opportunities that lie ahead, emphasising the need for responsible development and collaboration to ensure that AI benefits humanity as a whole.