Gary Marcus meets me outside the post office of Vancouver’s Granville Island wearing neon-coral sneakers and a blue Arc’teryx jacket. I’m in town for a family thing, and Marcus has lived in the city since 2018, after 20 years in New York City. “I just find it to be paradise,” he tells me, as I join him on his daily walk around Granville Island and nearby Kitsilano Beach. We’ve agreed to walk and talk about—what else—the current state of AI.
“I’m depressed about it,” he tells me. “When I went into this field, it was not so that we could have a massive turnover of wealth from artists to big corporations.” I take a big sip of my black dark-roast coffee. Off we go.
Marcus, a professor emeritus at NYU, is a prominent AI researcher and cognitive scientist who has positioned himself as a vocal critic of deep learning and AI. He is a divisive figure. You might recognize him from the spicy feuds on X with AI heavyweights such as Yann LeCun and Geoffrey Hinton. (“All attempts to socialize me have failed,” he jokes.) It is on walks like this that Marcus often does most of his tweeting.
This week has been a big news week in AI. Google DeepMind launched the next generation of its powerful artificial-intelligence model Gemini, which has an enhanced ability to work with large amounts of video, text, and images. And OpenAI has built a striking new generative video model called Sora that can take a short text description and turn it into a detailed, high-definition film clip up to a minute long. AI video generation has been around for a while, but Sora seems to have upped the ante. My X timeline has been flooded with stunning clips people have generated using the software. OpenAI claims that its results suggest that scaling video generation models like Sora “is a promising path towards building general purpose simulators of the physical world.” You can read more about Sora from Will Douglas Heaven here.
But—surprise—Marcus is not impressed. “If you look at [the videos] for a second, you’re like, ‘Wow, that’s amazing.’ But if you look at them carefully, [the AI system] still doesn’t really understand common sense,” he says. In some videos, the physics are clearly off, and animals and people spontaneously appear and disappear, or things fly backwards, for example.
For Marcus, generative video is yet another example of the exploitative business model of tech companies. Many artists and writers and even the New York Times have sued AI companies, claiming that their practice of indiscriminately scraping the internet for data to train their models violates their intellectual property. Copyright issues are top of Marcus’s mind. He managed to get popular AI image generators to generate scenes from Marvel movies or famous characters such as the Minions, Sonic the Hedgehog, and Darth Vader. He has started lobbying for clearer rules on what goes into AI models.
“Video generation should not be done with copyrighted materials taken without consent, in systems that are opaque, where we can’t understand what’s going on,” he says. “It shouldn’t be a legal thing. It’s certainly not an ethical thing.”
We stop at a scenic spot. It’s a beautiful route, with views of the city, the mountains, and the beach. A speckle of sun hits the peak of a mountain just across the bay. We could not be further away from Silicon Valley, the epicenter of today’s AI boom. “I’m not a religious person, but these kinds of tableaux … just continue to blow my mind,” Marcus says.
But despite the tranquility of the surroundings, it is on walks like this that Marcus often uses X to rail against the power structures of Silicon Valley. Right now, he says, he identifies as an activist.
When I ask him what motivates him, he replies without missing a beat: “The people who are running AI don’t really care that much about what you might call responsible AI, and that the consequences for society may be severe.”
Late last year he wrote a book, called Taming Silicon Valley, which is coming out this fall. It is his manifesto on how AI should be regulated, but also a call to action. “We need to get the public involved in the struggle to try to get the AI companies to behave responsibly,” he says.
There are a bunch of different things people can do, ranging from boycotting some of the software until people clean up their act to choosing electoral candidates around their tech policies, he says.
Action and AI policy are needed urgently, he argues, because we are in a very narrow window during which we can fix things in AI. The risk is that we make the same mistakes regulators made with social media companies.
“What we saw with social media is just going to be like an appetizer compared to what’s going to happen,” he says.
Around 12 000 steps later, we’re back at Granville Island’s Public Market. I’m starving, so Marcus shows me a spot that serves good bagels. We both get the lox with cream cheese and eat it outside in the sun before parting ways.
Later that day, Marcus would send out a flurry of tweets about Sora, having seen enough evidence to call it: “Sora is fantastic, but it is akin to morphing and splicing, rather than a path to the physical reasoning we would need for AGI,” he wrote. “We will see more systemic glitches as more people have access. Many will be hard to remedy.”
Don’t say he didn’t warn you.
_______________________________________
DEEPER LEARNING
A new satellite will use Google’s AI to map methane leaks from space
A methane-measuring satellite will launch next month that aims to use Google’s AI to quantify, map, and reduce leaks. The mission is part of a collaboration with the nonprofit Environmental Defense Fund, and the result, they say, will be the most detailed portrait yet of methane emissions. It should help to identify where the worst spots are and who is responsible.
Putting methane on the map: With methane responsible for roughly a third of the warming caused by greenhouse gases, regulators in the US and elsewhere are pushing for stronger rules to curb the leaks that spring from oil and gas plants. MethaneSAT will measure the plumes of methane that billow invisibly from oil and gas operations around the globe, and Google and EDF will then map those leaks for use by researchers, regulators, and the public. Read more from our new AI reporter James O’Donnell. James will cover the intersection between AI and hardware, such as robotics and chips.
_____________________________________________________________________
BITS AND BYTES
Is AI going to change how we define videos?
Systems like OpenAI’s Sora don’t make recordings. They render ideas. Does it matter that they’re not real? (New Yorker)
Early adopters of Microsoft’s AI bot are wondering if it’s worth the money
Testers have had it in their hands for six months now, and the results are mixed, to say the least. (WSJ)
The White House will spend $1.5 billion on a new chip factory
The massive grant, part of the CHIPS and Science Act, will help the US establish a homegrown supply for some of the most critical components of modern life. (WP)
AI hype has echoes of the telecom boom and bust
When a chief executive asks for trillions, not billions, when raising funds you know a sector might be getting a bit too hot. (FT)