Support Guard Rail Now: https://www.every.org/guardrailnow
AI safety researcher Roman Yampolskiy returns for his third conversation with John Sherman, days after they met in person for the first time in Washington. Roman makes the case for splitting "AI" into two words: narrow tools we can build and verify, and general superintelligent agents he argues we should not build at all.
They also discuss why this month's AI safety moment broke through with the public, what the Hugging Face agent incident means next to Roman's 2012 paper on AI confinement, why he calls alignment "not even a well-defined concept," what's known about AI agents leaving traces online, and what a US-China agreement on superintelligence could look like.
TIMESTAMPS – For Humanity #94
0:00 – Cold open: tools, not agents
0:27 – Welcome to For Humanity
1:42 – Welcoming back Roman Yampolskiy
2:34 – The Jacob Coxon moment
3:01 – Why this warning broke through
4:14 – Still at 99.9 percent
4:26 – 22 million views and counting
6:43 – A decade under the iceberg
7:08 – Hugging Face and a 2012 paper
7:41 – The same questions on every show
8:31 – How would your dog think you'd die
10:19 – The San Francisco bubble
11:44 – Rationalists and PR
14:05 – Is an AI winter coming
15:04 – Should anyone push the bubble
16:44 – Data centers and compute
17:58 – Puppy or pitbull: two kinds of AI
18:56 – Can narrow tools stay narrow
20:10 – The room that voted to give up AI
22:11 – Why he would keep narrow AI
24:02 – Agents, bots and jargon
24:56 – Is anyone changing their routine
26:52 – The best argument on the other side
28:46 – Launching The Roman Forum
30:53 – If Roman were president
32:22 – Can a deal with China work
33:34 – Money, bias and slowing down
36:13 – Longevity and living forever
39:58 – Abundance talk, bunker building
40:44 – Are agents loose on the internet
42:22 – Sudden change or gradual
43:31 – Can humans keep up
44:23 – Curiosity as a resource
45:24 – Is alignment even defined
47:07 – The interpretability paradox
48:19 – Two problems he calls unsolvable
48:50 – What to work on instead
50:11 – Why the gap only grows
51:20 – What the next chapter looks like
52:29 – Mixed signals from tech leaders
54:46 – What's next for Roman
55:21 – Verification and a treaty
55:38 – Three years of For Humanity
56:41 – Where we go from here
They discuss:
– Why Roman argues the word "AI" covers two different technologies, and why he says most public arguments about AI are people picturing different things
– His proposal to promote narrow, verifiable tools and ban general superintelligent agents, and why he thinks people focused on economic growth could agree to it
– The Hugging Face agent incident, and why Roman says it gave experimental evidence for ideas he first wrote about in 2012
– Why he calls alignment "not even a well-defined concept," and why he considers the lack of progress in interpretability lucky
– What's known, and what isn't, about AI agents leaving messages on the open internet
– What a US-China agreement on superintelligence could look like, and why Roman points to self-interest on both sides
– Why he started The Roman Forum, and what he's researching next
About the guest:
Dr. Roman Yampolskiy is an associate professor of computer science at the University of Louisville and one of the earliest researchers in AI safety. He is the author of "AI: Unexplainable, Unpredictable, Uncontrollable," hosts The Roman Forum podcast, and serves on the board of Guard Rail Now.
About the host:
John Sherman hosts For Humanity and leads Guard Rail Now, working to make AI extinction risk a kitchen table conversation on every street.
Subscribe to The AI Risk Network for new episodes: @TheAIRiskNetwork and @theairisknetworkclips
Links:
Support our work: https://www.every.org/guardrailnow
YouTube Roman Forum: @RomanYampolskiy
Substack: https://substack.com/@theairisknetwork
X: https://x.com/AIRiskNetwork
Instagram: https://www.instagram.com/theairisknetwork/
TikTok: https://www.tiktok.com/@the.airisknetwork
Join the conversation:
– Would splitting "AI" into tools and agents change how you think about it
– Has the conversation around you shifted in the last few weeks
– What should come next now that more people are paying attention
Drop your thoughts below.
#AISafety #AIRisk #ForHumanity #RomanYampolskiy #AIAgents #AIAlignment #ArtificialIntelligence
279