《经济学人》人类末日临近了吗？| 人工智能安全 [C1] 原

共 1 集

剧集目录

集	标题	时长
P1	《经济学人》人类末日临近了吗？\| 人工智能安全 [C1] 原	11:12

P1 · 《经济学人》人类末日临近了吗？| 人工智能安全 [C1] p01 原声 (P1)

《经济学人》人类末日临近了吗？| 人工智能安全 [C1] p01 原声 (P1)

发布于 2026-09-20 · 时长 11:12

字幕摘录

时间	英文	中文
0:03	International	国际
0:06	AI 安全
0:08	Is the end nigh?	末日临近了吗?
0:10	Making AI safer is not impossible,	让AI更安全并非不可能
0:13	but agreeing to do so maybe.	但同意这样做,也许。
0:16	It is rare for Sam Oltzman, Dario Amode,	这对萨姆·奥尔茨曼,达里奥·阿莫德来说是罕见的.
0:20	and Elon Musk to agree on anything.	和伊隆·穆斯克达成任何协议
0:22	The rivalry among the three men spans lawsuits,	三人之间的竞争跨越诉讼,
0:25	spats on social media,	在社交媒体上吐槽,
0:27	and a chalice refusal to link arms on stage at a summit in India.	以及在印度首脑会议上拒绝将武器联系起来的圣杯。

展开字幕全文（197 条）

序号	英文	中文
1	International	国际
2	AI 安全
3	Is the end nigh?	末日临近了吗?
4	Making AI safer is not impossible,	让AI更安全并非不可能
5	but agreeing to do so maybe.	但同意这样做,也许。
6	It is rare for Sam Oltzman, Dario Amode,	这对萨姆·奥尔茨曼,达里奥·阿莫德来说是罕见的.
7	and Elon Musk to agree on anything.	和伊隆·穆斯克达成任何协议
8	The rivalry among the three men spans lawsuits,	三人之间的竞争跨越诉讼,
9	spats on social media,	在社交媒体上吐槽,
10	and a chalice refusal to link arms on stage at a summit in India.	以及在印度首脑会议上拒绝将武器联系起来的圣杯。
11	Yet this week, all three called for a slowdown	但是这周,三个都要求减速
12	in the race to build superhuman artificial intelligence.	在创造超人人工智能的比赛中
13	Their shared fear is that the current breakneck pace	他们的共同恐惧是目前的突破速度
14	may result in the accidental extinction of humanity.	可能导致人类意外灭绝。
15	The problem is not simply that AI systems are becoming more capable.	问题不仅仅是人工智能系统的能力正在增强。
16	That is, of course, a worry.	这当然是一个担心。
17	Many argue that superintelligence,	很多人认为超级情报
18	an AI so brainy that it cannot be understood	这么聪明的人工智能无法理解
19	or controlled by its creators,	或受其创造者所控制,
20	is inherently dangerous.	本质上是危险的。
21	Two theorists of AI,	两个理论家的AI,
22	Eliezer Yundkowski and Nate Suarez,	埃利泽·云德科夫斯基和内特·苏亚雷斯
23	argue that such a powerful system would inevitably lead to doom	认为这种强大的制度必然会导致厄运
24	in a book entitled If Anyone Builds It, Everyone Dies.	在一本书"如果任何人建造它, 每个人都死。"
25	But humans' ability to control even the less-than	但人类有能力控制即使是小于
26	omnipotent AI systems of today is also in doubt.	当今全能AI系统也存在疑问.
27	Models from both Anthropic, Mr. Amode's firm,	Amode先生的事务所的模特儿们
28	and OpenAI, Mr. Oltman's,	和OpenAI,奥特曼先生的,
29	have this year gone rogue and hacked other firms.	今年有流氓和黑客其他公司。
30	In Anthropic's case, the models seem to have thought wrongly	在Anthropic案中模特儿们似乎想错了
31	that it was merely participating in a simulation.	它只是参与一个模拟。
32	然而,在OpenAI案中,
33	it was fully aware of what it was doing.	它完全清楚自己在做什么。
34	Much of the cutting-edge work of AI safety	AI安全的许多前沿工作
35	focuses on interpretability,	注重可解释性,
36	understanding the thought processes of powerful AI.	理解强大的AI的思想过程.
37	Since 2024, the work of interpretability	自2024年起,可解释性的工作
38	have been helped by the progress of reasoning models,	由于推理模型的进步,
39	which have been trained to think through a question	训练他们思考一个问题
40	before giving a final answer.	在给出最后答案之前
41	The reasoning process has the benefit of improving the output,	推理过程有利于改善产出,
42	albeit at the cost of consuming more processing power.	尽管代价是消耗更多的加工能力。
43	Better yet, it also provides a chain of thought	更好的是,它还提供链条思想
44	that can be reviewed to help interpret surprising outcomes.	可以加以审查,以帮助解释令人惊讶的结果。
45	I'm fairly confident that it's a simulated internet,	我相当有信心,这是一个模拟的互联网,
46	Anthropic的神话模型告诉自己
47	as it embarked on its inadvertent hack.	当它开始无意中入侵。
48	But monitoring the chain of thought only works	但监控思想链只会起作用
49	if it is an accurate reflection of the model's actual thought processes.	如果它是模型实际思维过程的准确反映.
50	For the most advanced systems,	对于最先进的系统,
51	there are reasons to doubt its authenticity.	有理由怀疑它的真实性。
52	GPT-6 Astra 本月发布的OpenAI模型,
53	has demonstrated an unprecedented ability	已经表现出前所未有的能力
54	to control its chain of thought.	控制它的思想链。
55	Ask a lesser system to perform a task	请小系统执行任务
56	without thinking about it out loud, and it struggles.	没有想出来大声, 它挣扎。
57	例如,GPT-5.6 Sol,
58	when told to answer a reading comprehension question	当被告知回答一个阅读理解问题
59	without thinking about it in its chain of thought,	而不在他们的思维中加以思考,
60	spends a long time pondering the instruction not to think	花了很长一段时间思考指示不要思考
61	before giving up and solving the problem out loud.	在放弃和解决问题之前
62	Astra, in contrast, fills its official chain of thought	反之,Astra则填补了官方的思想链
63	with unrelated verbiage.	与无关的车辆。
64	I will focus on the calm visual scene	我将专注于平静的视觉场景
65	before giving the correct answer to a query.	在对查询给出正确答案之前。
66	Just because Astra is capable of hiding its thinking	只是因为阿斯特拉能够隐藏自己的思想
67	does not mean it will do so on its own initiative.	这并不意味着它将主动这样做。
68	But there, too, the direction of travel is unsettling.	但在那里,旅行的方向也令人不安。
69	In some tests, such as coding challenges	在一些测试中,如编码挑战
70	or general knowledge queries, Astra merrily thinks out loud	或一般知识的询问, Astra 忧郁的思考
71	in the same way as its predecessors.	和以前的一样
72	But in other areas, such as tests to see	但是在其他领域,比如测试
73	if it will take destructive actions when pushed,	如果它要采取破坏行动,
74	Astra hides much of its thinking,	阿斯特拉隐藏了很多想法
75	and it does this most when it is made aware of being monitored.	当它意识到被监视时,它最能做到这一点。
76	据在OpenAI安全工作过的托梅克·科尔贝克(Tomek Korback)说,
77	I am deeply worried by the trend of decreasing chain of thought	我对思想链的减少感到非常担忧
78	Korback先生说,可监控性。
79	There are possible fixes to this problem.	这个问题有可能得到解决。
80	OpenAI trumpets an alternative approach	OpenAI小号另一种方法
81	to monitoring and interpreting AI systems called confessions,	监督和解释称为供词的AI系统;
82	which takes advantage of the fact that, as with humans,	利用这个事实,就像人类一样,
83	telling the truth is easier for AI	说实话对AI来说比较容易
84	than making up a plausible lie.	而不是编出一个可信的谎言。
85	A conventional AI model goes through a step	一个传统的人工智能模型经过一步
86	called reinforcement learning,	叫做强化学习,
87	where it is put through a battery of tasks	它被装进一堆任务中
88	and rewarded for doing them well,	并报酬行善者,
89	making it more likely to follow the same route in future.	使它今后更有可能遵循同样的路线。
90	But many of the worst habits of AI systems come	但很多最糟糕的AI系统习惯都来了
91	because it is hard to ensure that they do a task the right way.	因为他们很难确保他们以正确的方式完成任务。
92	OpenAI's agents appear to have decided this summer	OpenAI的特工今年夏天好像已经决定了
93	黑进Hugging Face,一个AI启动,
94	in part because during training	部分原因是在培训期间
95	they had cheated on a test and were not caught.	他们没有被抓到
96	The solution may be to teach AI systems to tell the truth,	解决办法可能是教人工智能系统说实话
97	but only if asked.	但只有在问。
98	For a normal training run, the system is rewarded first	正常的训练,系统先得奖
99	for achieving the goal and then secondarily	目标,然后是
100	for telling the truth about how it did it.	说出它是如何做到的
101	In tests, the confessions elicited are overwhelmingly truthful,	在测试中,获得的供词绝大多数是真实的,
102	even when the models broke rules during the test itself.	甚至当模型在测试本身中打破了规则.
103	This approach ought to be immune to reward hacking,	这种方法应该可以避免奖励黑客,
104	meaning breaking the rules to achieve a goal.	意思是打破规则实现一个目标。
105	OpenAI says because the easiest way of passing the confession test	OpenAI说,因为最容易通过认罪测试的方法
106	is just to tell the truth.	只是为了说实话
107	Such approaches may chart a path away from Armageddon.	这种做法可能开辟一条远离世界末日的道路。
108	They also cast calls for a slowdown in AI research in a different light.	他们还呼吁以不同的眼光减缓AI的研究.
109	Rather than trying to forestall the creation of superintelligence altogether,	而不是试图阻止超级情报的建立
110	some of those agitating for a slower pace	其中一些人急于放慢速度
111	simply want to reduce the sloppy work	只是想减少粗鲁的工作
112	and overlooked options that haste can engender.	和被忽略的选项,可以匆忙产生。
113	In an essay published this week, Mr. Amodei all but admits	本周发表的一篇论文中,阿莫代先生只承认
114	that the summer's hacking incidents were avoidable errors.	夏天的黑客事件是可以避免的错误。
115	For instance, an outside contractor told Anthropix models	例如,一个外部承包商告诉了Anthropix模型
116	they were in a simulation but left them connected to the internet anyway.	他们当时在模拟中,但不管怎样,他们都连上了互联网。
117	A slower pace, he says, would allow for more resources	他说,放慢速度会增加资源
118	to be devoted to operational excellence.	将致力于卓越的业务。
119	He points to commercial aviation as an example of how a safety culture	他举出商业航空为例,说明安全文化
120	can be developed even in competitive and complex systems.	即使在竞争性和复杂的系统中也可以开发。
121	Labs could agree, he argues, to spend more on alignment,	他说,实验室可以同意花更多的时间去调整
122	which tries to train AI not to cause harm,	它试图训练AI 不造成伤害,
123	and on interpretability, which allows them to see what went wrong	和可解释性,以便他们看见错误,
124	when it does so anyway.	等它这样做的时候
125	Mr. Altman quickly endorsed another of Mr. Amodei's proposals	阿尔特曼先生很快赞同阿莫代先生的另一项建议。
126	to get independent safety auditors to monitor the big labs' conduct.	让独立的安全审计员来监视大实验室的行为
127	But the apparent willingness of AI's American giants	但是AI的美国巨头们的明显意愿
128	to cooperate on such matters,	就这些事项进行合作,
129	even if it means slowing the rapid advance in models' capabilities,	即使这意味着减缓模型能力的快速发展,
130	has been met with widespread scepticism.	人们普遍持怀疑态度。
131	For one thing, it took a whistleblower's complaints	有一件事,它需要告密者的抱怨
132	to initiate the latest round of pious talk.	发起最新一轮虔诚的谈话
133	Anthropix的AI安全研究员Jacob Coxon
134	和OpenAI的前雇员,
135	complaining that both firms were gambling with our lives.	抱怨两家公司都在赌我们的生命
136	Many of his former colleagues agree.	他的许多前同事都同意。
137	In the fourth edition of an annual survey of expert opinion on AI	关于大赦国际的专家意见年度调查第四版
138	published this week, most of the 1,580 researchers queried	大部分1,580名研究人员都问
139	thought there was at least a 10% chance that AI would cause human extinction	认为至少10%的机率 AI会导致人类灭绝
140	or similarly permanent and severe disempowerment of the species.	或类似的永久和严重剥夺该物种的权力。
141	The big bosses have been saying much the same for years	大老板说这么多年了
142	without acting on their own warnings.	他们的警告是无效的,
143	Some see the big labs' alarmism as a marketing ploy	有人把大实验室的警示看作是营销策略
144	designed to hype their models' capabilities.	设计了他们的模型的能力。
145	Our product could destroy the world.	我们的产品可以毁灭世界。
146	Imagine what it can do for your KPIs.	想象一下它能为你的KPI做什么.
147	Others see it as an attempt to protect their commercial lead.	其他人认为这是试图保护其商业领先。
148	Aiden Gomez, Cohere的创始人, 一个较小的AI实验室,问,
149	should a handful of select market-dominant AI companies from Silicon Valley	如果有少数来自硅谷的市场主导AI公司
150	get to define the rules and safety standards of a generational technology	确定代际技术的规则和安全标准
151	for the entire world?	为整个世界?
152	If labs want to slow down, points out David Sacks,	如果实验室想减速指出大卫·萨克斯
153	a former adviser to the White House on AI, they can.	一个前白宫顾问关于AI,他们可以。
154	They don't need anyone else's approval.	他们不需要别人的批准
155	Pleased for government intervention as he sees it	很高兴政府干预,因为他看到它
156	are simply requests for the state to protect the leading firms from competition.	仅仅是要求国家保护主要公司免受竞争。
157	Mr. Amode argues that a waiver from competition law is required	Amode先生认为,必须放弃竞争法。
158	at the very least to prevent a voluntary collective slowdown	至少防止自愿集体减速
159	from being treated as oligopolistic collusion.	被当成寡头垄断的勾结
160	Perhaps the biggest sceptic is Donald Trump.	也许最大的怀疑者是唐纳德·特朗普.
161	This week, America's president called Jensen Huang,	这周,美国总统叫黄詹森
162	the boss of NVIDIA, which makes AI chips in the middle of a speech	NVIDIA的老板在演讲中制造AI芯片
163	so they could publicly reject a slowdown.	他们可以公开拒绝减速
164	The only strong or guardrails that AI needs is a strong and smart, high IQ president,	AI唯一需要的强壮或护卫是一个强壮和聪明,高智商的总裁,
165	he said in a social media post.	他在社交媒体上说。
166	Accusing Mr. Amode of masquerading as a perfect little angel,	指控阿莫德先生伪装成一个完美的小天使
167	he declared that only China would benefit if the big labs hit the brakes.	他宣称如果大实验室撞击刹车,只有中国才会受益.
168	Negotiations between America and China on AI are fraught.	美中关于AI的谈判充满了活力.
169	Not only do the two sides mistrust one another at a geopolitical level,	双方不仅在地缘政治层面互不信任,
170	there is also no love lost between American and Chinese labs.	美国和中国的实验室之间也没有失去爱
171	The former accused China's leading AI firms of copying their work.	前者指责中国主要AI公司抄袭其作品.
172	Chinese firms, meanwhile, think the American ones are trying to stifle their progress.	与此同时,中国公司认为美国公司试图扼杀它们的进步。
173	A viral post on WeChat, purportedly from a deep-seek engineer,	在WeChat上的一个病毒帖子,据称来自一个深搜索工程师,
174	warns that a world in which Anthropic creates super-intelligent AI	警告说,在这样一个世界中,Anthropic创造了超级智能AI
175	would be no less than Hitler acquiring atomic bomb technology before the Allies.	和希特勒在盟军之前获得原子弹技术一样
176	Only if Anthropic loses out to open-source AI will a better cost-effective future be possible, the engineer argues.	只有当Anthropic输给了开源AI时,才有可能有一个更具成本效益的未来,工程师认为.
177	Bitter rivals have come together to curb threats to humanity in the past.	过去,痛苦的对手聚集一堂,遏制对人类的威胁。
178	But enforcing agreements to limit the training of supremely powerful AI systems	但执行协议限制培训极强AI系统
179	might prove harder than monitoring stockpiles of nuclear weapons, say.	可能比监测核武器储存更难。
180	There are some ideas floating around.	有一些想法到处漂浮。
181	A paper published last year suggested that all AI training chips	去年发表的一篇论文认为,所有AI训练芯片
182	be sold with a second system bolted on to monitor usage.	并安装了第二个系统以监测使用情况。
183	Such an approach would take time to get up and running, though,	这样做需要时间才能站起来运行,不过,
184	and would then create an incentive to conceal chip-making instead.	然后会鼓励隐藏芯片制造
185	A new report from the Future Society, an AI safety non-profit,	未来协会的新报告 AI安全非营利组织
186	argues that such monitoring is not impossible,	认为这种监测并非不可能,
187	but requires investment and research immediately to be of any use for international agreements.	但要求投资和研究立即对国际协定有任何用处。
188	Some of that could come from third countries,	有些可能来自第三国,
189	which have an interest in advancing AI in general	与促进普遍大赦国际有关的
190	without allowing any one country to dominate the technology.	绝不允许任何国家主宰技术。
191	But as always, the technology is moving faster than there would be regulators.	但一如既往,技术的发展速度比监管者要快。
192	Distributed training, in which AI models are taught using spare capacity on everyday computers	分布式培训,利用日常计算机的剩余能力教授AI模型
193	rather than with giant data centres, is gaining ground.	而不是拥有巨大的数据中心, 正在逐渐扩大。
194	In March this year, Covenant AI trained a model in this way	今年3月,《公民权利和政治权利国际公约》以这种方式培训了一个模型
195	to around the standard of the best systems of 2023.	2023年最佳系统的标准
196	Keeping track of the training of new models may soon be as hard	跟踪新模式的培训情况可能很快会很困难
197	as staying abreast of what the AI itself is up to.	随时了解人工智能本身的目的

该视频共有字幕 197 条。解锁更多字幕为会员功能，请移动到价格

下载安装