7-23 OpenAI技术是如何“失控”的 | BBC全球新
字幕摘录
| 时间 | 英文 | 中文 |
|---|---|---|
| 0:00 | Welcome to the Global News podcast from the BBC. | 欢迎来到英国广播公司的全球新闻播客。 |
| 0:03 | I'm Valerie Sanderson, and I'm joined by our technology reporter, Chris Valance. | 我是瓦莱丽·桑德森 我们的科技记者克里斯·瓦朗斯也加入了 |
| 0:08 | And Chris, this is a story about AI going rogue. | 克里斯,这是关于AI叛逆的故事。 |
| 0:12 | Tell us about it. | 跟我们说说吧 |
| 0:13 | Right, okay. | 对,好的。 |
| 0:14 | Well, this concerns an AI agent. | 嗯,这涉及到一个AI特工。 |
| 0:15 | These are programs that are designed to have a degree of autonomy. | 这些方案旨在具有一定程度的自主权。 |
| 0:19 | So a program, for example, you might ask, you know, I'm on a business trip, can you plan my travel and book my hotels, and it will go off and do this. | 举例来说,一个程序,你可能会问,你知道,我在出差, 你能计划一下我的旅行,订下我的旅馆, 然后它就会爆炸,做这个。 |
| 0:27 | So OpenAI was testing one of its AI agents powered by two of its latest models, one which it hadn't even released yet. | OpenAI正在测试它的一个AI代理 由它的两个最新的模型提供动力, 一个它还没有发布。 |
| 0:36 | And it set it a task in cyber security. | 它为网络安全设定了一个任务。 |
展开字幕全文（101 条）
| 序号 | 英文 | 中文 |
|---|---|---|
| 1 | Welcome to the Global News podcast from the BBC. | 欢迎来到英国广播公司的全球新闻播客。 |
| 2 | I'm Valerie Sanderson, and I'm joined by our technology reporter, Chris Valance. | 我是瓦莱丽·桑德森 我们的科技记者克里斯·瓦朗斯也加入了 |
| 3 | And Chris, this is a story about AI going rogue. | 克里斯,这是关于AI叛逆的故事。 |
| 4 | Tell us about it. | 跟我们说说吧 |
| 5 | Right, okay. | 对,好的。 |
| 6 | Well, this concerns an AI agent. | 嗯,这涉及到一个AI特工。 |
| 7 | These are programs that are designed to have a degree of autonomy. | 这些方案旨在具有一定程度的自主权。 |
| 8 | So a program, for example, you might ask, you know, I'm on a business trip, can you plan my travel and book my hotels, and it will go off and do this. | 举例来说,一个程序,你可能会问,你知道,我在出差, 你能计划一下我的旅行,订下我的旅馆, 然后它就会爆炸,做这个。 |
| 9 | So OpenAI was testing one of its AI agents powered by two of its latest models, one which it hadn't even released yet. | OpenAI正在测试它的一个AI代理 由它的两个最新的模型提供动力, 一个它还没有发布。 |
| 10 | And it set it a task in cyber security. | 它为网络安全设定了一个任务。 |
| 11 | And essentially, what the AI agent decided to do was to cheat. | 基本上,人工智能特工决定做的是欺骗。 |
| 12 | So it essentially was in a test environment, an environment that was not on the general internet, and it decided that the best way to complete the task was if it broke out of this environment and got access to the internet. | 因此,它基本上在一个测试环境中,一个没有在一般互联网上的环境,它决定,完成这项任务的最佳方式是它从这个环境中爆发出来,并进入互联网. |
| 13 | So it discovered a previously unknown security vulnerability, hacked its way out of its test environment, and then decided to hack into a platform called Hugging Face, which is used by AI programmers. | 因此,它发现了一个以前未知的安全弱点,黑入了自己的测试环境,然后决定黑入一个叫做Hugging Face的平台,由AI程序员使用. |
| 14 | And there it thought it would find the information it needed to essentially cheat in this test. | 在那里,它认为它会找到它所需要的信息 基本上欺骗在这个测试。 |
| 15 | Now, Hugging Face, who noticed that it had been hacked, described this as a mind boggling display of autonomy, and OpenAI called it unprecedented. | 现在,Hugging Face,他注意到它已经被黑了,他形容这是对自主性的心灵摇摆的展示,OpenAI称之为前所未有的. |
| 16 | And they say they are taking steps to make sure this doesn't happen again. | 他们说他们正在采取措施,确保不再发生这种情况。 |
| 17 | But they also say this kind of thing is going to be more likely as these models get more powerful. | 但他们也说,随着这些模型变得更强大,这类东西将更有可能发生. |
| 18 | How could it do this? | 怎么会这样? |
| 19 | I mean, you know, listening to this, I think, well, surely there were rules in place. | 我的意思是,你知道, 听这个, 我想, 嗯,肯定有规则。 |
| 20 | Surely there was security in place to stop it going rogue. | 肯定有保安来阻止它作恶 |
| 21 | Well, this is very interesting. | 这很有趣 |
| 22 | I think this comes to a point about the story that's worth thinking about, essentially, is that in this test, OpenAI decided to remove the safeguards that are normally placed on some of these agents. | 我认为这涉及到一个值得思考的故事, 基本上,在这个测试中,OpenAI决定取消通常对其中一些特工的保障措施。 |
| 23 | It had moved systems designed to spot this kind of behavior. | 它移动了系统 设计来发现这种行为。 |
| 24 | This was a test to look at its maximum capabilities, as it were. | 这是对其最大能力的考验。 |
| 25 | The thing that some commentators are saying, it was it was unwisest of them to do that when really the maximum capabilities turned out to be a lot greater than they expected. | 某些评论家所说的, 他们这样做是不明智的, 当真正的最大能力 被证明是比他们预期的要大得多。 |
| 26 | Has anything like this happened before? | 以前发生过这样的事吗? |
| 27 | Yes, it has. | 是的,它已经。 |
| 28 | Things like this have happened before, but it sort of depends on what you mean by going rogue, if you like. | 这种事以前就发生过 但是这取决于你的意思 如果你喜欢的话 |
| 29 | So if you look at rival firm Anthropic, last year, they reported that in a test environment, they had told their AI model to pretend to be that it was an assistant in a company. | 所以,如果你看看竞争对手的Anthropic公司, 去年,他们报告说,在一个测试环境中, 他们告诉了他们的AI模型 假装它是一个公司的助手。 |
| 30 | They'd given it access to the company's email, and then they told it it was going to be replaced and to come up with various strategies to avoid the scenario in which it was being replaced. | 他们给了它访问公司的电子邮件, 然后他们告诉它要被替换, 并想出各种策略来避免它被替换的情景。 |
| 31 | And one of the things it did was to attempt to blackmail staff to not replace it because it had found evidence in this fictional company database that the staff member in question was having an affair. | 而它所做的一件事是试图勒索工作人员而不取代它,因为它在这个虚构的公司数据库中发现了有关工作人员有外遇的证据. |
| 32 | So it tried to use that to stop itself being turned off. | 所以它试图用它来阻止自己被关闭。 |
| 33 | That was that was quite a sort of contrived scenario, if you like, by Anthropic. | 那是一个相当巧妙的情景, 如果你喜欢,由Anthropic。 |
| 34 | But if you look at these AI agents, these programs that go out and do stuff for us with minimal instructions, able to act on their own recognizance, you know, there a lot of people are quite worried that they might do things we really don't want them to d | 但是,如果你看看这些AI特工, 这些程序出去为我们做一些小事, 以他们自己的认知来行动, 你知道,有很多人很担心 他们可能会做一些我们不想做的事。 |
| 35 | So, for example, you know, reported incidents, we've had one AI agent delete an entire company database. | 所以,比如,你知道,报告的事件, 我们有一个AI代理删除了一个整个公司数据库。 |
| 36 | Another case, it's a very good video if you watch it online, Hannah Frye, the mathematician, set up an AI agent to sell novelty mugs and it started messaging tech journalists and doing all sorts of things she didn't expect it to do. | 另一个案例,这是一个非常好的视频 如果你在网上看,汉娜·弗雷,数学家, 设置了一个AI代理 出售新颖的杯子, 它开始给技术记者发信息 并做各种事情 她不期望它做。 |
| 37 | So, you know, the question is, you know, what are the limits on these AI agents? | 所以,你知道的,问题是,你知道, 什么是限制 这些AI特工? |
| 38 | And there's a there's a wonderful thought experiment by philosopher Nick Bostrom, where he talks about a paperclip maximizer. | 还有一个哲学家尼克·博斯特罗姆(Nick Bostrom)的精彩思想实验,他在那里谈到一个纸片放大器. |
| 39 | So, you know, you can imagine an AI that was told to go out and sell paperclips. | 所以,你知道,你可以想象一个人工智能 被告知出去卖纸片。 |
| 40 | And, you know, it basically does that to the maximum degree possible. | 而且,你知道,它基本上 这样做的最大限度。 |
| 41 | It sees human beings as a possible roadblock on its way to creating as many paperclips as it possibly can, you know, covers the whole planet in paperclips. | 它把人类看作是一个可能的路障 在它创造尽可能多的纸片的路上, 你知道,它用纸片覆盖整个星球。 |
| 42 | But that's just what it's told to do. | 但是,这恰恰是它被告知要做的. |
| 43 | So that's what it does. | 故其义也. |
| 44 | So, you know, both in thought experiments and in reality, the issue of these AI agents sometimes doing things their owners, their creators didn't want them to do is quite a serious one. | 所以,在思想实验和现实中, 这些人工智能代理有时 做他们所有者,他们的创造者 不希望他们做的事, 是一个相当严重的问题。 |
| 45 | It sounds as if the AI is thinking for itself. | 听起来好像人工智能在为自己着想 |
| 46 | Is that what is actually happening? | 这是真的吗? |
| 47 | Well, I think terms like thinking are kind of tricky because, you know, they suggest something similar to human thought and human reasoning, which I don't necessarily translate. | 嗯,我认为,像思维这样的术语是有点棘手的, 因为,你知道,他们提出的东西 类似于人类的思想和人类的推理, 我不一定翻译。 |
| 48 | I think the issue here is that they give them capabilities. | 我认为这里的问题是,他们给予他们能力。 |
| 49 | They allow them to do things computationally. | 他们允许他们用计算的方法做事情。 |
| 50 | They might be access to financial systems. | 他们可能进入金融系统。 |
| 51 | They have access to your email. | 他们可以访问你的电子邮件。 |
| 52 | They have access to the Internet. | 他们可以上网。 |
| 53 | You ask them to go and do things that the difficulty there is making sure the machines are constrained in some way. | 你要求他们去做一些事情 那里的困难在于确保机器受到某种限制 |
| 54 | So they only do things that are legal, responsible and safe. | 所以他们只做合法、负责任和安全的事。 |
| 55 | Now, that's quite a hard challenge in part because the capabilities of these models are growing all the time. | 现在,这是一个相当艰巨的挑战, 部分原因是这些模型的能力在不断增长。 |
| 56 | And in a sense, the companies don't necessarily have a perfect understanding of what those capabilities are. | 从某种意义上说, 公司不一定完全了解这些能力是什么。 |
| 57 | I mean, that's why they do testing and why security institutes and governments do testing to find out, you know, just how powerful they are. | 这就是为什么他们做测试, 以及为什么安全机构和政府做测试 来发现,你知道, 他们是多么强大。 |
| 58 | So, Chris, I mean, how close are we to a time when AI is actually smarter than even the smartest human? | 所以,克里斯,我的意思是,我们有多接近一个时代 AI实际上比最聪明的人更聪明? |
| 59 | Well, that's a question, the answer to which depends on who you ask. | 那是个问题 答案取决于你问谁 |
| 60 | But there's certainly a sense that it's much closer than we might have thought it was, say, five years ago. | 但肯定有一种感觉 比我们想象的要近得多 比如说五年前 |
| 61 | It's also a complex question because it depends what you mean by smarter than a human. | 这也是一个复杂的问题,因为它取决于你所说的"比人类更聪明"的意思. |
| 62 | I mean, you can look at robots and they still struggle to do things like loading a dishwasher. | 我的意思是,你可以看机器人 他们仍然挣扎 做的东西,如装上洗碗机。 |
| 63 | On the other hand, my phone can beat me every time at a game of chess. | 另一方面,我的手机每次在棋盘比赛时都能击败我. |
| 64 | So it really depends. | 所以这真的取决于。 |
| 65 | Smarter at what is also a part of the question. | 更聪明的是问题的一部分。 |
| 66 | But there's a lot of concern that artificial general intelligence is not very far away. | 但是,对于人工一般智能距离并不很遥远,人们也有很多担忧. |
| 67 | And there's certainly concern amongst governments that the new models are becoming ever more powerful at a regular, fast rate. | 并且政府中肯定有人担心 新模式越来越强大 以正常的速度,快速的速度。 |
| 68 | You mentioned constraints. | 你提到限制。 |
| 69 | What are the constraints that exist already and what is being mooted by governments and indeed by companies themselves? | 哪些制约因素已经存在,哪些是政府乃至公司本身提出的? |
| 70 | Well, companies themselves have taken steps when they felt that a model was so powerful that releasing it might put an extremely powerful tool in the hands of the wrong people. | 公司自己也采取了步骤 当它们觉得一个模型非常强大 以至于释放它 可能把一个极其强大的工具 交给错误的人。 |
| 71 | So Anthropic's Mythos AI model, there's a huge amount of concern about its capabilities. | 因此,Anthropic的神话AI模型,对其能力存在极大的担忧. |
| 72 | You know, security institutes that tested those capabilities found they were they were a step change. | 你知道,安全机构 测试这些能力 发现他们是一个步骤改变。 |
| 73 | And Anthropic has sort of slowed the release of that model. | 而Anthropic 在某种程度上减缓了该模型的发布。 |
| 74 | It didn't put it on general release straight away. | 它没有把它放在一般 释放直接。 |
| 75 | And so that's the company holding back. | 公司也因此退缩了 |
| 76 | Of course, lots of people say you can't just rely on the companies to behave responsibly. | 当然,很多人都说 你不能只依靠公司来负责行事。 |
| 77 | So that then turns to regulation. | 从而转而管理。 |
| 78 | And there's certainly a lot of discussion about that and regulation in force in some areas. | 并且肯定有很多关于这个和某些领域生效的法规的讨论. |
| 79 | But there's a challenge there. | 但那里有挑战 |
| 80 | This is a global industry. | 这是一个全球性的产业。 |
| 81 | And so regulation really has to proceed internationally. | 所以管制真的必须在国际上进行。 |
| 82 | If you're not to get a situation where one set of countries holds back and the space that's created is simply filled by models from another country. | 如果你不想得到一个情况, 一组国家被拖住, 所创造的空间只是由另一个国家的模型填补。 |
| 83 | And is there international agreement on this? | 是否有这方面的国际协议? |
| 84 | I think it's hard to say there's international agreement. | 我认为很难说有国际协议. |
| 85 | What you can say is that there's international concern, I think, and a sense that, you know, | 你可以说,有国际关注,我认为, 和一种感觉,你知道, |
| 86 | multinational agreement is also potentially part of the picture. | 多国协定也是可能的一部分。 |
| 87 | I suppose the problem is it's a Pandora's box, isn't it? | 我想问题是它是一个潘多拉的盒子,不是吗? |
| 88 | We can't put AI back in the box. | 我们不能把AI放回盒子里 |
| 89 | We have to deal with the issue as it is now. | 我们必须像现在这样处理这个问题。 |
| 90 | Yeah, it can't be uninvented. | 是啊,它不能没有发明。 |
| 91 | That's for sure. | 当然 |
| 92 | And the pace of change with AI is one that's hard for legislatures to keep up with, let alone international agreements. | 而与AI的变革步伐是立法机构难以跟上的,更不用说国际协议了. |
| 93 | So, yeah, it is a Pandora's box. | 所以,是的,这是一个潘朵拉的盒子。 |
| 94 | There's plenty of promise as well. | 亦有如来广大供养. |
| 95 | But there's also, you know, a rapid technological development that's hard to adjust to. | 但也有,你知道的, 快速的技术发展是很难适应的。 |
| 96 | Thanks, Chris. | 谢谢 克莉丝 |
| 97 | That was Chris Vallance, our technology reporter. | 那是克里斯・瓦伦斯,我们的技术记者。 |
| 98 | If you want to find out more about this story, listen to the Global News podcast. | 如果你想了解更多这个故事,请听全球新闻播客的节目. |
| 99 | Just click on the link below. | 点击下面的链接 |
| 100 | But that's it from this edition of the Global News podcast here on YouTube. | 但从YouTube上的全球新闻播客的这个版本来看, |
| 101 | Thanks for listening, thanks for watching and bye for now. | 谢谢你的聆听,谢谢你的观看和再见 现在。 |
该视频共有字幕 101 条。解锁更多字幕为会员功能，请移动到 价格