美国国家公共电台 NPR--We asked the new AI to do some simple rocket science. It crashed and burned

We asked the new AI to do some simple rocket science. It crashed and burned

Transcript¹

Tiera Fletcher carefully read through an artificial intelligence chatbot's attempt at rocket science.

"That's true, that's factual," she said thoughtfully as she scanned the AI-generated description of one of the most fundamental equations, known simply as "the rocket equation."

Then she got to the bot's attempt to write the rocket equation itself – and stopped.

"No ... Mmm mmm ... it would not work," she said. "It's just missing too many variables."

Fletcher is a professional rocket scientist and co-founder of Rocket With The Fletchers, an outreach organization. She agreed to review text and images about rocketry generated by the latest AI technology, to see whether the computer programs could provide people with the basic concepts behind what makes rockets fly.

The results were far from stellar. In virtually every case, ChatGPT – the recently released chatbot from the company OpenAI – failed to accurately² reproduce even the most basic equations of rocketry. Its written descriptions of some equations also contained errors. And it wasn't the only AI program to flunk³ the assignment. Others that generate images could turn out designs for rocket engines that looked impressive, but would fail catastrophically if anyone actually attempted to build them.

A real schematic of a rocket engine used by NASA's Apollo program (left), and one imagined by Midjourney's image-generating software (right). "What are those bellows⁴ for at the bottom?" asked Paulo Lozano, a puzzled rocket scientist.

Left: NASA; Right: NPR staff generated imagery using Midjourney

OpenAI did not respond to NPR's request for an interview, but on Monday it announced an upgraded version with "improved factuality and mathematical capabilities⁵." A quick try by NPR suggested it may have improved, but it still introduced errors into important equations and could not answer some simple math problems.

Independent researchers say these failures, especially in contrast to the successful use of computers for half-a-century in rocketry, reveal a fundamental problem that may put limits on the new AI programs: They simply cannot figure out the facts.

"There are some people that have a fantasy that we will solve the truth problem of these systems by just giving them more data," says Gary Marcus, an AI scientist and author of the book Rebooting AI.

But, Marcus says, "They're missing something more fundamental."

Calculating liftoff

Since the 1960s, computers have been essential tools for space travel. The enormous Saturn⁶ V rockets that carried astronauts to the moon used an automatic launch sequence to guide the spacecraft into orbit. Today, rockets are still flown mainly by computers, which can monitor their complex systems and make adjustments far quicker than their human cargo⁷.

Oh yeah, this is a fail.

Paulo Lozano, MIT Department of Aeronautics⁸ and Astronautics

"We cannot operate rockets without computers," says Paulo Lozano, a rocket scientist at the Massachusetts Institute of Technology. Computers also play a central role in the design and testing of new rockets, allowing them to be built faster, cheaper and better. "Computers are key," he says.

Midjourney's attempted to recreate the path of a rocket travelling from the earth and the moon. The result, says Tiera Fletcher, is beautiful but too complex: "It should look a lot simpler than this."

NPR staff generated image using Midjourney

The latest round of artificial intelligence programs are impressive in their own right. After its release in November, ChatGPT has been tested by human users from virtually every corner of the Internet. A doctor used it to generate a letter to an insurer. The media company Buzzfeed recently announced it would use the program to create personalized quizzes. And colleges and universities have raised fears of rampant⁹ cheating using the chatbot.

It seemed possible that AI could be used as a tool to do some basic rocket science.

But so far, ChatGPT has proven inept¹⁰ at reproducing even the simplest ideas in rocketry. In addition to messing up the rocket equation, it bungled¹¹ concepts such as the thrust-to-weight ratio, a basic measure of the rocket's ability to fly.

"Oh yeah, this is a fail," said Lozano after spending several minutes reviewing around a half-dozen rocketry-related results.

Image-generating programs, such as OpenAI's DALL?E2, also came up short. When asked to provide a blueprint¹² of a rocket engine, they produced complex-looking schematics that vaguely¹³ resemble rocket motors but lack things like openings for the hot gasses to come out of. Other graphics¹⁴ programs including those from Midjourney and Stable Diffusion¹⁵ produced similarly cryptic¹⁶ motor designs, with pipes leading nowhere and shapes that would never fly.

I'm sorry Dave, I'm afraid I can't do that

The strange results reveal how the programming behind the new AI is a radical¹⁷ departure from the sorts of programs that have been used to aid rocketry for decades, according to Sasha Luccioni, a research scientist for the AI company Hugging Face. "The actual way that the computer works is very, very different," she says.

A traditional computer used to design or fly rockets comes loaded with all the requisite¹⁸ equations. Programmers explicitly¹⁹ tell it how to respond to different situations, and carefully test the computer programs to make sure they behave exactly as expected.

By contrast, these new systems develop rules of their own. They study a database filled with millions, or perhaps billions, of pages of text or images and pull out patterns. Then they turn those patterns into rules, and use the rules to produce new writing or images they think the viewer wants to see.

The results can provide an impressive approximation of human creativity. ChatGPT has generated poems and songs on things like how to get a peanut butter sandwich out of a VCR. Luccioni thinks AI like this might someday help artists come up with new ideas.

"They generate, they hallucinate, they create new combinations of words based on what they learned," Luccioni says.

But the limitations become clear when the program is asked to use its talents for generating new material related to factual information – for example, when it is asked to write out the rocket equation.

"What it's doing is mimicking²⁰ a bunch of physics textbooks that it's been exposed to," she says. It can't tell if the mashed-up text it's produced is factually correct. And that means anything it generates can contain an error.

Moreover, the program may generate inconsistent results if asked to deliver the same information repeatedly. If asked the capital of France, for example, Luccioni says the program is statistically²² very likely to say Paris, based on its self-training from millions of texts. But because it's trying to simply predict the next word in the exchange with its human counterpart, every once in a while it might choose a different city. (This could explain why ChatGPT produced multiple versions of the rocket equation, some better than others.)

Luccioni points out that these shortcomings shouldn't surprise anyone. At its core, she says, ChatGPT was trained explicitly to write, not to do math. The program has been fine-tuned to respond to human feedback, so it's particularly good at following prompts from people and interacting with them.

"It gets things wrong, because it's not actually designed to get things right," says Emily M. Bender, a professor of linguistics²³ at the University of Washington who studies AI systems. "It's designed to say things that sound plausible²⁴."

Bender believes that ChatGPT's prowess with language, combined with its disregard for facts, makes it potentially dangerous. For example, some have proposed using ChatGPT to generate legal documents and even defenses for lesser²⁵ crimes. But an AI program "doesn't know the laws, it doesn't know what your current situation is," Bender warns. "It can pull together scraps²⁶ from its training data to make something that looks like a legal contract, but that's not what you want." Similarly, using ChatGPT for medical or mental health services could be potentially catastrophic, given its lack of understanding.

Getting the facts straight

Just what it would take to get ChatGPT to sort fact from fiction remains²⁷ unclear. An effort by Meta, the parent company of Facebook, to use an AI system for scientific papers was taken down in a matter of days, in part because it generated fake references.

Because these systems are designed to generate human-sounding text through statistical²¹ analysis of enormous databases of information, Bender wonders if there really is a straightforward²⁸ way to make it select only "correct" information.

"I don't think it can be error-free," she says.

If improvements can be made, then Luccioni and Bender say they will come from using different training programs to teach the AI systems. Some researchers are already making efforts to improve that training. For example, Yejin Choi, an AI researcher at the University of Washington and the Allen Institute for Artificial Intelligence, has experimented with training an AI program using a virtual textbook of vetted²⁹ information. The result seemed to improve its ability to understand new situations.

Choi told NPR's Short Wave that the goal of her work is to teach these new AI systems about more than just language: "Really, beneath the surface, there's these huge unspoken assumptions about how the world works," she said.

Autocomplete on steroids

AI researcher Gary Marcus worries that the public may be radically³⁰ overestimating³¹ these new programs. "We're very easily pulled in by things that look a little bit human, into thinking that they're actually human," he says. But these systems, he adds, "are just autocomplete on steroids."

Marcus agrees with Bender's assessment³² that the new systems' propensity³³ for producing errors may be so innate³⁴ that there will be no easy way to get them to be more truthful³⁵. Although it may be possible to tweak the training to improve their results, it's unclear exactly what's required because these self-taught programs are so complex.

"There's still no fundamental theoretical understanding of exactly how they work," Marcus says.

Ultimately, he believes that AI may need a more head-on approach to figuring out whether it's telling the truth.

"We need an entirely³⁶ different architecture that reasons over facts," he says. "That doesn't have to be the whole thing, but that has to be in there."

1 transcript   

n.抄本，誊本，副本，肄业证书

参考例句：

• A transcript of the tapes was presented as evidence in court.一份录音带的文字本作为证据被呈交法庭。
• They wouldn't let me have a transcript of the interview.他们拒绝给我一份采访的文字整理稿。

2 accurately   

adv.准确地，精确地

参考例句：

• It is hard to hit the ball accurately.准确地击中球很难。
• Now scientists can forecast the weather accurately.现在科学家们能准确地预报天气。

3 flunk   

v.(考试)不及格(=fail)

参考例句：

• I will flunk him if my student doesn't learn the material in the course.如果我的学生没有掌握课程的内容，我就会让他不及格。
• If you flunk finals,you don't get the chance to do them again.如果你没通过期末考试，就没有机会再考一次了。

4 bellows   

n.风箱；发出吼叫声，咆哮（尤指因痛苦）( bellow的名词复数 )；（愤怒地）说出（某事），大叫v.发出吼叫声，咆哮（尤指因痛苦）( bellow的第三人称单数 )；（愤怒地）说出（某事），大叫

参考例句：

• His job is to blow the bellows for the blacksmith. 他的工作是给铁匠拉风箱。 来自辞典例句
• You could, I suppose, compare me to a blacksmith's bellows. 我想，你可能把我比作铁匠的风箱。 来自辞典例句

5 capabilities   

n.能力( capability的名词复数 )；可能；容量；[复数]潜在能力

参考例句：

• He was somewhat pompous and had a high opinion of his own capabilities. 他有点自大，自视甚高。 来自辞典例句
• Some programmers use tabs to break complex product capabilities into smaller chunks. 一些程序员认为，标签可以将复杂的功能分为每个窗格一组简单的功能。 来自About Face 3交互设计精髓

6 Saturn   

n.农神，土星

参考例句：

• Astronomers used to ask why only Saturn has rings.天文学家们过去一直感到奇怪，为什么只有土星有光环。
• These comparisons suggested that Saturn is made of lighter materials.这些比较告诉我们，土星由较轻的物质构成。

7 cargo   

n.(一只船或一架飞机运载的)货物

参考例句：

• The ship has a cargo of about 200 ton.这条船大约有200吨的货物。
• A lot of people discharged the cargo from a ship.许多人从船上卸下货物。

8 aeronautics   

n.航空术，航空学

参考例句：

• National Aeronautics and Space undertakings have made great progress.国家的航空航天事业有了很大的发展。
• He devoted every spare moment to aeronautics.他把他所有多余的时间用在航空学上。

9 rampant   

adj.(植物)蔓生的；狂暴的，无约束的

参考例句：

• Sickness was rampant in the area.该地区疾病蔓延。
• You cannot allow children to rampant through the museum.你不能任由小孩子在博物馆里乱跑。

10 inept   

adj.不恰当的，荒谬的，拙劣的

参考例句：

• Whan an inept remark to make on such a formal occasion.在如此正式的场合，怎么说这样不恰当的话。
• He's quite inept at tennis.他打网球太笨。

11 bungled   

v.搞糟，完不成( bungle的过去式和过去分词 )；笨手笨脚地做；失败；完不成

参考例句：

• They bungled the job. 他们把活儿搞糟了。
• John bungled the job. 约翰把事情搞糟了。 来自《现代英汉综合大词典》

12 blueprint   

n.蓝图，设计图，计划；vt.制成蓝图，计划

参考例句：

• All the machine parts on a blueprint must answer each other.设计图上所有的机器部件都应互相配合。
• The documents contain a blueprint for a nuclear device.文件内附有一张核装置的设计蓝图。

13 vaguely   

adv.含糊地，暖昧地

参考例句：

• He had talked vaguely of going to work abroad.他含糊其词地说了到国外工作的事。
• He looked vaguely before him with unseeing eyes.他迷迷糊糊的望着前面，对一切都视而不见。

14 graphics   

n.制图法，制图学；图形显示

参考例句：

• You've leveraged your graphics experience into the video area.你们把图形设计业务的经验运用到录像业务中去。
• Improved graphics took computer games into a new era.经改进的制图技术将电脑游戏带进了一个新时代。

15 diffusion   

n.流布；普及；散漫

参考例句：

• The invention of printing helped the diffusion of learning.印刷术的发明有助于知识的传播。
• The effect of the diffusion capacitance can be troublesome.扩散电容会引起麻烦。

16 cryptic   

adj.秘密的，神秘的，含义模糊的

参考例句：

• She made a cryptic comment about how the film mirrored her life.她隐晦地表示说这部电影是她人生的写照。
• The new insurance policy is written without cryptic or mysterious terms.新的保险单在编写时没有隐秘条款或秘密条款。

17 radical   

n.激进份子，原子团，根号；adj.根本的，激进的，彻底的

参考例句：

• The patient got a radical cure in the hospital.病人在医院得到了根治。
• She is radical in her demands.她的要求十分偏激。

18 requisite   

adj.需要的，必不可少的；n.必需品

参考例句：

• He hasn't got the requisite qualifications for the job.他不具备这工作所需的资格。
• Food and air are requisite for life.食物和空气是生命的必需品。

19 explicitly   

ad.明确地，显然地

参考例句：

• The plan does not explicitly endorse the private ownership of land. 该计划没有明确地支持土地私有制。
• SARA amended section 113 to provide explicitly for a right to contribution. 《最高基金修正与再授权法案》修正了第123条，清楚地规定了分配权。 来自英汉非文学 - 环境法 - 环境法

20 mimicking   

v.（尤指为了逗乐而）模仿( mimic的现在分词 )；酷似

参考例句：

• She's always mimicking the teachers. 她总喜欢模仿老师的言谈举止。
• The boy made us all laugh by mimicking the teacher's voice. 这男孩模仿老师的声音，逗得我们大家都笑了。 来自辞典例句

21 statistical   

adj.统计的，统计学的

参考例句：

• He showed the price fluctuations in a statistical table.他用统计表显示价格的波动。
• They're making detailed statistical analysis.他们正在做具体的统计分析。

22 statistically   

ad.根据统计数据来看，从统计学的观点来看

参考例句：

• The sample of building permits is larger and therefore, statistically satisfying. 建筑许可数的样本比较大，所以统计数据更令人满意。
• The results of each test would have to be statistically independent. 每次试验的结果在统计上必须是独立的。

23 linguistics   

n.语言学

参考例句：

• She plans to take a course in applied linguistics.她打算学习应用语言学课程。
• Linguistics is a scientific study of the property of language.语言学是指对语言的性质所作的系统研究。

24 plausible   

adj.似真实的，似乎有理的，似乎可信的

参考例句：

• His story sounded plausible.他说的那番话似乎是真实的。
• Her story sounded perfectly plausible.她的说辞听起来言之有理。

25 lesser   

adj.次要的，较小的；adv.较小地，较少地

参考例句：

• Kept some of the lesser players out.不让那些次要的球员参加联赛。
• She has also been affected,but to a lesser degree.她也受到波及，但程度较轻。

26 scraps   

油渣

参考例句：

• Don't litter up the floor with scraps of paper. 不要在地板上乱扔纸屑。
• A patchwork quilt is a good way of using up scraps of material. 做杂拼花布棉被是利用零碎布料的好办法。

27 remains   

n.剩余物，残留物；遗体，遗迹

参考例句：

• He ate the remains of food hungrily.他狼吞虎咽地吃剩余的食物。
• The remains of the meal were fed to the dog.残羹剩饭喂狗了。

28 straightforward   

adj.正直的，坦率的；易懂的，简单的

参考例句：

• A straightforward talk is better than a flowery speech.巧言不如直说。
• I must insist on your giving me a straightforward answer.我一定要你给我一个直截了当的回答。

29 vetted   

v.审查（某人过去的记录、资格等）( vet的过去式和过去分词 )；调查；检查；诊疗

参考例句：

• The recruits were thoroughly vetted before they were allowed into the secret service. 情报机关招募的新成员要经过严格的审查。 来自《简明英汉词典》
• All staff are vetted for links with extremist groups before being employed. 所有职员录用前均须审查是否与极端分子团体有关。 来自辞典例句

30 radically   

ad.根本地，本质地

参考例句：

• I think we may have to rethink our policies fairly radically. 我认为我们可能要对我们的政策进行根本的反思。
• The health service must be radically reformed. 公共医疗卫生服务必须进行彻底改革。

31 overestimating   

对（数量）估计过高，对…作过高的评价( overestimate的现在分词 )

参考例句：

• I think you're overestimating his abilities. 我看你对他的能力评价过高。
• With hindsight, he was overestimating their desire for peace. 事后看来，他高估了他们对和平的渴望。

32 assessment   

n.评价；评估；对财产的估价，被估定的金额

参考例句：

• This is a very perceptive assessment of the situation.这是一个对该情况的极富洞察力的评价。
• What is your assessment of the situation?你对时局的看法如何？

33 propensity   

n.倾向；习性

参考例句：

• He has a propensity for drinking too much alcohol.他有酗酒的倾向。
• She hasn't reckoned on his propensity for violence.她不曾料到他有暴力倾向。

34 innate   

adj.天生的，固有的，天赋的

参考例句：

• You obviously have an innate talent for music.你显然有天生的音乐才能。
• Correct ideas are not innate in the mind.人的正确思想不是自己头脑中固有的。

35 truthful   

adj.真实的，说实话的，诚实的

参考例句：

• You can count on him for a truthful report of the accident.你放心，他会对事故作出如实的报告的。
• I don't think you are being entirely truthful.我认为你并没全讲真话。

36 entirely   

ad.全部地，完整地；完全地，彻底地

参考例句：

• The fire was entirely caused by their neglect of duty. 那场火灾完全是由于他们失职而引起的。
• His life was entirely given up to the educational work. 他的一生统统献给了教育工作。