Pitfalls of impression-based technology policy analysis

In late 2022, OpenAI released a transformer-based large-scale language model (LLM) called “ChatGPT.” Contrary to OpenAI staff's expectations, ChatGPT became the fastest growing web-based app in history, increasing the number of active users to his 100 million in two months (more than only Meta's Threads) . ChatGPT's initial public impressions were both of a sublime quality and a portent of doom. In February 2023, Henry Kissinger, Eric Schmidt, and Daniel Huttenlocker proposed that generative artificial intelligence (AI) is comparable to the intellectual revolution started by the printing press, this time integrating humanity's storehouse of knowledge and creating a ” Then he writes. In March 2023, Eliezer Yudkowsky foresaw extinction-level risks and implored the world's governments and militaries to halt AI projects and “be willing to destroy rogue data centers with airstrikes.” .

Although these first impressions represent opposite ends of the spectrum, reasoning that occupies the space between them is common in technology policy analysis. In other words, personal impressions of generative AI permeate the assumptions underlying the policy analysis. When fundamentally important assumptions are left unchallenged, it is easy to fall into the trap of extrapolating future technological wonders from the current state of technology. Technology policy analysts from all walks of life are doing great work, but it's time to identify the gaps in our reasoning and aim higher, individually and collectively.

An example illustrates the general trend. Paul Scharre of The Center for a New American Security hedges about the future of AI in his book Four Battlegrounds (a treasure trove of insights overall), but he leans toward the idea that: . This results in a more robust model. Multimodal datasets can be useful for building models that can relate concepts expressed in multiple formats, such as text, images, video, and audio. This expectation is based on the idea that scaling up AI systems (increasing their internal capacity and training datasets) will give them new capabilities, and is based on Richard Sutton's article on the benefits of such technology. It positively references the famous argument in “The Bitter Lesson.''

Shortly thereafter, Microsoft researchers helped set the tone for a series of overly optimistic claims about the future of LLM in their provocatively titled paper on GPT-4, “Sparks of Artificial General Intelligence.” . It's not hard to see how personal impressions of GPT-4 can lead to an equivalent feeling that “we're on the brink of something big here.” However, this does not justify allowing the assumptions tied to this sentiment to deteriorate in the analysis.

Extensive research has revealed the limitations of LLM and other Transformer-based systems. Hallucinations (authoritative but factually incorrect statements) continue to plague LLM, with some researchers suggesting that they are simply an innate feature of the technology. A recent study found that voters using chatbots to get basic information about the 2024 election are more likely to be misinformed about phantom polling places and other false or outdated information. There is a possibility that you will get it. Other studies have shown that LLMs lag behind human abilities in forming abstractions and generalizing them. The reasoning power of multimodal systems is a similar story. OpenAI's latest development, the text-to-video generator Sora, is good at realism, but it creates objects and people out of thin air and doesn't adhere to real-world physics.

So much for the idea that new modalities like images and videos will lead to the reliable, robust, and explainable AI systems we desire.

None of this suggests that there is only The hype in the world of technology.Carnegie's Matt O'Shaughnessy rightly points out that talk of “superintelligence” can have a negative impact on policymaking. because Describe the basic limitations of machine learning. Furthermore, while the Biden administration's sweeping executive order on AI issued in October 2023 dramatically invokes the Defense Production Act to authorize surveillance of certain computationally intensive AI systems, the tone of the It was more diverse than I expected.

But the problem we've identified here is not one of hype itself. What is Hype? result Falling into a framework of analysis that is too easily ignored in favor of easy publications and personal or organizational self-promotion. Lest you mistakenly believe that this is simply an idiosyncratic tendency peculiar to his LLM, the disappointment of AI-powered autonomous drones on the battlefields of Ukraine is about the alleged rapid emergence of radical advances in 2023. That should raise some eyebrows. Moreover, it is easy to find nuances. But at the same time, there seems to be little personal or collective reflection as we begin to think that the future of its crown jewel, quantum computing, will be downgraded.

Nevertheless, today's generative AI is beginning to look like a parody of Mao's Continuing Revolution. To transform this technology into a “general” intelligence like humans, or any other technological wonder of imagination, he always needs one model upgrade, which is not allowed. succumbing to challenges from regulatory bodies and popular movements;

Importantly, policy analysts make choices when evaluating technologies. Choosing a particular assumption over others presents the analyst with a particular set of possible policy options. at the expense of others. Individuals' first impressions of new technologies are inevitable and can be a source of diversity of opinion. The problem with policy analysis is that practitioners fail to pour their first impressions (or second impressions, third impressions, etc.) into a shared melting pot that exposes unstable ideas to hot intellectual criticism, thereby making concrete This occurs due to the inability to guide the clarification of policy issues. Solutions can be derived without unduly ignoring other possibilities.

Policy analysis typically combines elements from industry, domestic politics, and international affairs.Simply identifying that a policy challenge exists is not enough de novo But it is from an intuitive link between a society's needs and values ​​and the expected or actual impacts of developments within its borders or abroad. This intuition is something we all have and should be the focus of our honest and shared scrutiny.

Vincent J. Kalkidi He is a non-resident scholar in the Strategic Technology and Cyber ​​Security Program at the Middle East Institute. He is also a member of America's Next Generation Initiative Foreign Policy 2024 Cohort.