A 70-year-old lesson reemerges in OpenAI’s development of GPT-4

In a recent episode of his Two Minute Papers series, Dr. Károly Zsolnai-Fehér delves into the captivating capabilities of GPT-4, an AI model that exhibits a spark of intelligence. This AI model has left many in awe with its ability to create drawings based on textual descriptions and solve complex mathematical problems comparable to those found in the International Mathematical Olympiad.

Complex Capabilities

One of the most striking aspects of GPT-4 is its capacity to create intricate drawings despite never having seen an image. This capability stems from its ability to understand and interpret textual descriptions. Dr. Zsolnai-Fehér demonstrates this by providing a textual description, which GPT-4 uses to generate a detailed drawing.

Mathematical Problem-Solving

GPT-4’s performance in solving complex mathematical problems is also noteworthy. The model can tackle problems that would be challenging for even the most skilled mathematicians. However, as Dr. Zsolnai-Fehér highlights, GPT-4’s performance varies depending on how a problem is presented to it.

A Step-by-Step Approach

When provided with a step-by-step prompt, GPT-4 accurately listed and counted prime numbers between 150 and 250. However, when asked directly to count the prime numbers within the same range, GPT-4 failed to give the correct count. Dr. Zsolnai-Fehér attributes this inconsistency to the AI’s inability to think in a step-by-step manner unless explicitly directed to.

Limitations and Inconsistencies

Despite its impressive capabilities, GPT-4 struggles with solving simple arithmetic problems – a task that can be easily accomplished by a child. This inconsistency raises questions about the model’s ability to understand basic mathematical concepts. However, GPT-4 excels in areas such as:

• Storytelling: GPT-4 demonstrates exceptional storytelling capabilities, generating coherent and engaging narratives.
• Reading and Evaluating Electrocardiograms: The model showcases its ability to read and evaluate electrocardiograms with impressive accuracy.

Classical Thought Experiments

GPT-4’s performance in solving classical thought experiments with logic is also noteworthy. Dr. Zsolnai-Fehér uses the example of the "Two Enigma" thought experiment, which GPT-4 successfully solves using logical reasoning.

The Struggle with Time and Date

Despite its impressive capabilities, GPT-4 struggles to correctly identify the current year. This limitation highlights the model’s vulnerability to specific types of knowledge and information.

The Deployment of GPT-4 in ChatGPT

Dr. Zsolnai-Fehér ends his discussion by highlighting the deployment of GPT-4 in ChatGPT, enabling users to experience the future of AI. He marvels at how millions of users interact with AI technology, often without realizing it.

The Future of AI Research

Despite its shortcomings, GPT-4 symbolizes the incredible strides being made in AI research. Dr. Zsolnai-Fehér’s discussion provides a fascinating glimpse into the capabilities and limitations of this cutting-edge AI model.

Additional Resources

• The paper ‘Sparks of Artificial General Intelligence: Early experiments with GPT-4’ is available here: https://arxiv.org/abs/2303.12712
• Dr. Károly Zsolnai-Fehér’s Twitter account can be accessed at: https://twitter.com/twominutepapers
• More information about Dr. Zsolnai-Fehér and his work can be found on his website: https://cg.tuwien.ac.at/~zsolnai/