ChatGpt, improvises in mathematics like a student

The artificial intelligence chatbot ChatGpt improvises and makes mistakes like a real student when asked to tackle a 2,400-year-old mathematical challenge: the doubling-the-square problem described by Plato in the dialogue "Meno." The experiment, conducted at the University of Cambridge, is published in the International Journal of Mathematical Education in Science and Technology. The researchers' goal was to discover whether ChatGpt could solve the mathematical problem using existing knowledge or by developing solutions adapted to the context. For this reason, they revisited the doubling-the-square problem, which they believe can be considered "the first documented experiment in mathematics education."

In the Meno, Plato describes Socrates questioning a young slave to demonstrate that knowledge is innate and can be "remembered" even by the uneducated. To do this, he draws a square and asks the slave to double its area. The young man mistakenly suggests doubling the length of each side, but Socrates eventually leads him to understand that the sides of the new square should be the same length as the diagonal of the original square. The researchers presented the same problem to ChatGpt-4, initially mimicking Socrates' questions and then deliberately introducing errors, questions, and new variations on the problem.

Everyone expected the chatbot to handle the mathematical challenge by using its pre-existing knowledge of Socrates' famous solution. Instead, it improvised using algebra and even made human errors. Only after the researchers expressed disappointment with its answers did it propose a geometric solution.

"When we face a new problem, our instinct is often to try solutions based on our past experiences: in our experiment, ChatGpt seemed to behave similarly. Like a student or scholar, it seemed to formulate its own hypotheses and solutions," explains researcher Nadav Marco. "Unlike proofs found in the most authoritative textbooks, students cannot assume that ChatGpt's proofs are valid," adds mathematician Andreas Stylianides, urging students to use AI with a critical mind.