Saul Perlmutter: ChatGPT essays show students how not to think

Students should be encouraged to consider how ChatGPT-generated essays fall short of the genuine critical thinking required for great science, a Nobel prizewinning astrophysicist has argued.

Asked how educators should deal with the rising tide of AI-created content submitted by students, Saul Perlmutter told delegates at the annual Lindau Nobel Laureate Meeting in southern Germany that he used ChatGPT answers as a teaching prompt.

His students are invited to dissect robot-written scripts to understand the limitations in ChatGPT’s reasoning and compare it with more robust scientific analysis, he said.

“We ask students to use it as a trigger – to see what is wrong with it, and why it did not achieve the type of scientific thinking it was trying to achieve,” explained Professor Perlmutter, professor of physics at the University of California, Berkeley who won the 2011 Nobel Prize in Physics for discovering, alongside Brian Schmidt and Adam Riess, that the universe’s expansion is accelerating.

Since 2018, Professor Perlmutter has run a critical thinking class at Berkeley – titled “Sense and Sensibility and Science”, which aim to instil the precepts of scientific thinking across different disciplines and discourses.

Noting how political discourse had an “emphasis on overconfidence”, Professor Perlmutter said he was keen for students of all types of reflect on the inherent uncertainty that scientists faced when carrying out experiments.

“As scientists we have learned it is very easy to see patterns, repetitions in noise or correlations that are not there – we have to learn about how easy it is to fool ourselves,” he continued.

“Almost everything [in science] comes with a sense of probability and we need to become more comfortable with that.”

The US astrophysicist, whose recent book Third Millennium Thinking outlines the principles behind his Berkeley critical thinking course, said he was disappointed that just one US congressional representative had a PhD in physics, but he hoped a broader appreciation of how scientists think could help inform political debate, which was often based on dogma similar to “religious discussions”.

“My goal is that every member of parliament or congress [in every country] can use these ideas when talking to each other,” he said.

Given the polarisation of political debate – both in Congress and online – it was becoming increasingly difficult for those with different ideological views to converse, said Professor Perlmutter.

However, scientific discourse offered a potential model for resolving disputes, he suggested.

“People run off to their corners and it’s difficult to have those conversations – to disagree well. Scientific tradition is very good at getting people to disagree – whether in debate or peer review,” he said.

Contrasting science with politics, he added: “There is also a humility if you get something wrong but [that it is good], because at least there is a desire to get the answer right.”

jack.grove@timeshighereducation.com