After a year of ChatGPT, is academia getting to grips with generative AI?

At a recent academic summit, I was asked how universities would be using generative AI in 2030. I chuckled a little as most universities have no idea how best to handle AI this semester, let alone in seven years. Next week’s first anniversary of ChatGPT’s launch into the public domain seems an apt moment to focus on its impact on universities thus far and what we can and must do about the ongoing generative AI revolution.

The most surprising thing about the arrival of ChatGPT is how surprising it was to most people and organisations. Universities were no exception, even though most research institutions had been studying and using AI technologies for years. At Times Higher Education’s World Academic Summit in Sydney in September, even EdX’s founder Anant Agarwal confessed to having been blown away by the sophistication of OpenAI’s product. The human-like responsiveness of ChatGPT supercharged its immediate adoption beyond the tech community.

On first download, the plain white box with the send icon looked almost too simple to be the real thing. It made Google’s interface look busy. Once you got past that shock, the impressive quality of the results, the ease of prompting to refine them, and the plain fun of playing with ChatGPT’s command of genres and forms of diction propelled its use by businesses, governments, educational institutions and civilians everywhere. It has jolted universities even more profoundly than Covid-19 did, when it forced teaching online. And ChatGPT is only the most prominent of a fast-growing set of generative AI tools that are becoming available.

In the decade ahead, no university will survive without a generative AI strategy. Nor will any university prosper without leaders who are prepared to engage with its breathless evolution. The pace and productivity of university research and its commercialisation will be scaled up. Our research methods will change and throw up new ethical challenges to be solved and monitored. Our operations also stand to benefit from streamlining some overly complex administrative processes.

The most urgent impact of generative AI on universities, however, is on our teaching mission. The advent of ChatGPT and similar tools that can write and edit essays, solve maths problems, write code and teach how to debug it, tells us that tried-and-true tools of student assessment need to be transformed or even abandoned. Quickly following the earliest public release of ChatGPT, before most of us had heard of “prompt engineering”, scientists, engineers and social scientists from New York University Abu Dhabi examined assessment on 32 courses at NYUAD. They found that while students bested ChatGPT on specific questions related to mathematics and economics, as well as “trick questions”, the AI performed as well as or better than our students in nine courses, based on comparison of answers to 10 questions. Not surprisingly, those nine courses were in science and social science, but the range of topics was wide, from computer science, biology and psychology to management, materials engineering and climate change. 

Source: 

Alamy/istock montage

Our researchers also examined the reliability of two AI detection tools developed by OpenAI and by Turnitin, the company whose plagiarism discovery tool many universities have integrated into their learning management systems. This spring, Turnitin started building the tool GPTZero into its standard product. As other universities also learned, the NYUAD researchers found that both tools returned too many false positives to make them a reliable weapon against deceptive use of AI in assignments. Moreover, further experiments confirmed that it was easy to fool the detectors in the other direction: adjusting ChatGPT answers just a little to make them more humanlike or idiosyncratic almost always returned a false negative result for an entirely AI-generated product.

Generative AI detection algorithms will presumably get better and identify plagiarism and cheating with greater reliability. The real challenge of ChatGPT for universities, however, is not how to police plagiarism more effectively. The question we need to think about is whether plagiarism or cheating are even useful categories of pedagogic concern when the world is adopting generative AI at breakneck speed. If ChatGPT has undone the time-honoured system of tests, take-home exams, essays and problem sets, how should universities measure, assess and verify learning? On what skills, abilities and dispositions should we actually assess our students? What and how should we be teaching them?

For the first time in generations, universities have to create and study new feedback loops between what a graduate will need in the world, how to measure that they get it, and how to teach to it. Not even the PC and internet revolution forced universities to grapple with questions that go so close to the core of their mission.

Adapting our universities for AI will be difficult. Early research into the perceptions of generative AI held by the two key university constituencies – faculty and students – shows a generational bifurcation. Numerous anecdotal reports on “the first year of AI college” report this split, and it is substantiated by our research at NYUAD which also surveyed 1,601 professors and students in Brazil, India, Japan, the UK and the US.

The survey shows how faculty have largely reacted to ChatGPT’s arrival on campus with alarm, even as they are curious about its research potential. Their concerns are easily summarised. If generative AI detectors will not detect computer-generated contributions to student work, or if they make us accuse students falsely, how will we know whether our students are learning at all? A first impulse is often to forbid the use of generative AI in assessments, but many faculty soon realise that tactic is doomed to fail. Most faculty understand they will need to change their means of assessment, and that to do so they will need to change their pedagogic methods.

Some want to throw in the towel and leave the academy, but retirement is not an option for most. When the remainers turn to their universities for guidance on handling academic integrity and retooling their assignments in the age of ChatGPT, they find that policy development and practical training are variable across and within institutions, or even non-existent. Simply put, instructors are telling their universities: “Help me!”

Students, on the other hand, are overwhelmingly delighted by the marvels of ChatGPT. While there is some regional variation in their views, the overall response of students is to lean into ChatGPT and expect universities to deal with it. Most students grasped more quickly than their elders that they will need to get a handle on generative AI to get the kinds of innovative jobs that the Fourth Industrial Revolution and their very first employers will demand.

Indeed, consultant and research reports show that employers are overwhelmingly adopting generative AI as fast as they can, and pushing its development along. Does anyone even remember the letter signed by 1,100 tech leaders asking for a six-month moratorium on the training of AI more powerful than ChatGPT-4? That was six months ago and, meanwhile, ChatGPT-4+ and other AI products have amped up their game with visual and audio capabilities of great interest to industries from security, advertising and medicine to counselling, music and art.

Students acknowledge some ethical qualms about cheating on exams and assignments, but they say that if universities worry about plagiarism, it is their problem to solve. They are quick to point out the technology’s benefits. ChatGPT is tutor-like and helps them learn. For students for whom English is not a first language, ChatGPT levels the playing field for them. Some students say that ChatGPT helps them save time on dozens of mundane tasks and makes time for the other parts of life that matter to them: sleep, sports, hobbies, community volunteering, friends. This helps them become purposeful, healthy adults able to withstand the stress and anxiety that has become such a prominent feature of student lives today.

Students want universities to help them optimise the upside of generative AI and minimise the risks. They want professors to teach them how to use it well. In short, students are also telling their universities: “Help me!”

The second year of generative AI on campus may still be chaotic, and the divide between faculty and student concerns makes consistent policy development illusory for now. However, universities can weather this period. They are full of creative people, and they have had to adapt to massive disruptions over the centuries. Many professors are excited to experiment with generative AI as a matter of research and professional development, and they are committed to the well-being and success of their students. Some remember the early days of the Covid-19 pandemic, when lecturing and standard assignments proved ineffective online and many faculty at last learned to flip the classroom.

At NYUAD, we were fortunate that the director of our Center of Teaching and Learning is a scholar with a keen interest in generative AI with a hopeful, can-do demeanour. She is helping our faculty lean into their interests in achieving the outcomes they want for their students in new ways. A finance professor, Raša Karapandža, immediately adjusted his approach to classroom activities and assessments, for example. In his course on fintech innovation and regulation, large language models are routinely used as teaching and learning tools. Assignments require students to conceptualise, design and execute machine learning and blockchain projects – and then make them look good using stable diffusion, a text-to-image model. In the past, students with little or no background in software development may have underperformed in such tasks compared with computer science majors. Now, Karapandža is observing improved collaborative learning and student outcomes for all his students, from finding problems and biases in code to becoming ingenious prompt engineers and creators of exceptional working prototypes.

Universities have always taught the skills and ideas needed by society at any one time – sometimes a little too slowly, sometimes perhaps too fast. We now must expand our basic remit of promoting understanding of the human, natural and cultural worlds with producing fluency in generative AI. We cannot be careless techno-optimists about it, but here, too, universities have resources in their departments of philosophy, law, political science, anthropology and the humanities at large.

Their task is daunting but also inspiring. As our students will bear the burden of making the most of generative AI while fighting its dangers, we must teach them the ethics of putting these technologies in the service of humanity, the Earth and all living things. A year into this task, it is fair to say that we have made a decent start on this formidable challenge.

Mariët Westermann is vice-chancellor of New York University Abu Dhabi.