ChatGPT narrowly passes US law school test
Even though ChatGPT passed the test, the bot was not a very good law student.
The ChatGPT conversational robot, created by the Californian start-up OpenAI, has passed the exams of an American law school. The artificial intelligence tool has written dissertations on topics ranging from constitutional law, to taxation, to torts.
The prowess of the conversational robot has aroused, since its launch in November, admiration, but also many concerns about its use for malicious purposes. The tool is powered by masses of data from the web and is particularly capable of generating realistic texts in response to simple questions.
ChatGPT is currently free, but the company OpenAI, which develops the tool, is working on a paid version.
Some results have been so compelling that faculty members at several universities are concerned about the risk of widespread cheating and the end of traditional classroom teaching methods.
Jonathan Choi , a professor at the University of Minnesota Law School, gave ChatGPT the same test that students take for their graduation—95 multiple-choice questions and 12 capstone questions across four subject areas.
In a scientific paper published on Monday, Choi and his co-authors said the robot had an overall grade of C+.
While this rating is good enough to pass, the robot finished near bottom of the class in most subjects, according to academics.
In essays he wrote, ChatGPT demonstrated that he mastered the basic legal rules and that its organization and composition were still sound, the article reads.
However, the chatbot often had trouble spotting problems when asked an open-ended question, which is an essential skill in law school exams, the research team added.
Authorities in New York and other jurisdictions have banned the use of ChatGPT in schools. Even at the University of Montreal, the rector has sent a communication to students in which he insists that the use of the software is prohibited for school work. But Choi believes the bot could be a great teaching aid.
“Overall, ChatGPT doesn't is not a very good law student when acting alone.
— Jonathan Choi, Professor, University of Minnesota Law School
We believe that working with humans, language models like ChatGPT can be very helpful to law students taking exams and practicing lawyers, he added.
To allay concerns about cheating, Mr. Choi also indicated that two out of three proofreaders had spotted the work written by the chatbot.
[These people] had a hunch and their hunch was right, because ChatGPT had perfect grammar and was somewhat repetitive, he wrote.