But she can’t do math
AI software ChatGPT passes law exam
How good is artificial intelligence? Based on machine learning, the ChatGPT software can write texts and solve tasks. Now she has even passed a law exam – but she has difficulties with math.
Published: 01/26/2023 at 09:07
The AI software Chat GPT is based on «machine learning» and can write texts.
ChatGPT is very trendy: The AI software should be able to understand, write and summarize texts. But she can do even more: ChatGPT passed a law exam at the University of Minnesota.
In a research paper by law professor Jonathan Choi on Monday, Choi and colleagues describe that the artificial intelligence (AI)-based software achieved an overall pass mark on tasks in constitutional law, tort law and tax law.
ChatGPT got the same test from Choi as the human fellow students. It consisted of 95 multiple-choice questions and twelve essay tasks.
Problems with open questions
“When writing essays, ChatGPT demonstrated a good understanding of basic legal regulations and demonstrated solid organization and composition throughout,” the authors wrote. In addition, the AI quoted accurately. But the bot “often had difficulty identifying problems with open-ended tasks, a core competency in legal exams.”
Although the test was graded blindly, according to Choi, two out of three examiners had an idea which exam answers came from ChatGPT. The grammar was perfect, but the explanations were “a bit repetitive,” Choi wrote on Twitter.
ChatGPT sucks at math
As an independent student, Choi explained that ChatGPT was “not great”, and he even “failed” in mathematical tasks. However, his research group assumes that language models such as ChatGPT “in cooperation with people” could be very useful for law students taking exams and for practicing lawyers.
ChatGPT from the US company OpenAI uses artificial intelligence to generate text. Users can specify individual commands or sentences, which the system then supplements independently based on vast amounts of data from the Internet.
The system, which is based on machine learning, reacts to user feedback in a similar way to a chatbot. Among other things, it writes computer codes, poems and user manuals. In some schools in the USA the use of the software is already prohibited.