AI Unable to Tap Into Human Sensibilities
Each case reviewed in the study was designed to be ambiguous in order to evaluate the level of decision-making. However, this did not seem to affect GPT in its rulings either.
In an attempt to bridge the gap between the AI and human judges, the researchers prompted GPT to think beyond simply following the letter of the law. This involved tweaking the information fed into the bot with the aim of replicating the responses of the human judges.
The researchers trained the AI on legal realism theory, which suggests that judges use factors outside of the law to determine a ruling, such as emotional and social contexts. They also urged GPT to think more broadly about justice beyond the law, and even explicitly told it to consider sympathetic defendants in a different light.
Despite these adjustments, the AI judges remained focused on legal precedent, and did not consider the emotional aspects of a case. Researchers concluded this to be an example of legal formalism, where rules and precedents are followed with no regard to personal feelings.
