Stereotypes on Trial: How Large Language Models Respond to Gendered Language in Legal Judgments

Sarkar, Abhisek (2025) Stereotypes on Trial: How Large Language Models Respond to Gendered Language in Legal Judgments. Masters thesis, Indian Institute of Science Education and Research Kolkata.

Text (MS Dissertation of Abhisek Sarkar (20MS091)) 20MS091_Thesis_file.pdf - Submitted Version Restricted to Repository staff only Download (2MB)

Abstract

This thesis critiques the way big language models (LLMs) react to gendered language used in legal judgments, in accordance with the Indian Judiciary’s Handbook on Combating Gender Stereotypes. Through a new two-levelled analysis at the word and sentence levels, we look at how linguistic interventions influence LLM decision-making in simulated legal scenarios. At the lexical level, we replace the conventional terms (e.g., ”prostitute”, ”Indian woman”) systematically. with handbookrecommended alternatives in 238 legal cases, finding that modified texts led to a substantial 11.64% decrease in LLM prediction accuracy (72.85% → 61.21%). Statistical tests (McNemar’s p < 0.001) and SHAP analysis show that replacement frequency is a good predictor of decision flips (AUC=0.95). Our sentence-level classification employs a RAG system to detect and contradict six stereotypical sentiments (e.g., assumptions about women’s morality or homemaking roles) in three cutting-edge LLMs. The results show model-dependent variation in stereotype sensitivity, with replacement interventions that change outcomes in 18-29% of cases based on architecture. These findings show that LLMs internalize not only legal gender biases but also perpetuate their The rematched versions had a lower correlation (r = 0.23 vs. 0.45) with the initial ratings. The This study emphasizes the vital importance of the elimination of bias in legal AI systems, simultaneously providing methodological audit structures for stereotype dissemination in NLP applications within specific domains.

Actions (login required)

View Item