LLM Based Multi Agent System for Data Annotation and Courtroom Simulation

Majumdar, Gourish (2025) LLM Based Multi Agent System for Data Annotation and Courtroom Simulation. Masters thesis, Indian Institute of Science Education and Research Kolkata.

Text (MS Dissertation of Gourish Majumdar (19MS056)) 19MS056_Thesis_file.pdf Restricted to Repository staff only Download (1MB)

Abstract

Multi Agent Systems are experiencing a recent surge in interest owing to the development of large language models.capabilities of LLM powerd agents are being tested in Education ,Management, Legal and Medical domains to automate workflows .This will ease up workload on human experts and also save some costs. Multi Agent Systems have been found to be performing better than single agent at complex tasks. They solve difficult problems by coordinating, collaborating and even contradicting each other. In this regard, many such coordinating strategies have been developed like MAD(Multi Agent Debate), PREDICT,etc.While all these strategies are great in their own accord they also have their own limitations.There is no Multi Agent setup which performs the best in all kinds of situations and all kinds of problems.Here we have created our own customMulti Agent setups to tackle 2 new problems namely-1)US Court Room Simulation 2)Data Annotation which hasn’t been done before to the best of our Knowledge.While agentic systems have been used in legal domain,the prime focus have always been judgement prediction.Very little has been done to study the argument and response generation capabilities of LLMs in this kind of setting.We have simulated a courtroom conversation between judge petitioner and plaintiff based on real cases from the US court and successfully shown that they can generate the arguments and responses pretty close to what had happened in the real case.We have also shown how this conversations can be improved by using few-shot, Guided-fewshot, and fine tuning.For Data Annotation task, we have shown by breaking down a simple problem into multiple smaller parts and assigning each of these parts to agents, far better results than previous benchmarks have been obtained.

Actions (login required)

View Item