Seesame: Automated Monitoring of Legal Documents

Seesame partnered with KInIT in an effort to improve the efficiency of retrieving information from Slovak legal documents relevant for Seesame’s business clients, and presenting them in a way each client can understand easily.

As an established Slovak PR company, Seesame, among other activities, monitors proposed changes to the state legislation that could affect the business of a variety of clients (such as a supermarket chain), including the tax policy, environmental law or construction law. This task typically involves sifting through vast volumes of legal documents, which are often difficult to interpret, and then presenting the information in a clear and concise format. Now, imagine having to repeat this process for a multitude of clients, each having their own priorities of what matters most. This is a laborious and highly repetitive task. Not to mention, the clients are unlikely to tolerate lengthy delays in receiving critical information.

To address this, we employed AI, specifically a large language model, to automate the process of detecting relevant documents and content therein for a particular client. The model is provided with a description of the client (background and topics of interest) and a legal document, and determines whether the document as a whole is relevant or not. If the document is relevant, the most relevant paragraphs are extracted from the document. As the second step, the extracted relevant content is summarised and simplified, likewise using a large language model.

To support this workflow, we developed a web application designed for easily setting up the client description and uploading documents. The application then detects relevant content in the documents and summarises the content. Besides PDF and DOCX as the supported input document formats, users can also submit media monitoring alerts in the form of exported email notifications (EML files), in which case documents will be retrieved automatically from links contained in the alerts.

While initial results on a historical set of documents proved promising in terms of accuracy (reported as satisfactory), another round of testing of the application, this time on a fresh set of documents, revealed major deficiencies. Seesame employees responsible for testing the application (having a varying degree of expertise in determining relevance) unanimously stated they were required to re-run the application and/or correct the relevance by hand. Thus, no significant reduction in processing time and manual labour was reported either. A particular feature that was perceived as sorely lacking was the ability to engage in a conversation with the language model in order to nudge the model into, for example, extracting different contents from parts deemed more relevant.

Despite the results falling short of Seesame’s expectations, the collaboration allowed to lay solid groundwork for more efficient future research in this domain, such as fine-tuning using historical data or experimenting with newer generations of large language models. Seesame has also garnered valuable insights into designing a user-friendly AI-assisted application.