Seesame: Ethics-based Assessment of a Sentiment Analysis Tool
Natural Language Processing (NLP) is a rapidly growing field within AI. Applications such as sentiment analysis unlock features that are attractive for various industry partners. KInIT has been collaborating with the PR agency Seesame on understanding social media conversations through AI by creating sentiment analysis tools. At KInIT, we are well aware of the potential benefits of sentiment analysis such as improved content moderation, but we also know that it comes with risks. We strive to develop and use such systems responsibly and in accordance with ethical principles of trustworthy AI. Therefore we decided to assess one of the sentiment analysis tools developed collaboratively by KInIT NLP team and Seesame from an ethical perspective.
Our research team Ethics and Human Values in Technology has launched a series of collaborative workshops where researchers from Seesame and KInIT had the opportunity to discuss the moral and societal consequences of sentiment analysis tools operating with real-world user data.
Following the Assessment List for Trustworthy AI , augmented with our own approach to the assessment process, we have systematically mapped out the direct and indirect stakeholders that could be affected by the system.
This led us to identifying various risks in domains of explainability, accuracy or human autonomy that could emerge from using sentiment analysis on social media platforms (e.g. Facebook). During the workshops we also established the mitigation process of such situations in real life.
Dealing with ethics in AI is however not a linear process – it calls for constant iterations, ethical considerations and consequences evaluation at every stage of the development process.
If done correctly, ethics-based assessments can result in a set of countermeasures and actionable steps with assigned responsible roles. This way, values such as privacy and transparency are directly translated into the design of the technology and ethical risks are minimized.
The workshop had a good structure, we went from identifying the stakeholders and their issues to specific tasks that will be transferred to the product backlog. We are now more aware of the ethical impacts of decisions in software development. We no longer focus on the legal aspects only, we also consider the impact that software could have on people’s lives if, for example, such data were leaked.Juraj Caránek, Head of Corporate Team & Partner
I think the ethical assessment is very valuable for both developers and business owners. It could be incorporated by different types of organizations into the design of their projects to provide additional benefit to their clients.
Our values are firmly embedded within our company culture, and respecting ethical norms is very important to us. We believe that living our values is one of our competitive advantages so we strive to build our values into our services as well.
Juraj Caránek, Head of Corporate Team & Partner
Ethics and Law Specialist