Xolution: How to transform business responsibly

One of the greatest promises of AI is to achieve more effective business processes and to minimize dull and repetitive work. Deploying such systems in process automation may help simplify complex tasks and ultimately automate some businesses to save money and increase revenues. However, such a bright vision brings considerable ethical and societal questions that need to be answered before such systems will begin to transform our markets completely.

Our ethics team had a chance to facilitate a series of expert workshops during which we conducted an ethics-based assessment of X-bot, a sophisticated AI system designed for comprehensive process automation, from simple tasks to the most complex ones involving asynchronous activities and unstructured data. This system doesn’t just automate individual tasks; it models entire processes, enabling comprehensive optimization and streamlining. It can also handle and analyse various types of unstructured data, such as emails, tickets, chat conversations, and manuals, gaining a comprehensive overview of the process by utilising narrow AI tools, such as classifiers, entity extraction, taxonomical and ontological models, and expert systems, to analyse data, categorise information, and manage processes.

The ethics-based assessment of X-bot consisted of six facilitated meetings, held from January to May. This process involved seven team members from the Xolution company and three KInIT specialists on trustworthy AI, who served as the facilitators.  Participants in the workshops had varying expertise and held different positions within the company (CEO, CTO, AI team lead, solutions architect, senior developer, senior researcher, and tester), which was favourable due to the different perspectives they were able to provide in the process of ethics-based assessment. 

Before the assessment, the Xolution team proactively identified several possible ethical concerns about their AI system. These included potential disruption of free market principles, like stifling competition and diversity. They also identified concerns about losing control of the system, the ethical implications of underlying models and the challenges of explaining or predicting its behaviour properly. Together we framed these concerns and addressed them in the broader context of the seven requirements of trustworthy AI. Importantly, we analysed the ethical and societal issues of X-bot with respect to their impact on various direct and indirect stakeholders. Besides the direct stakeholders such as end users, „old fashioned“ companies, „Facebook era“ companies, startups or specialist staff of clients, there were also many indirect, but nevertheless strongly affected stakeholders. Among the most prominent indirect stakeholders were relatives of end users and the competition of Xolution clients.

After the assessment, it is evident that the development, deployment, and use of the X-bot may bring some ethical and societal risks. Most of the ethical and societal issues were tied to Privacy and Data Governance (mainly regarding privacy rights and the chilling effects), Technical Robustness and Safety (regarding accuracy issues, system errors, or problems with uncertainties) and Human Agency and Oversight (mainly regarding over-reliance and the so-called “Eliza effects”). There are also some considerable risks regarding Diversity, Non-discrimination and Fairness (potential discrimination or issues with the universality of design) and Societal and Environmental Well-being (potential de-skilling or issues in communication regarding work impacts).

Regarding the ethics-based assessment, Xolution expressed a strong desire to identify and mitigate ethical and societal risks of its system and its impact on stakeholders. The company fully cooperated during the assessment process, including the preparation and evaluation phase. Their enthusiasm for tackling different challenges and addressing tough questions led to a smooth and productive discussion during the workshops. The next step in mitigating ethical and societal risks identified in the ethics-based assessment should be to focus on the most imminent risks from the risk list and to validate proposed countermeasures.