Uncovering the dark side of AI-based decision-making: A case study in a B2B context

Abstract

Over the last decade, many organizations worldwide have been assimilating Artificial Intelligence (AI) technologies to increase their productivity and attain a competitive advantage. As with any technology, intelligence systems come with potential downsides. Despite the efforts to mitigate any negative consequences of AI, businesses and employees continue to confront the dilemmas of adopting AI, so it is essential to explore in detail the rising concerns around such technologies. In this paper, we used a single case study to investigate the dark aspects of AI in a Norwegian energy trading firm. We gathered data through semi-structured interviews and secondary data. Specifically, we interviewed AI managers, traders and developers who have worked on deploying and using AI tools over the last three years. Our aim is to identify the dark side of AI use in trading, how AI trading bots affect the relationship between traders and AI developers and how the firm adjusts to this new reality. The findings indicate that negative or unintended consequences of AI can be grouped into three clusters related to (1) the nature of the work; (2) conflicts and effects; and (3) responsibility. The paper concludes with future research and practical implications that can help organiziations mitigate the negative aspects of AI use.