An innovative big data analytics method for detecting data abnormalities in business organisations

The increasing volume and complexity of customer transactions in today’s digital landscape pose significant challenges to organisations in ensuring transaction security and detecting data abnormalities. Numerous abnormalities detection techniques are inadequate in maintaining sufficient accuracy in the presence of “Big Data”, particularly in datasets with high dimensions, has the potential to impact the efficacy and precision of established techniques within a given business domain. This research presents the development and evaluation of an artificial intelligence (AI)-oriented data analytics model designed to address these challenges effectively. The primary objective of this study is to enhance transaction security and the detection of irregularities through the application of cutting-edge AI techniques. Leveraging a collaborative design science approach (DSR) with industry partners using extended action design research principles, we have elicited critical domain knowledge and decision-making insights to inform the requirements of designing a decision solution model. Our research follows DSR methodology under a constructivist research paradigm that outlines a comprehensive framework for detecting abnormalities in customer transactions data, which includes data preprocessing, feature engineering, and the application of advanced machine learning (ML) algorithms. To ensure alignment with industry-specific requirements, we have incorporated interpretability and transparency features into the proposed analytics model for serving effective decision-making process. Through experimental & observational evaluation methods, experimental data and feedback from industry stakeholders, demonstrates the model’s effectiveness. The study also highlights the model’s adaptability to various industry contexts such as clinical care and higher education using extended meta design theory and its potential to improve key performance indicators related to abnormality detection. Furthermore, this research underscores the importance of collaboration between academia and industry in developing AI-driven solutions tailored to real-world challenges. The insights gained from industry partners have been invaluable in refining the model’s capabilities and ensuring its practicality in diverse organisational settings.