The Complexities of Measuring AI ROI

In the race to embrace artificial intelligence, organisations face a challenge: while AI promises benefits, measuring its actual return on investment seems complex. A recent study from Dataiku found that while 65% of organisations report positive returns from their generative AI investments, the overall ROI from data, analytics, and AI initiatives remains surprisingly static.

Download Now: Free AI Strategy Playbook
[New 2025]

This disconnect between enthusiasm and measurable returns raises questions for business leaders. Are we capturing the value of AI investments? How do we reconcile the promising feedback from early adopters with the broader plateau in returns? As companies worldwide pour resources into AI initiatives, understanding and quantifying AI’s business impact is essential.

Traditional ROI metrics may no longer suffice in evaluating AI’s multifaceted impact on business operations. While financial returns remain crucial, AI’s strategic value extends beyond the balance sheet—from enhanced decision-making capabilities to improved customer experiences and operational efficiencies. This complexity demands a new framework for measuring success that captures immediate financial returns and long-term strategic advantages.

This analysis explores how organisations can confidently invest in AI, understand its value, and measure AI ROI.

In this article, you’ll read about:

• Key Factors Driving AI Return on Investment
• Measuring and Tracking ROI
• Maximising AI ROI: The Path Forward

---

Key Factors Driving AI Return on Investment

Several interconnected factors influence the path to achieving meaningful returns from AI investments, which organisations must carefully consider and manage.

Strategic Alignment and Business Objectives

Success in AI implementation begins with precise business objectives that align with organisational strategy. Rather than adopting AI for its own sake, organisations must identify specific business challenges or opportunities where AI can create measurable value. This requires a clear roadmap that defines expected outcomes and establishes metrics for success.

Data Infrastructure and Quality

The quality of organizational data is the biggest anticipated challenge to AI strategies in 2025 (85%). According to KPMG AI Pulse Survey findings, organisations recognise this reality, with business leaders prioritising scalability (66%) and technological expertise (61%) when selecting AI providers. This focus reflects an understanding that enterprise-grade solutions require robust data governance and management systems to deliver consistent results.

Jacob Axelsen, AI Expert at Devoteam in Denmark, emphasises the need for a governance platform to manage the lifecycle of AI assets:

Talent Distribution and Adoption Patterns

Current adoption for workspace AI patterns reveal notable disparities in AI utilisation across organisational hierarchies. KPMG’s Survey indicates that while 71% of senior leadership actively uses generative AI tools, only 26% of middle managers and 15% of entry-level employees use these technologies. This adoption gap presents a significant challenge, as only 24% of employees currently use AI tools in their workflows.

Implementation Strategy and Best Practices

Organisations achieving positive ROI typically follow a structured implementation approach that includes:

• Responsible AI Culture Development

Organisations must build use cases and minimum viable products while focusing on governance and establishing appropriate guardrails. This includes developing factory like delivery process and implementing change management processes.

• Systematic Auditing

Regular auditing helps protect businesses from risks such as copyright infringement and proprietary data leakages. This process needs to be integrated into the AI implementation lifecycle rather than treated as an afterthought.

• Expertise Centralization

Creating a Center of Excellence supports employee upskilling in AI technologies. This central will gather experts for building AI applications.

• Controlled Innovation

While encouraging ideation across the organisation, successful implementations start with an early deployment of a secure solution. This prevents the proliferation of untested and unauthorised AI tools that could undermine ROI.

• Data Management Capabilities

Generating dynamic data through AI systems, including tables, code, and images, requires organisations to develop agile approaches to managing data sources and technology infrastructure.

These factors demonstrate that achieving positive ROI from AI investments requires a balanced approach considering technical, organisational, and human elements. Success depends on selecting the right technology and creating an environment where AI can deliver sustainable value while managing associated risks.

Download Now: Free AI Strategy Playbook
[New 2025]

Measuring and Tracking AI ROI

AI ROI KPI Framework

This  KPI framework helps organisations measure and optimise their AI investments across six dimensions.

The metrics span financial impact (ROI and cost savings), operational efficiency, customer experience, workforce productivity, AI adoption, and risk management.

By tracking these indicators, organisations can evaluate AI project success, identify areas for improvement, and make data-driven decisions about future AI investments.

Financial Metrics

• Return on Investment (ROI): Measure the profitability of AI investments by comparing the gains to the costs.
• Cost Savings: Track reduction in expenses due to automation or efficiency gains from AI.

Efficiency Metrics

• Time To Market: Evaluate the speed of developing and deploying new products or services with AI
• Process Efficiency: Track improvements in process performance and cycle times with AI-driven optimization.
• Automation Rate: Measure the degree to which manual tasks are automated using AI-powered systems.
• Productivity: Gauge how AI tools impact the productivity and efficiency of the workforce.

Qualitative Metrics

• Customer Satisfaction: Evaluate how AI impacts customer experience and satisfaction, through surveys and other feedback channels.
• Customer Retention: Monitor the ability of AI solutions to improve customer loyalty and reduce churn rate.
• Improved Customer Service: Measure how AI enables quicker and more accurate response to customer inquiries.

However, measuring AI assets’ return on investment (ROI) is challenging because AI’s impact is often difficult to quantify.

In particular, there is the phenomenon of productivity leakage, as Jakob Axelsen explains:

Examples of AI Use Cases and KPIs

According to the IDC study, companies see significant returns on their GenAI investments, with every dollar invested yielding a 3.7x return across different industries and geographical regions. Financial services lead in ROI, followed by the media and telecommunications sectors, with GenAI generally outperforming traditional AI investments across all industries.

Notably, 43% of organisations reported that productivity-focused applications, particularly those enhancing individual employee efficiency and reducing task completion times, delivered the highest ROI among all AI use cases.

According to Olivier Mallet, AI projects fall into three broad categories with distinct ROIs:

1. Augmented Office Worker uses tools like Gemini to improve daily work (emails, summaries, translations). Productivity gains are modest, between 0.5% and 5%, with a phenomenon of “productivity leakage” where these gains do not necessarily translate into measurable business impact. The cost was around €20 per user per month until the beginning of 2025. Google Workspace now includes Gemini AI. Microsoft is bundling its AI-powered Office features into Microsoft 365 subscriptions as well.

2. Augmented Developer is a more promising particular case. Using tools like GitHub Copilot generates productivity gains of 10% to 30%, with variations depending on the developer’s level of experience. Companies understand this use case well, but it raises questions about the potential impact on employment.

3. AI-based Applications represent the most significant segment. For example, chatbots for customer service can generate productivity gains of 40% to 100% for call center teams, with a notable improvement in customer satisfaction. Intelligent document processing can even achieve 500% to 1000% for specialised teams.

Download Now: Free AI Strategy Playbook
[New 2025]

Examples of AI ROI KPI

When Devoteam deployed a secure chatbot platform based on AWS services within a media group, we used several KPIs to assess the service’s adoption and quality:

• Adoption KPIs:
  • Number of registered users
  • Number of active users per day
  • Number of sessions and number of exchanges per day

• Quality of service KPIs:
  • Uptime/availability of the service
  • Evaluation of quality scores measured on conversations in production:
  • Ability to respond in a relevant manner
  • Ability to respond to requests / expected features
  • Ability to retain information

However, it is difficult to estimate all gains, such as reducing the risk of data leakage or the overall increase in productivity across all use cases (specific media use cases, access to enterprise drive, contextual access rights).

Establishing Performance Metrics

Successful AI implementation requires establishing clear metrics early in the process. These key performance indicators (KPIs) must align with specific business objectives and provide a framework for measuring immediate and long-term impact. Essential metrics typically include:

• Cost savings and operational efficiency
• Revenue growth and new revenue streams
• Productivity gains
• Customer satisfaction scores
• Time saved through automation
• Marketing effectiveness
• Operational efficiency improvements

Quantifying Direct and Indirect Benefits

The immediate benefits of AI implementation often manifest in tangible ways, particularly through enhanced productivity. Organisations experience direct productivity gains as AI enables the automation of repetitive tasks across various business functions. This automation, combined with AI-enhanced decision-making capabilities, leads to more efficient resource allocation throughout the organisation. Employees can redirect their time to strategic activities, creating a multiplier effect on productivity.

Indirect value creation emerges over time as organisations leverage AI for long-term benefits. Companies gain competitive advantages through innovation, using AI to identify and capitalise on new market opportunities. Market intelligence becomes sharper and more actionable, supported by AI-driven analytics and insights. The quality of decision-making improves across all organisational levels as AI provides data-driven recommendations and forecasts. Perhaps most significantly, AI enables organisations to identify and develop new products and services, opening additional revenue streams and market opportunities that might otherwise remain undiscovered.

ROI Calculation Framework

A structured approach to calculating AI ROI involves several key components:

• Cost Assessment includes Environment Setup with development environment costs, testing and production environment expenses, and quality assurance infrastructure. Implementation Expenses include tool acquisition and usage fees, training and development programs, advisory services, and governance structure implementation. Ongoing Operations include maintenance requirements, system updates, additional staffing needs, and continuous monitoring costs.

• Benefit Evaluation includes Operational Benefits with automation-driven cost reductions, process efficiency improvements, and resource optimisation. Revenue Impact includes sales performance improvements, new market opportunities, and customer retention rates. Strategic Value includes innovation capabilities, market competitiveness, decision-making quality, and customer experience enhancement.

How to Measure

• Establish a Baseline: Before implementing AI, establish a baseline for the metrics you intend to track. This allows you to measure the change or impact AI brings.
• Use a Dashboard: Set up a centralised dashboard to display real-time metrics and KPIs related to your AI initiatives.
• Regular Audits: Audit your AI implementation regularly to protect against risks like copyright infringement and data leakage. Auditing needs to be part of the AI implementation lifecycle, not an afterthought.
• Feedback Loop: Create a process for capturing user feedback and use that to refine models and implementations.

Monitoring and Optimisation

Continuous monitoring of AI initiatives requires a three-pronged approach. First, organisations must conduct regular performance reviews that encompass tracking KPIs against established baselines while measuring progress toward objectives. These reviews systematically identify areas that require optimisation, ensuring the AI implementation remains aligned with business goals.

Second, robust adjustment mechanisms need to be in place for ongoing refinement. This includes fine-tuning AI models and applications based on performance data, updating metrics as business needs evolve, and incorporating valuable user feedback and lessons learned during implementation.

Third, organisations must consistently focus on long-term value assessment. This involves evaluating the strategic impact of AI initiatives on the business, measuring concrete innovation outcomes, and assessing how AI implementations affect the organisation’s competitive positioning in the market.

This comprehensive approach to measuring and tracking AI ROI enables organisations to make informed decisions about their AI investments while ensuring continuous improvement in their implementation strategies.

Download Now: Free AI Strategy Playbook
[New 2025]

The Complex Landscape of AI ROI Measurement

Calculating ROI for AI initiatives is complex due to several interrelated challenges:

• Data quality issues can impair model performance, while technical complexities in model training and deployment often exceed initial estimates.
• Customisation needs, including fine-tuning and prompt engineering, add substantial costs.
• Many organisations lack specialised AI talent, requiring additional investment in recruitment or partnerships.
• Governance challenges include model degradation over time and data staleness. Unrealistic expectations about AI capabilities often lead to costly rework. 
• Organisations frequently underestimate ongoing maintenance costs and struggle with siloed implementations that complicate enterprise-wide ROI assessment.

Success requires understanding these challenges and taking a comprehensive approach that aligns technical implementation with business strategy.. Many organisations lack these specialised skills in-house, necessitating additional investment in talent acquisition or external partnerships.

Governance issues present ongoing complications

AI models naturally degrade over time, leading to reduced accuracy and effectiveness. Organisations often struggle to respond rapidly to technological changes while maintaining a holistic enterprise view of their AI implementations. Data staleness can impair strategic decision-making, undermining the core purpose of AI investments.

Expectation management poses another significant challenge

Organisations frequently harbour unrealistic expectations about AI capabilities and implementation timelines. These misaligned expectations can lead to extensive rework and increased costs, distorting ROI calculations.

Initial ROI projections often fail to consider ongoing costs

The continuous nature of AI project maintenance, monitoring, and updates requires sustained investment, which must be factored into comprehensive ROI calculations.

Finally, siloed implementation approaches can severely impact ROI measurement

When organisations deploy AI initiatives in isolation, they risk misalignment with broader enterprise strategy, resulting in higher costs and diminished benefits. This fragmented approach makes it difficult to assess the true enterprise-wide impact of AI investments.

Understanding these challenges enables organisations to develop more realistic ROI projections and implement more effective measurement strategies. Success requires a holistic approach that accounts for technical and organisational factors while aligning with strategic business objectives.

Maximising AI ROI: The Path Forward

As organisations navigate the complexities of AI implementation and ROI measurement, success increasingly depends on adopting a strategic, methodical approach. Experience shows that organisations achieve the most substantial returns when they start with focused pilot projects demonstrating concrete value while managing risks effectively. These initial successes and a commitment to quick wins help build momentum and secure stakeholder buy-in for broader AI initiatives.

The path to maximising AI ROI requires both technical excellence and organisational adaptation. Through iterative development and continuous improvement, organisations can refine their AI solutions based on performance data and user feedback. However, sustainable success ultimately depends on building a data-driven culture that embraces cross-functional collaboration and innovation. Organisations that combine this cultural transformation with rigorous measurement practices will be best positioned to realise meaningful returns on their AI investments in today’s rapidly evolving digital economy.