Six must-know tips for CIOs on generative AI implementation

Generative AI has become a major priority for CIOs, as boardrooms increasingly recognise its potential to transform business models. While GenAI promises ‘transformative potential’, recent pilots highlight that turning this promise into real business value can be challenging.  

According to industry analysts Gartner, by 2025 30% of GenAI projects will be abandoned after the Proof of Concept (PoC) stage, due to uncertain business value, inadequate risk management, and poor data quality. However, with the right strategy, CIOs can avoid these challenges.   

Here are six key tips to help CIOs move beyond pilot phases and successfully implement generative AI.  

Tip 1. Anchor GenAI in business strategy and value creation 

Adopting new technology without a clear strategic purpose rarely leads to lasting success. The same can be said for GenAI.  

Before diving headfirst into implementation, you want to ensure your GenAI projects are anchored firmly in your organisation's wider strategy, and that it truly adds value. 

Just because you can use GenAI for a particular use case doesn't mean you should. Will your GenAI project increase revenue, drive efficiencies, or accelerate innovation?  Every GenAI initiative should have a well-defined business case supporting these objectives. You might find that in some cases, a more traditional technology solution may be a better fit.  

To align your GenAI initiatives with business strategy you should: 

• Identify specific business objectives that GenAI could support 

• Only focus on use cases with measurable outcomes 

• Get your leadership team on board and encourage inter-departmental collaboration  

• Set clear success metrics to demonstrate return on investment 

By anchoring GenAI initiatives into your strategic priorities, you avoid the hype-driven trap of ‘technology for technology’s sake’. Focusing instead on real, measurable business value.  

Tip 2. Choose the right deployment approach 

Should you build your own GenAI solution, buy one off the shelf, or take a hybrid approach?  

Your choice should depend on your specific business needs, technical resources, data availability and anticipated GenAI use cases. 

Buy: implementing pre-built generative AI solutions like Microsoft Copilot provides productivity gains with lower upfront investment. This approach is best suited for general business use cases, like summarising documents, generating code or writing emails. However, it is less suited for highly specialised business tasks that require in-depth domain expertise. 

Build: creating and training Large Language Models from scratch provides the highest level of customisation to address unique business challenges. But requires substantial upfront investment, technical expertise and massive amounts of resources to maintain it. This approach is the least common today and best suited for organisations with highly specialised business needs and access to significant technical manpower.  

Hybrid: a hybrid approach offers a middle ground, providing customisation and efficiency, without the hefty price tag or need to build from scratch. This strategy uses Retrieval Augmented Generation (RAG) to bring in real-time domain-specific knowledge or fine-tuning to improve the outputs of specific tasks. Both techniques can also be used in tandem. 

Tip 3. Adopt a ‘lighthouse’ cycle strategy 

Lighthouse projects are small-scale, high-impact initiatives that act as blueprints for larger digital transformation initiatives. By implementing a ‘lighthouse' cycle approach, you can experiment, refine, and validate the value of GenAI, before scaling it more widely across the business.    

Not every GenAI initiative delivers meaningful impact. The secret is to focus on lighthouse use cases that align with your strategic objectives and offer clear, measurable benefits.  

Remember the importance of keeping humans in the process. Build key capabilities by upskilling your team, with a focus on AI literacy and prompt engineering, to fast-track experimentation. Consider establishing an AI innovation lab or sandbox environment where you can safely test and refine GenAI models before full-scale deployment.  

Here are the steps in the lighthouse cycle approach:

Tip 4. Watch out for data quality 

Whether you’re building custom GenAI models, fine-tuning existing ones, or using Retrieval Augmented Generation (RAG), it’s important to use high-quality and clean data. Using outdated or poor-quality datasets can introduce biases, reduce model reliability, and lead to misleading outputs. 

When applying RAG, you need to ensure that the retrieved data is current, accurate and relevant. In the case of fine-tuning AI models, high-quality and diverse datasets are a must.  

To increase your data quality, ensure that your data governance programme is up to scratch. This includes: 

• Regular data quality audits to identify and remedy issues 

• Systematic management of both structured and unstructured data 

• Protocols for data accessibility  

• Maintaining data completeness 

Tip 5. Identify the real risks and mitigate them     

Identify the potential risks that generative AI could introduce to your business early on. Prioritise these risks based on severity using a case risk profile and implement strategies to mitigate them. 

Build AI governance and risk management frameworks with clear policies on GenAI usage, bias mitigation, and explainability. Keep up to date on changing legal, ethical, and regulatory factors such as the EU AI Act, the AI Bill of Rights, and existing legislation like GDPR, which could shape your approach to GenAI.  

Human oversight is critical. GenAI should enhance decision-making, not replace human judgment. Keeping people in the loop helps catch errors, biases, or unexpected AI behavior before they create problems.  

Watch out for: 

• Hallucinations even with RAG or fine-tuning 

• Low data quality, which can result in incorrect decision-making 

• Unprotected internal unstructured data, introducing security risks  

With thoughtful governance, forward-looking risk management, and human oversight, your GenAI initiatives can deliver results safely and responsibly. 

Tip 6. Monitor performance closely  

Once your GenAI solution is up and running, you'll need to monitor its impact closely to determine its ROI. If you’ve followed tip 1, (strategic alignment of GenAI), you should already have key success metrics in place.  

Whether it’s productivity gains, positive user feedback, cost savings, or time reductions on key tasks, these benchmarks are essential for understanding the value of your GenAI initiative, (something many organisations struggle to measure today). 

Regularly track your GenAI model’s performance to spot trends and areas for improvement. For example, if you’re seeing a high rate of hallucinations, it might be time to fine-tune your data or implement Retrieval-Augmented Generation (RAG).  

Continuous monitoring ensures your GenAI delivers real value and stays aligned with your business goals.  

From Proof of Concept to generating revenue 

Only 15% of organisations have reported earnings improvements from their GenAI initiatives. By following these six tips, you can position your organisation to be among the few successfully leveraging GenAI for improved competitive advantage. 

Not sure where to start? Reach out to our experts today.