Reflections from Red Hat Summit Connect: Practical Steps for AI in Production

There is something unique about standing in front of a live audience at a tech conference. The energy is different, the questions are sharper, and you can feel how hungry people are for practical answers. This year at Red Hat Summit Connect Brussels, I was fortunate enough to deliver a session called Finally Getting Started with AI: Leveraging OpenShift AI. AI has dominated the headlines for years. It has been labeled everything from the greatest opportunity of our generation to a threat to human creativity. But in enterprise IT, the conversation is often much more pragmatic: “How do we actually get value out of this?”

When I talk to organizations, the story is surprisingly consistent: “We built a prototype that worked really well in the lab. Everyone was excited. Then… nothing happened.” This is what I call the ambition gap, that frustrating distance between wanting to leverage AI and actually running AI-powered workloads in production where they can make a difference. My talk was designed to close that gap, step by step. It was not about adding more hype to the conversation. It was about giving teams a practical path forward, a way to move from small, isolated successes to enterprise-scale, reliable deployments that deliver measurable business impact.

Before we dive into the “how,” it is worth looking at the “where we are.” Gartner predicts that by 2026, more than 80% of enterprises will have used generative AI APIs or deployed AI-enabled applications. This is an astonishingly fast rate of adoption for any technology. But Gartner also sounds a warning bell: nearly 30% of these initiatives will fail after the proof-of-concept phase. Why? Because running a successful experiment is not the same thing as running a reliable, scalable product. A demo can impress stakeholders, but production requires security reviews, monitoring, compliance, cost controls, and integration with existing business processes. I shared with the audience that this is where I see organizations falter. They succeed at proving that AI can work, but they have not planned for what comes next, for the day when real users start depending on it.

To illustrate this, I introduced the concept of Toy AI versus Enterprise AI. Toy AI projects are experimental, fun, sometimes even groundbreaking, but they are not built for the demands of production. They might be a single script on a data scientist’s laptop, with no version control, no monitoring, and no way to scale. Enterprise AI, by contrast, is designed from the ground up to be robust. It integrates into CI/CD pipelines, has automated testing, comes with observability baked in, and is secure by default. Making the leap from toy to enterprise is not just a matter of “adding more data.” It is about adopting a product mindset, where you think about lifecycle, reproducibility, and collaboration between teams.

Before building solutions, we must take a clear look at the challenges. When I work with organizations, the same barriers surface again and again. Sometimes it is about the data itself: information scattered across silos, inconsistently labeled, or simply outdated. Other times it is about trust: leaders are hesitant to deploy a model whose decision-making process they cannot explain, and end-users may resist using a system they do not understand.

There is also the ever-present challenge of proving value. AI projects often promise significant impact, but the benefits are not always immediate or easy to measure. Without a clear ROI, executives may hesitate to invest further, leaving projects stranded after their first proof-of-concept. And even when the business case is solid, there can be fear of change within the organization. Teams worry about job displacement or simply feel uneasy about shifting to a more automated way of working. By putting these challenges on the table early, we can address them head-on. If trust is the issue, we invest in interpretability and transparency so that stakeholders can see how the model makes decisions. If organizational resistance is high, we engage people from across the company, building champions who will advocate for the technology. Overcoming these hurdles is just as important as choosing the right algorithm or framework.

Once the obstacles are acknowledged, the next question is always, “Where do we start?” The encouraging truth is that most organizations already have more of the foundation in place than they realize. I often tell teams that they do not need to rebuild their entire IT landscape to be AI-ready. Instead, they need to connect and extend what is already there. Linux is often quietly running under the hood of critical infrastructure, providing the stability and performance needed for heavy workloads. Kubernetes might already be orchestrating containers somewhere in the environment, ensuring portability and scalability. And automation is not a foreign concept, many teams are already using CI/CD pipelines or Infrastructure-as-Code to reduce manual work. The opportunity is to bring all of these elements together into a coherent platform that is capable of supporting machine learning workflows at scale. 

This is precisely where OpenShift AI comes in. Rather than introducing a completely new stack, it builds on what is familiar and proven, stitching together these technologies into a platform purpose-built for data science and AI. For many organizations, this is the turning point: the realization that they are not starting from zero, but rather evolving toward a more connected and capable future.

Once the foundation is understood, it is time to talk about process. This is where I introduce the concept of MLOps, the natural extension of DevOps principles to machine learning. Most organizations have already spent years perfecting their DevOps practices, creating pipelines for software builds, tests, and deployments. MLOps brings that same discipline to AI models.

Instead of thinking only about writing and shipping code, we begin to think about the entire model lifecycle. Data must be gathered and prepared, models must be trained and validated, and once deployed, they must be monitored to ensure that their predictions remain accurate over time. When performance drifts, and it inevitably will, the model must be retrained, redeployed, and retested. This creates a continuous improvement cycle where AI becomes a living part of the business rather than a one-off experiment. When teams embrace MLOps, something remarkable happens: the gap between data science and operations begins to close. Data scientists can focus on exploration and innovation, knowing that their work can be reliably moved into production. Operations teams gain confidence that deployments are predictable, observable, and secure. Leadership gets faster iteration and clearer visibility into what is running and why.

At this point in the talk, I introduced OpenShift AI as the enabler of everything I had just described. OpenShift AI is not just another set of tools. It is a collaborative environment where data scientists, developers, and platform engineers can finally work together without friction. Within this platform, teams can provision GPU-powered workbenches for training, create repeatable data pipelines, deploy models into production with consistency, and monitor them for performance, drift, and bias. All of this happens on a single, unified platform that supports both hybrid and disconnected environments. The result is a system where innovation can happen quickly, but without sacrificing security, governance, or reliability. One of the most common pain points I hear about from teams is the jump from model development to model deployment. Training a model can feel exhilarating you tweak hyperparameters, you see accuracy climb, and you get a result you are proud of. Then comes the dreaded question: “How do we actually put this in production?” 

OpenShift AI removes much of the pain from this process. Distributed training makes it easy to run workloads across clusters and use hardware accelerators efficiently. Once training is complete, deployment becomes a clear and repeatable process: select the serving runtime, specify the resources, connect it to the object store, and publish an endpoint. Applications can then consume this endpoint directly for inference, turning a research artifact into a live, production service. Perhaps the best part is how scalable this becomes. If demand spikes because a new feature powered by the model goes live, the platform can automatically allocate more resources to keep latency low. There is no need for manual intervention or middle-of-the-night firefighting.

Speed is powerful, but it must be balanced with responsibility. In today’s regulatory environment, you cannot simply deploy an AI model and hope for the best. You must be able to explain how it works, detect bias, and ensure compliance with privacy and safety requirements. This is where OpenShift AI provides peace of mind. It includes tools to monitor models over time, detect data drift, and highlight potential bias in predictions. Its guardrails for large language models help organizations moderate outputs and prevent inappropriate or non-compliant responses. I like to say it is like having a built-in quality assurance officer who never sleeps. This governance capability is no longer optional. With the EU AI Act and similar regulations on the horizon, organizations will soon be required to prove that their AI systems are transparent, fair, and safe. Building this into the platform from the beginning is far easier than trying to bolt it on later.

Another highlight I shared during the talk was the ability to create data science pipelines visually. This is more than just a convenience. It changes how teams work together. Instead of passing scripts back and forth, they can build an end-to-end workflow that covers data preparation, model training, validation, and deployment. The benefits are profound. Work becomes reproducible and auditable, results are consistent across environments, and collaboration becomes transparent. Everyone can see exactly what is running, when it ran, and which version of the model is currently serving predictions. This level of visibility builds trust, reduces errors, and accelerates iteration cycles.

By the end of the session, it was clear how all the pieces fit together: closing the ambition gap, addressing organizational and technical barriers, connecting existing infrastructure, adopting MLOps practices, and leveraging OpenShift AI to orchestrate it all. AI is no longer a luxury project for innovation labs. It is rapidly becoming the deciding factor between companies that lead their industries and those that are left behind. OpenShift AI gives organizations the confidence to deploy AI at scale, responsibly and securely. And while it still cannot schedule your holidays, it can at least make sure your models keep running while you are away, which is almost as good. This is where I left the audience with a challenge: do not let your AI ambitions gather dust. If you are serious about turning prototypes into production-ready solutions, now is the time to act.

My team runs AI readiness workshops designed to help you align your infrastructure, your data science teams, and your business strategy so that you can get to value faster. If this resonates with you, reach out to me to schedule a workshop or attend one of our upcoming Devoteam + Red Hat events. Together, we can bridge the ambition gap and help you deliver AI that truly drives business outcomes.