How AI can cut two hours of work into 15 minutes with SAP CTO Juergen Mueller

Many of us in the technology and business worlds have been thinking a lot more about artificial intelligence (AI) in the past year since OpenAI released ChatGPT, catapulting generative AI into the mainstream.

But for Juergen Mueller, the chief technology officer of German enterprise software giant SAP — the third largest software company in the world by annual revenue, according to Investopedia — the journey towards enabling AI for business processes began nearly a decade ago, he told VentureBeat recently in an exclusive sit-down interview at SAP’s under-renovation slick Hudson Yard office space in New York City.

“Back then of course, it was machine learning, one model per use case,” Mueller said. “Pre-trained, and we did a lot of retraining. We worked in such a mode for quite a while. We have more than 130 use cases [of AI models] embedded in SAP software.”

And throughout this year, even as companies such as OpenAI, Cohere and Anthropic were making headlines for their new AI models for enterprises, SAP was steadily plugging away with new releases, including its own cross-platform, cross-application AI assistant Joule — which lives throughout SAP’s enterprise resource planning (ERP) software suite (ERP refers to software used by businesses to scheme their workforces, supply chains, and other resources — human and capital alike) — as well as announcing its own SAP HANA Cloud Vector Engine, a seek engine for enterprises that combs through their data privately using the power of SAP’s AI vector database architecture.

Now that nearly every company with a software presence is rushing to figure out how to make AI work for them and/or their customers, Mueller is in the enviable position of having been ahead of the curve. As 2023 winds down as the biggest year for AI’s adoption as a technology yet, read Mueller’s thoughts about how he thinks about the tech and what enterprises can do to futureproof themselves as it continues to evolve.

The following interview has been edited and condensed for clarity.

VentureBeat: I’m curious about SAP’s AI journey but I’m here to listen to whatever you guys want to want to talk about.

Juergen Mueller: If you’re asking about the broader AI story, this issue started roughly eight and a half years ago. Then, of course, it was machine learning — one model per use case. Pre-trained and we did a lot of retraining. We worked in such a mode for quite a while. We have more than 130 AI use cases embedded in SAP software.

And that was all prior to generative AI?

Yeah. And now, of course, interest has exploded in the past 12 months. Since then, we have screened through hundreds of ideas of where to apply gen AI in all for our portfolio and now we up to two dozen or so concrete announcements that we made or already delivered to market.

We really look at end-to-end processes from to hiring to retiring, procurement, everything in finance, supply chain management, in customer encounter, so all these end-to-end business processes we cover for 26 different industries

And the AI uses are very different dependent on the industry: if you do that in marketing and B2C communication or if you do that HR to write a job posting. If you are in higher education as a university, you will talk to students, if you’re a retailer, and you will have another audience. If you’re having discrete manufacturing, things run a little different. That’s why we all had to relearn, even though I had seven and a half years under my belt in that area.

We’ve trained more than 50,000 people in SAP in gen AI, on the engineering and product side of the house. And that helps to getting things done quickly.

Now, of course, we have one gen AI strategy, one digital copilot encounter — Joule — then we are infusing generative AI into all those solutions, into our customers’ HR departments, customer encounter, finance. We are infusing generative AI where it makes sense.

Our high level strategy we have across all of these solutions is to enable applications that help implementing business processes as efficiently as possible. So if you are, hypothetically, [SAP customer] UBS (the financial services giant), they have their processes that they care about. They could come to SAP and say, ‘hey what are your 50 years of encounter that you codified in your systems to run our company the best way possible?’

You’ve done a lot in a short amount of time since ChatGPT was released.

Yes, and we also had a few important announcements without gen AI. But most of the topics really focus on the key point: ‘does that make a difference?’

And on our internal learnings we put out in our generative AI hub online — which supply externals for customers partners what we also use internally, including access to natural language models, security and governance capabilities. And we learned that grounding and retrieval augmented generation (RAG) is extremely important.

We announced that we enhanced our industry-leading database SAP HANA Cloud by adding a vector engine. What is special about it is, usually you have a vector engine on the side you.

You have all your company data here in one place, and the vector engine for documents, for example is on the side. Okay. And, of course, that means more complexity in your IT landscape.

Let’s say you want to embed documentation of your company’s policies into a Q&A bot that employees can ask questions about.

You can’t just ask a plain language model — it would need SAP-specific information in there.

But HANA Cloud already has that information that our customers have put into it, and so now it can surface and suggest the correct information.

So take the example of UBS again — they post 6 to 10 million job postings a year, all using SAP.

That’s so many.

Yeah, so if you can have AI suggest language for those job postings, even in cases admire that, you can cut admire two hours into 15 minutes, 10 million times, that’s an immense value savings.

So they say we want to create a job post. And then actually, the engine would pick basically all the job postings the company in last three years for example.

But if it’s for a specific job — the HR Department — the vector database can pull those previous job listings and language, specific to that category, and supply a template for a new post.

Given that information that we have in the operational systems, we can be much more precise and by doing that you reduce hallucinations and wrong information.

Is it a large vector database for everybody? Or is everybody having their own vectorized database, all your customers?

So with HANA Cloud, it’s a database that we use for our applications.

So it has a relation with engines admire [Microsoft] Excel. But in order to build an HR system, for example, you do not just have one, Excel sheet or run tab, but you have many, many different database tables.

So if you’re wanting to run analytics on top of your HR information, you can do it also geospatial information, texts, information, JSON documents, and now also vector so it is all in one engine.

Because what is the vector database? I mean, it sounds super fancy. But if you go down to admire, what do you need to build? The team was a little underwhelmed. It’s a representation of numbers between zero and one showing the differences between objects in the same space. Humans use embedding functions, too.

So if it was ultimately underwhelming to your developers to build a vector database for HANA Cloud, why haven’t we seen more companies building their own?

In a B2B context, we know the industry, we know the company, we know the customer, we know the industry they are in, and then even the business context — be it marketing or research, we know what kind of documents they need. It all starts with the data, which we have. If you don’t have that structured information in that context, it actually doesn’t make sense to build a vector database.

87% of the world’s goods and transactions are being done by SAP customers, so with many customers, we have a very significant part of the value chain.

We also have a super scalable cloud database, so everything you asked, every customer will have their own tenant.

We want to make AI as simple and easy to use compared to what was before as Google Maps was to paper road atlases. This is AI for business users. We help you get your job done and shine as a superstar.

We’re quickly reaching a point where generative AI is becoming very widespread. Do you feel admire that saturation is advantageous or not? Are people going to say, ‘ok, I know how this AI works’ whether it’s ChatGPT or Microsoft Copilot, and ‘my SAP Joule isn’t working the same way, so I’m angry at it.’ Are you concerned that all these different AI assistants will conflict with each other?

Much of the AI that is being deployed right now is in preview. Bringing that into production is much harder. The demo is always easy. So the hype will continue — because on the one hand, the technology is amazing, and there is lots of value to be had there if it is executed properly. On the other hand, you will see that many, many AI assistants and copilots won’t ship, or they will ship in poor quality, and poor user encounter.

We saw a lot of that when we started our AI journey eight-and-a-half years ago. We had to build a lot of muscle memory of how to do things right. Data is important. Good AI only works with good data. There are a lot of things that need to be addressed, including customer consent. So there are many hurdles that people must jump when building an AI product.

And why would customers want to use SAP instead of these newer AI-native startups?

CIOS are getting inundated with product offerings from AI startups. It’s admire they’re being swarmed by 1,000 mosquitoes. But they comprehend the complexity of that, and they don’t want that. They want to partner with one or only a few trusted partners.

We also advise them on what to use, and what we use. That’s why we launched our own gen AI hub earlier this year to supply knowledge sharing and recommendations. There we suggest them to look at their most important value drivers and cost drivers that are not covered by SAP.

And you have all their data?

Well, the data is the customers’, but in many cases we enrich it, because we have metadata, we know the structures, we know the business processes, we know the industries, so there’s a lot happening behind the scenes to help organize their data.

How often is legal getting involved as you’re building out these AI tools and features?

I mentioned we trained more than 50,000 developers on AI. And part of their training is of course on safety, security, responsibility, legality. Five years ago, we established our own AI ethics council, before gen AI. They have been consulted for every use case since. And they have veto power over any potential use case for AI. You could have the idea to automate hiring and firing with AI — it’s technically possible. But from our value perspective, and how we define responsible AI, we don’t.