Name: Karl Havard | Taiga Cloud
Uploaded: 2023-11-30T12:00:00.000Z
Duration: 15 min 8 s

Clips
More from HPE Discover Barcelona 2023

Karl Havard

Managing Director

Taiga Cloud

play_circle_outline Karl Havard, managing director at Taiga Cloud, as a guest speaker

play_circle_outline Taiga Cloud's focus on generative AI and GPU-based infrastructure

play_circle_outline Democratization of access to generative AI for startups and enterprises

Info

Karl Havard | Taiga Cloud

Karl Havard

Managing Director Taiga Cloud

(upbeat music) (air whooshing) (attendees chattering)

>> Hello, everyone, and welcome back to theCUBE's live, ongoing covering of HPE Discover Barcelona 2023. I'm your host, Rebecca Knight, along with my cohost and analyst, Rob Strechay. This is day two. There's a lot of energy and excitement in the room. I feel it.

>> Yeah, and we're getting into the meat of things here as well and getting some great partners as part of that, and I'm really excited

about this next guest, so. >> Yes, yes. He was on the main stage this morning, Karl Havard, managing director at Taiga Cloud, great name for the company. Well done. Congratulations.

>> Thank you. >> On being up there

with Antonio this morning. >> Yeah, it was fun.

It was fun, yeah. >> What really struck me was how you began with such a note of optimism about this moment in time.

>> Yeah, yeah, and I mentioned my gray beard

at the time as well. >> Yeah, you did. You did.

>> Which I related to.

>> Yeah, yeah, yeah, awesome. >> Yeah, it's relatable, right.

>> So yeah, it is, isn't it? I mean, and I don't know if you caught the lady towards the end who wanted to, well, was going to be a judge and decided not to be, she took the other side of the coin as well. And I think absolutely right. It's a balance thing. I think we're our moment in time with generative AI and AI as a whole. You know, the way technology has evolved to support human creativity, you know, who knows what's on the horizon? Which is great. You know, people's imagination can now be brought to life, but then on the flip side of that, you know, imagination can go everywhere, so I think we have to do it in absolutely the most ethical way we possibly can and support organizations, the right organizations, to, I guess, develop what they want to develop and qualify those, call them rogue type of organizations that want to get hold of this technology and prevent them from doing wrong things with it.

Call it that, yeah. >> Right, so doing AI right, doing AI ethically, responsibly is critical. Tell our viewers a little bit about Taiga Cloud.

>> Okay, so Taiga Cloud is Europe's first largest and cleanest generative AI cloud service provider. So we focus on generative AI, and then our infrastructure is all GPU-based. We don't do any app migration or any big data storage. It's purely about helping organizations get access to this technology to train to and infer, you know, the large language models or the models that they're building for AI purposes. So the way we have gone about building our business so far and the way we'll continue to build it is, as well as being the first and largest in Europe, which is great, and that's a lot of help with HPE here, we're also doing it back to your ethical way of making sure that this compute power, which is really high-density power drawing equipment, is housed in the right way. And what I mean by that is data centers that are clean energy driven, so hydroelectric powered, not necessarily nuclear, so I should say renewable energy. So hydroelectric power predominantly, but also cooled in the same way as well, through natural sources. And I think I gave the example this morning that our data center in Sweden just outside the Arctic Circle is completely naturally cooled. Our Lefdal Datacenter in Norway is in a disused mine below sea level, so it uses a natural environment for cooling. So that means that people can then consciously decide how to train their generative AI because we're transparent about it. The other thing we're transparent about is not just the source of energy. It's the efficiency of the energy we use. So I don't know that you're familiar with PUEs, the power usage effectiveness, yet, so I think acceptable levels are 1.4, 1.45. All our data centers are 1.2 or below. Our Boden data center is 1.06,

which is, you know. >> Wow.

>> That's a good number. >> Yeah.

>> We know we can do better, so we want to draw that down, but so a sustainable way of creating your applications through generative AI is absolutely the heart of what we do. There's other things I can keep going on about if you want me too. (Rebecca and Rob laughing)

(Karl and Rebecca laughing) >> Well, I think it's, I mean, today's the first day of COP28 down in Dubai, so I think it's very timely to be talking about this, and I think that when you start to look at GenAI and all of the data and all of the crunching and how they have, you know, I mean, people are talking tens of thousands of GPUs

to go and train models. >> Yeah.

>> What are you seeing from your customers? 'Cause you talked about customers of all different sizes and democratizing AI, and kind of get into that a little bit.

>> Yeah, I mean, that's one of the things, core values, which I think Taiga shares with HP and vice versa. Because if you look at the market dynamics, which, clearly, you have, there is a big demand for this compute capability, and organizations are buying it in bulk. And some of those organizations are buying it purely for their own usage, which is fine because they're big hyperscalers and, you know, they have to develop their and keep enhancing their products. However, I think that limits the access to startups and enterprises who have great ideas. They've got the right skillsets. They just don't have access to the right compute power to bring it to life, so our whole ethos is to democratize access to this. So buying quite a few GPUs, I mean, the announcement we made today was an additional 330 million invested with HPE. That's on top of what we've previously invested, so we're now at 800 million total. So they're crazy big numbers, which is, you know, a nice place for us to be, but the whole point of us is to say, "We'll buy these GPUs. We'll make them live. We'll put 'em in the right environments, but what we'll do is make these open and accessible." If you're a small startup and you only need access to a couple of H100 machines, you can do that with us. If you're a bigger enterprise and you've got this great model you wish to train but you only need it for a couple of months, we'll do that as well. Organizations that we've had knock on our door say, "We'll take everything you got for three years." We won't do those deals. We want to actually be able to have, I guess, fueling the European economy from a generative AI point of view, to be that part of the ecosystem which, again, was mentioned this morning, allow people to roll on and roll off as and when they want to. And do you know what, sorry, the second part of your question, I think, was the types of customers. So we're not focused on specific organizations with, you know, who are going to take a lot of GPUs. You know, startups that share the same values as us are absolutely at the heart of our target audience. Call it that. Because if we can help them succeed, great. We've done part of our job. Likewise, life science companies around drug discovery, universities on research and around things like that, perfect customers as well. So yeah, we want to make sure everyone has a fair share.

>> Yeah, are you seeing that, I'm sorry, are you seeing that people are coming to you, especially being European-based and European-headquartered, coming to you and saying, "Hey, I want to build a European foundational model so it comes out of Europe," versus, "Hey, we're taking Llama 2," or, "We're taking one of these other foundational models."

>> Yeah, yeah, we are, more so now. You know, a few months ago, we weren't necessarily seeing that, and I think there's been a few scares from a EU compliance perspective, which affects not just European organizations. It affects, as you know, global organizations that have European customer bases. Then, we've had a lot of people come to us to say, "Okay, we like what you're doing. You're a European organization. You can offer us true sovereignty, true compliance with this GDPR, et cetera." And, you know, we know if anything happens inadvertently, and, I don't know, a diplomatic disagreement between countries either side of the Atlantic, and it won't happen, but if it did happen, you know, we can help them tick their compliance boxes. They have full control over their data in the region that they're within. And I think we're getting more US companies requesting this too, which is interesting.

>> Well, it's an uncertain geopolitical environment.

>> Yeah, yeah, yeah, yeah. >> I can understand that.

>> Yeah. >> One of the things you had just mentioned is having the same kind of values as the startups you work with and how that's a real priority for you. It's also, I would imagine, something

that you share with HPE. >> Oh, completely.

>> This idea of a shared value, shared priorities, shared commitment to sustainability and to democratizing AI. Why is that so important for your working relationship, and what does that bring to customers? What's the value that customers feel from that?

>> Yeah, I think it's like, not just what we offer with generative AI, but if you got that common purpose, you know, that common aim of objective and it's not a monetary objective. I mean, that is important in some instances, but you say, "Okay, here's this customer here. They can accelerate what they're doing in genomic research in order to find a cure for cancer or whatever it might be." Then HPE, obviously, are very interested in helping those, and in partnership with us, we can help that customer in a specific area, but HPE have a breadth and depth of other things they can also bring to the table. So you know, that's great for the customer. If you take the customer-centric view, together, Taiga and HPE, we can sort of offer that customer a full-rounded solution all with the intent of helping them succeed in their own objectives ideally for the benefit of the planet or humankind or whatever it may be. But yeah, I just feel with working with these guys for quite some time now, even on a personal level, you can sense those values are true, so yeah.

>> How do you see the customers? Are they transient, or do they come and host? They build and host there? How does that work for you?

Because, I mean. >> So yeah-

>> 'Cause if you have a lot of GPUs, and if you're doing inference, for instance, maybe you train the model and then do inference, maybe you don't need as many, you know, H100s, for instance.

>> Yeah, I mean, yeah, when you move from the training and tuning to inference, absolutely, it's a slightly, I won't say different, but a config that's required to help organizations move to inference. So what we do is, as Taiga, is we have infrastructures of service. So simple terms, we rent the compute power out and host the customer, et cetera, et cetera. We do have a sister company in our group called Ardent who are managing a lot of data centers, building 'em out. The will offer the co-load capabilities as well. So if organizations have access to their own compute power, we can bring them on-stream as well and offer them that hybrid model should they want to. So yeah, did that answer your question all right?

>> Yeah, I guess are people coming and, you know, renting it for a month, two months?

>> Oh, got it, yeah. >> Training, or is it, hey, they come and, you know, not through your contract, but they're building out their apps. They're training, then they're tuning the models. Then, they start doing some inference there, and they're re-tuning the models, or what?

>> So that's the, I guess, the beauty of what we're trying to construct as a European cloud for GenAI. So we have these islands of 2,032 GPUs all configured in the same way, located in different locations. Now, if someone wants to train a model for, say, a three or six-month period, we can do that. When they move to inference, we can give them a slightly different config and move them to a data center probably more suited for inference. So in our suite, we have tier three data centers, which means high levels of security.

>> Right. >> Minimal downtime, I think 1.6 hours per year downtime that you're allowed. With also round-trip times of 12 milliseconds or less. So when it comes to inferencing, even though it may be less demand on the GPUs, the CPUs come into play, though.

>> Right. >> But you need that balance. So that's when we'd probably move them to the slightly different area of our cloud to help them, well, they can pick and mix. That's the beauty, and to your point about contract lengths, now, one, three, six months, fine. 12 month, 24 months, 36 months is fine. Our longer-term contracts tend to be with organizations that serve a number of customers themselves.

>> Oh. >> So they reserve a bank of compute power, and they actually serve several customers within that, so it still fits within democratizing access.

>> So almost doing LLMs as a service or something like that.

>> To a degree, yeah, yeah. >> Yeah.

>> Absolutely, I mean, and we need to get better at that because at the moment in our evolution path, we are very much infrastructure in service, but we know we're having to enhance our managed services on top of that and also our software skills. So we're ramping up our skillsets around people that really understand how to construct, advise, even audit LLMs in order that they are configured in the right way, et cetera.

>> Yeah, the skillsets is, and we talk about this a lot, the skillset around that is still so nascent for a lot of organizations. Are you finding that, and this is why you're going

down that path, is that it- >> Yeah, it is. I mean, it's like when any new, and I know GenAI is different, but when any new technology emerges, it merges rapidly.

>> Well, it's ChatGPT's birthday today.

>> Oh, is it? Oh, okay. >> So yeah, yeah.

(Rebecca laughing) >> Well, I didn't know that. I need to send a card. (Rebecca and Rob laughing)

>> (laughs) But yeah, the skills tend to follow, don't they? And therefore, there's always this catch-up. So we know we need to be ahead of the game. And being an elite partner of NVIDIA helps us because we get future sight of what's coming down the line, so we can see, you know, H200, Grace Hopper, et cetera, advance notice of what that entails. We get to see what they're developing on their software stack, same with HPE, so we're sort of slightly ahead of the game so we can attract the right talent in. But if we don't get the talent in ourselves, which is probably not our true model. We're not going to scale heavily that way. That's where the partnership with HPE comes in 'cause they have that talent influence.

>> They've got the talent. >> Yeah, exactly.

>> Right. Exactly. >> So yeah, yeah, yeah.

>> Yeah, yeah. >> Yeah, excellent.

>> Yeah. >> Well, Karl, thank you so much for coming on theCUBE. It was a pleasure having you here.

>> Thank you. Glad to be here.

>> Excellent. Excellent. >> Thanks.

>> I'm Rebecca Knight for Rob Strechay. Stay tuned for more of theCUBE's live coverage of HPE Discover Barcelona. You are watching theCUBE, the leader in high-tech enterprise coverage. (upbeat music)