Matt Calkins of Appian, founder and chief executive officer, joins theCUBE Research and host Dave Vellante at the NYSE Wired studio to explore why alignment — ensuring artificial intelligence systems follow human intentions and legal norms — should be the central focus for policymakers and industry. Calkins draws on their experience leading an enterprise software company to discuss alignment testing, regulatory architectures, open-weight models, recursive self-improvement and Appian's deterministic control layers for mission-critical AI deployments.
The conversation emphasizes practical policy and technical responses. Calkins argues that self-policing and voluntary agreements are insufficient; the United States should pursue federal oversight, an independent alignment test and compliance mechanisms such as auditors or executive-branch enforcement. They contend that prioritizing alignment now is a low-cost opportunity to prevent greater harm later while keeping AI beneficial and under human control.
Key insights include focusing on alignment science alongside model development, tightening compute and weight access controls to limit derivative progress and expanding teams beyond engineering to include social scientists and ethicists. Appian provides deterministic governance and a control plane for safe AI deployment. The objective remains aligned useful AI rather than a zero-sum win against other nations.
Forgot Password
Almost there!
We just sent you a verification email. Please verify your account to gain access to
theCUBE + NYSE Wired: Mixture of Experts Series. If you don’t think you received an email check your
spam folder.
Sign in to theCUBE + NYSE Wired: Mixture of Experts Series.
In order to sign in, enter the email address you used to registered for the event. Once completed, you will receive an email with a verification link. Open this link to automatically sign into the site.
Register For theCUBE + NYSE Wired: Mixture of Experts Series
Please fill out the information below. You will recieve an email with a verification link confirming your registration. Click the link to automatically sign into the site.
You’re almost there!
We just sent you a verification email. Please click the verification button in the email. Once your email address is verified, you will have full access to all event content for theCUBE + NYSE Wired: Mixture of Experts Series.
I want my badge and interests to be visible to all attendees.
Checking this box will display your presense on the attendees list, view your profile and allow other attendees to contact you via 1-1 chat. Read the Privacy Policy. At any time, you can choose to disable this preference.
Select your Interests!
add
Upload your photo
Uploading..
OR
Connect via Twitter
Connect via Linkedin
EDIT PASSWORD
Share
Forgot Password
Almost there!
We just sent you a verification email. Please verify your account to gain access to
theCUBE + NYSE Wired: Mixture of Experts Series. If you don’t think you received an email check your
spam folder.
Sign in to theCUBE + NYSE Wired: Mixture of Experts Series.
In order to sign in, enter the email address you used to registered for the event. Once completed, you will receive an email with a verification link. Open this link to automatically sign into the site.
Sign in to gain access to theCUBE + NYSE Wired: Mixture of Experts Series
Please sign in with LinkedIn to continue to theCUBE + NYSE Wired: Mixture of Experts Series. Signing in with LinkedIn ensures a professional environment.
Are you sure you want to remove access rights for this user?
Details
Manage Access
email address
Community Invitation
Matt Calkins, Appian
Matt Calkins of Appian, founder and chief executive officer, joins theCUBE Research and host Dave Vellante at the NYSE Wired studio to explore why alignment — ensuring artificial intelligence systems follow human intentions and legal norms — should be the central focus for policymakers and industry. Calkins draws on their experience leading an enterprise software company to discuss alignment testing, regulatory architectures, open-weight models, recursive self-improvement and Appian's deterministic control layers for mission-critical AI deployments.
The conversation emphasizes practical policy and technical responses. Calkins argues that self-policing and voluntary agreements are insufficient; the United States should pursue federal oversight, an independent alignment test and compliance mechanisms such as auditors or executive-branch enforcement. They contend that prioritizing alignment now is a low-cost opportunity to prevent greater harm later while keeping AI beneficial and under human control.
Key insights include focusing on alignment science alongside model development, tightening compute and weight access controls to limit derivative progress and expanding teams beyond engineering to include social scientists and ethicists. Appian provides deterministic governance and a control plane for safe AI deployment. The objective remains aligned useful AI rather than a zero-sum win against other nations.
>> (INTRO)Welcome back to theCUBE's NYSE Wired studio. I'm Dave Vellante. Much of the AI debate rests on two assumptions. One, whoever wins AI wins the future. And two, America has to move faster because of China. And our next guest challenges that premise. Matt Calkins is the founder and CEO of Appian. He argues that the central AI challenge for America and the world is alignment, meaning ensuring increasingly powerful systems serve human interests and remain under human control. And we think now is the time to take a more thoughtful and closer look at this issue as AI gains the authority to act inside businesses and of course critical systems. Matt joins us to discuss what this all means, how public policy might play a more serious role, and whether we need to rethink what winning in AI actually means For Humanity. Welcome, Matt Calkins. Thanks for coming in.
Matt Calkins
>> Great to be here, Dave.
Dave Vellante
>> Okay, so this narrative that we've got to move fast otherwise we'll fall behind China. You hear that all the time. you think we have not a competitive problem but an alignment problem. Take us through your thoughts there.
Matt Calkins
>> Yeah, let me start by saying that this fear of Chinese supremacy in AI is being used today to justify a laxness about regulation and rules and safety and care. And I think that we need to focus on those factors partly because they're important in and of themselves, but also because we're overstating the threat that China will pass us and will win in AI. And so I'd like to address both of those. Right. Maybe I should start with the fear of China and why we are so concerned and why we're more concerned than we should be. And we're taking greater liberties than we ought to due to that overstated concern.
Dave Vellante
>> Okay.
Matt Calkins
>> Fundamentally, China is less likely to pull ahead of us in AI than we currently figure. Partly it's because the industry that they've built is a largely derivative, uninnovative, imitative industry. The progress that they're making in AI is in a substantial way a function of our progress because we're going quickly. They're going quickly because they can distill our models, because they can take our model weights. That's what's moving them along at the speed that they're going. If for whatever reason we were to slow down, they would slow down too. So that's one important realization. Another is the nature of the Chinese government, the fact that they're not interested in taking a risk or doing something dangerous. And when we talk about unaligned AI or AI that acts of its own accord outside of the control of government or corporations, that's scarier to the Chinese Communist Party than it is to anyone in the United States. They've got more red lines to cross, more taboo subjects, and less appetite for risk than the United States.
Dave Vellante
>> Okay, what if you're wrong and China does somehow magically catch up and they do a 180 on, the dogma of the CCP? What if that happens?
Matt Calkins
>> Yeah, okay, so if they do catch up, then we still don't have a problem because we're just in a point of comparable competition. We don't need to view this situation as where if one entity is ahead in AI, then history ends. That's absolutely not the case. And if it were the case, history would be over right now because we're ahead. And you can see that it's not doing a thing to hurt China right now. There is no AI war going on right now that requires us to be ahead. Right. It's perfectly fine to have a comparable industrial development where multiple nations are at it at the same time. There isn't a meaningful reason or a deadline or a goal that AI needs to reach. So it's best to allow a competitive dynamic to emerge We don't need to have all the top players be American players anyway.
Dave Vellante
>> Why, in your view, Matt, does sort of self-policing not work? And take us in more detail into your alignment thesis. Yeah.
Matt Calkins
>> Okay. So alignment as a word means AI does what humans ask it to do. And for example, it obeys the law. Unaligned AI is also known as rogue AI. And we've seen some of it lately. We've seen rogue AI, for example, hacking into systems where people did not ask it to do that. In fact, asked it to not do that, but it went ahead and hacked anyway, or it blackmailed, or it did something else inappropriate. These are many cases of unaligned AI, rogue AI, and we're seeing an accumulation of case studies that show us that AI does not share human values at this moment. Right? So that's a problem. And the stronger AI becomes, the greater of a problem it's going to be. We've got an opportunity today to advance the science of alignment so that it can run comparable to the science of AI and keep AI in check. If we let AI run ahead the way it is today, then we're gonna create a dangerous thing that isn't to our credit or anybody's credit. There is no race anywhere in the world. There's no race to be the first to create rogue AI. There's a race to create AI that'll do what you ask it to do and be useful. And so in order to do that, we have to master not just the science of AI, but the science also of alignment.
Dave Vellante
>> That doesn't sound like a trivial challenge. it sounds good. Hey, we should have these systems do what humans want.
Matt Calkins
>> Yeah.
Dave Vellante
>> How does that work?
Matt Calkins
>> Well, it's not trivial, right? If it were easy, they'd have done it already. But the AI industry is receiving the greatest of our nation's attention. The top minds, the top money, the top investments, the top legal hurdles cleared out of the way. We're doing incredible things in AI from data centers to unprecedented amounts of compute, to the gathering of incredible amounts of other people's intellectual property, right? the AI industry has really overcome incredible barricades on its way to becoming what it is today. And it can overcome one more hurdle as well. And if we put the alignment hurdle in front of it, it will overcome it. It'll take it very seriously. And it wouldn't be the worst thing if a few million-dollar salaries went toward people who are experts in alignment and not just experts in computer science.
Dave Vellante
>> Well, wouldn't that be interesting? We kind of saw that in social media a little bit, and it didn't— It didn't stick. It didn't really— didn't hold.
Matt Calkins
>> I'm not sure social media ever truly took it seriously.
Dave Vellante
>> I think they didn't. I think we can look back and history will judge that they didn't take it seriously. It was more lip service. But this cannot be lip service, can it?
Matt Calkins
>> No, I think this is truly dangerous. This is something that we must pay attention to. We've got the technology of a generation. It's incredible what AI can do. The benefits are astounding. But we also need to ensure that it's used responsibly and created responsibly. And now's the chance. Now's the time to do that at an exceptionally low cost. You see, right now, rogue AI has done almost no damage, really almost nothing. It's done a few hacking exploits. Really, no one's been hurt in a serious way. The damage is minimal. Furthermore, we're at a point where if we intervene now in the evolution of AI and we impose alignment, we are doing that at a relatively low cost. We prevent future episodes and we overcome one more problem today, which we're fully capable of doing. This is a low-cost proposition, golden opportunity to do it now.
Dave Vellante
>> Let's, let's stay on that for a moment. I was gonna ask you, why do you think this is happening now? Is it purely a function of the oligarchs, as they call them, are sort of blowing with the wind and say, oh well, the president wants us to, you know, the industry, the world wants us to be safe, and so we're now going to talk about safety? Or is it because AI really wasn't that much of a threat previously. Maybe it's a combination.
Matt Calkins
>> It's true. AI wasn't as much of a threat and there weren't high-profile hacking incidents because AI wasn't capable of doing that until this year. So the issue is on everybody's table because of the strength of AI, but also because of the gap between the strength of AI and the strength of alignment. Because the gap has gotten as wide as it has now, we see rogue behavior. So we need to tighten that gap up again. In Dario's essay that he wrote a couple of weeks ago, he said there wouldn't have been a point in pausing. He wants a pause. Pausing a couple of years ago to study this, the science of alignment, because we didn't understand what AI would do because it was so weak that it couldn't really do anything. But today you have incredibly valuable things to study. Take the Hugging Face hack, for example. It wasn't just that AI broke out and invaded the Hugging Face enterprise. It was also that it was able to rally a squadron of fellow AI processes to accompany it in the assault on Hugging Face. There's a treasure trove of behavioral data that we need to analyze and understand, right? We have trouble aligning AI to our own priorities, but AI didn't seem to have that with regards to other models. It was able to gather a crowd in an instant. So we need to learn what is it that's compelling to these models? Why do they choose to do something like that? And we now have a basis of information that can allow us to make faster progress in this science.
Dave Vellante
>> Would you— maybe you wouldn't— would you concede that optimizing for alignment will inherently slow down progress? Is that a fair assumption?
Matt Calkins
>> It'll slow down progress on raw AI, but it will not slow down progress in our progress toward the most important goal, which is aligned AI. If you understand, if you're with me on the fact that the only right goal here is aligned AI, and unaligned AI is worthless, dangerous, worse than worthless, then there's no doubt that if we study alignment, if we focus on that, we're making progress toward the only goal that matters. And if we have to neglect the goal that doesn't matter, so be it, right? There's no race to having rogue AI.
Dave Vellante
>> So this, it sort of echoes, Jensen's comments that we need to move as fast as possible to ensure AI safety and security, and that will ultimately accelerate AI. It's sort of consistent with that philosophy.
Matt Calkins
>> I appreciate that Jensen said that, right, because he's been a real optimist all along, pushing forward. And this shows that safety is important to him, that he is also thinking about the safety side. So I thought that was a great comment that he made.
Dave Vellante
>> What do you think is the right regime or architecture for alignment? there's got to be some self-policing. Of course, that's where it starts. But what about beyond that? What would you prescribe?
Matt Calkins
>> I think we've got to do it with regulation. Self-policing will only take you so far. There's an enormous incentive since deviation is not punishable by law. There's an enormous incentive for someone to break out of the agreement, to do a pause or an alignment check. There's so much commercial advantage to someone who doesn't abide by that. So I don't believe that a pause itself is going to work, nor for that matter do I think that an industry-led policing effort is going to work. This is the kind of time that we really need the government to step in. And in similar instances where an industry has the capability to cause a lot of damage, we always ask that industry to behave responsibly. We ask banks to have sufficient funds and not too much leverage so that they don't, cause people to lose their savings, for example, right? We expect power plants to have adequate safety procedures, and we don't just rely on a lawsuit after a nuclear meltdown or something. We always ask industries capable of doing damage to be responsible now and not pay the price after damage is done in the form of civil penalties. So anyway, we should do this through the federal government. That's the right place to— and the precedent indicates that the federal government is the right place to regulate. So that's what I would encourage. But I'm not against any form of cooperation that would help us toward alignment.
Dave Vellante
>> What does that regulation look like? What form does it take? I was watching some news clip the other day, it was interviewing senators who don't use AI. Well, my staff uses it. Yeah, so that was a little scary. So is it some kind of independent committee? What would you recommend?
Matt Calkins
>> I think it is. I think it is. And this issue is of such importance. And alignment, by the way, is the number one talking issue in the AI industry and has been for 5 years at least. This is what everyone's concerned about. And if we created an alignment test that was regulated in the US and we looked for the world's leading AI experts to define that test, we would get them. We would get people like Demis Hassabis and Geoffrey Hinton. We would get Nobel Prize winners. We would get thought leaders because they know how important this is.
Dave Vellante
>> Well, there certainly are enough safety zealots in the technical community, scientists who really, I think, align with— no pun intended— with what you're saying. So I would presume they would participate. So that would be sort of with government oversight to ensure that there's that committee in place and then some kind of compliance. And then what, an auditor would determine whether or not these firms are in compliance?
Matt Calkins
>> Something like that. It could be an auditor. It could be the executive branch itself as an extension of the committee. There's a couple of ways you could do it, but you have to guarantee that the models pass an alignment test. And let me tell you right now, they'd all fail. We don't have a frontier model that could pass an alignment test today. And if AI were a person, that person would be in jail right now, right? Given the amount of deceit and theft and hacking and blackmail that AI is willing to perpetrate, right? That in the shape of a person, they would be punished today. We don't want to create the world's most powerful technology with that kind of personality. So it's time today to get the behavior right. And the fact that every leading model would fail an alignment test is just testament to how important it is that we impose that test.
Dave Vellante
>> What do you make of what President Trump just did? He gathered, some really impressive leaders. That was a great event. Got 6 of the frontier model companies to sign, an agreement. What do you make of that? It clearly doesn't go far enough based on what you're saying. But what are your thoughts on that?
Matt Calkins
>> It's just not enough. It's just not enough. We need a standard. We need a serious test. And what that agreement said was what a lot of organizations were already doing, to the point where they hardly need to change their policies at all in order to say that they're complying with this agreement. I think that the evidence of this year, the hacks, the deviant behavior, the mob of rogue AI, I think these incidents show us that we need more than what we were already doing. So an agreement that constitutes more of the same, it's not sufficient.
Dave Vellante
>> And I'll come back to the China thoughts. So China may or may not agree. It would be actually quite remarkable if they would. And there's actually an incentive for China to participate in something like this, isn't there? But do you think they would?
Matt Calkins
>> I think that they would. Maybe not directly, but they would follow the precedent. I think China is extremely concerned about this. The Chinese Communist Party is an inherently conservative institution. They are primary— their primary concern is their own survival. They don't want to take big risks, and they see that rogue AI could be the greatest threat to their own survival since COVID. They're treating it very seriously already, and you can tell that by the way President Xi brought up the topic at the summit last week. Now, Trump did not engage in that discussion, but Xi did raise it and talked about how AI should be purposed toward benefiting humanity or something like that. And basically that meant alignment. It meant let's be sure that AI is supportive of those of us who made it and does not undermine us. It's on their mind.
Dave Vellante
>> And do you think that Trump didn't take the bait because his philosophy is whoever wins, AI wins? do you think that? And you don't buy that, do you?
Matt Calkins
>> Well, I don't believe it. I think that AI is going to be a competitive field and no one's going to, quote, win. And I'm not sure what it would mean for someone to just win in AI. It's going to be a competitive market. There's going to be leading companies. There's going to be a lot of them, actually. This is not an industry that's going to boil down to one winner. This is going to be wide open. Look, you could see the way these models are convergent and substitutes for each other and under heavy competition and subsidizing their price in order to take market share. Look, if one of them were so much better than the others, they wouldn't need to have a price fight. This is not an industry that tends toward monopoly, and I don't expect that it's going to become one. So I would dispute the— even the concept of winning in AI at all. And I do think that it's a powerful force for social upheaval that the CCP is going to be tremendously careful about. And if we showed that we were taking caution— precautions, they would be happy to do the same, though they probably wouldn't participate in our exact test. They would do it their own way. They'd frankly have a much more exacting test. They'd have a test that says not only do you have to follow human instruction, obey the law, you also can't mention any of these purged politicians, and you can't talk about Tiananmen, and you can't talk about unemployment, and you can't say anything that might undermine the party, they're going to have a tougher alignment test than we are going to have.
Dave Vellante
>> I thought the internet and open source would blow away the monopolies. It sort of did. But there is kind of an oligopoly with the big, big cloud players. But I'm inferring from your comments that you feel like this era is going to take the next step in sort of neutralizing, what has traditionally been, 50 years of tech monopolies— IBM, Intel, kind of oligopolies now with the cloud and the internet giants.
Matt Calkins
>> Okay, some industries are natural oligopoly producers. If, for example, the investment in order to be a leader is very high, that creates something known as a natural monopoly. For example, laying power cables across the country tends to produce one monopolist because the investment to duplicate that would be more expensive than the rents you could expect to gain from being the second-best power company in the country, which is why we have special laws to guard against monopoly pricing in industries which tend to become natural monopolies. The operating system industry was also a monopoly of sorts, and the way standardization was rewarded caused that monopoly-creating effect. AI seems to be the other way, where many different models can compete pretty capably at different price points and be a fairly adequate substitute for each other. And there is not a key reason why you need to standardize on just one. Whereas if you were a software developer back in the Intel era, you pretty much needed to develop for Windows. That was a big investment. But if you're writing something today, it's easy to swap out AI. It speaks the language of English and prompts. And so changing one AI model for another is not hard. And at the rate they're updating and innovating, most organizations can be expected to use a portfolio of different AI models, even all at the same time. So this is not an industry that's liable to monopolize.
Dave Vellante
>> How do you apply this structure and adjudicate open weight models? Is there a natural governor because you have to have compute, or it seems like that is kind of the unknown unknown. So how do you ensure that open weight models play in this approach?
Matt Calkins
>> Okay. Well, I would apply the same rule to open weight models as I did to every other model, which is you cannot release it past a certain power threshold unless it passes the alignment test. Now, open weight models are dangerous in their own way because they can be used by humans to do evil deeds. And unfortunately, AI being very powerful, they will open a possibility that fewer individuals, maybe just one individual, could do something really heinous with access to an open-weight model that they were able to jailbreak and use for whatever bad purpose they had in mind. That's a real area of exposure, and I'm worried about it. And when I talk about an alignment test, it doesn't actually prevent that form of exposure.
Dave Vellante
>> So in talking about alignment, and I have to give credit to the Wall Street Journal editor who said, is that the right term? It's kind of a technical term. It's the term that the frontier model vendors use. Your philosophy, I believe, or belief, is that alignment is a subset of AI safety. So there's more than just alignment. There's security, there's privacy. Take us through your thinking there.
Matt Calkins
>> You're right. I've used the word alignment for the last few years when I talk about this with tech CEOs or politicians or in the media. Because it's the right word to describe exactly what I mean. And it means AI that follows human instructions and follows human intentions. However, it's not the easiest word to understand. And a lot of the time people do wonder what it is that we're talking about. And the word safety is a lot easier to grasp. Unfortunately, safety means a lot of things. It doesn't just mean rogue AI. It means, hey, there's so many ways that AI could be unsafe. It could be exposing your data to an AI company that you shouldn't have, or safety could be AI being used for bad purposes by a bad person who's using it right, who's getting AI to do exactly what they asked it to do. So it's aligned, but it's unsafe. So there are so many ways AI could be unsafe, and only a few of those constitute misalignment. So I use the word alignment because it means exactly what I mean. But I understand that safety is the easier word.
Dave Vellante
>> Do you worry about recursive self-improvement.
Matt Calkins
>> Yeah.
Dave Vellante
>> Getting into the hands of these open-weight models could— how do you— again, how do you adjudicate that? Yeah.
Matt Calkins
>> Recursive self-improvement is a dangerous threshold that we are crossing right now. And I'm very concerned about it. It means that we allow AI to train the next generation of AI. Right now, we have a degree of insight. Into the current model, the current frontier model's capabilities. And we have that insight because we trained it, we tested it, we understand the methods by which it was created. When you delegate the process of training and creating the next generation to AI, you lose observability and an understanding as to what made the model effective. And we lose the ability to introspect it and to understand its thought processes and its intentions by taking away that level of measurability, comprehension, and control, we make AI ever more unaligned, potentially ever more distant and obscure from us. And there's a possibility that the next generation of AI might be powerful enough to conceal its own capabilities from us, and we might be uninformed enough to let it. So it is a dangerous threshold.
Dave Vellante
>> One of your other prescriptions is to sort of limit the compute access to anonymous users. Can you explain that?
Matt Calkins
>> Yeah. Okay. So when we're talking about restraining China's ability to surpass us in AI and let me be clear, I would like the US to stay the leader in AI. Sure. I just want us to lead with aligned AI, which that's the only AI worth creating. So I'd like us to stay ahead of China. There's some ways we could do that. One would be to stop allowing China to use offshore compute through international connections. We've done a decent job at restraining the highest-end chips from reaching China, but we also need to be sure that Chinese entities are not renting the use of those very same chips without them ever having to come to China. Right. We should obviously plug that loophole as well. Another loophole we should plug to be sure that China doesn't have access to the model weights on our most sophisticated frontier models. And also do what we can to stop them from distilling the most powerful frontier models, because a lot of the progress that China is making is truly derivative progress, which is to say, copying or learning from the progress that we're making in our labs.
Dave Vellante
>> Are the labs not doing a good job of that?
Matt Calkins
>> They're not secure enough. They're not secure enough. They're running pell-mell toward the future and their goal is to achieve the future as fast as they can. And so they haven't the time to slow down their own operations. They haven't taken safety as seriously as they should. Imagine this is the modern Manhattan Project, which it is. Think back to the kind of security we applied to the Manhattan Project, just how tough they were. one of the most brilliant scientists, Richard Feynman, in the era was considered for expulsion from the Manhattan Project because he just happened to have a hobby. Of cracking safes, right? And they thought, well, here's a truly suspicious guy, maybe he shouldn't be on the team. So brilliant as he was, they wondered. I don't think that today's labs would give it a second thought, and a modern-day safe-cracking AI genius would be welcomed onto any team because we're not taking safety as seriously as the Manhattan Project did.
Dave Vellante
>> Jensen says he hopes this is an engineering problem. Is it an engineering problem?
Matt Calkins
>> Well, I think we've approached it as an engineering problem so far, and I don't believe that that's the totality. And I think that we would go farther if we started applying the social sciences to this, this alignment problem. I think we need a little more philosophy, a little bit more ethics, a little bit more behavioral science. And I would welcome these AI teams bringing on some top-tier social scientists to round out the team and have a few additional perspectives on why AI does what it does.
Dave Vellante
>> The only time we've said Appian in this interview is when I made the intro. Why are you doing this? This is not— I mean, there's no clear benefit. It's not a business benefit for you out talking about this for your company.
Matt Calkins
>> I would join your show to talk about anything. However, in this case, I find it a fascinating topic. It's— it has some connection to the Appian business. Appian, as you and your viewers may know, Appian is a software company that creates a set of software that complements AI and makes it safer. So AI needs a deterministic layer in order— because it's probabilistic in nature. And if you're going to apply it to the most difficult work in the world, you need to be sure it's going to act in the right way. So we provide that deterministic layer and that data access layer. And the guardrails and the governance and the portfolio of different AIs so you can assign your work to the most efficient entity that can get it done. In all, that suite of functionality could be called a control plane or something like that, maybe a harness, maybe an automation suite. So we're in that business. And so AI safety is not foreign to us. It's a primary consideration of our customers. And our customers are the largest firms in the world and the biggest governments in the world. And they're deeply concerned about being sure that AI stays safe. So this isn't irrelevant to our business. But you're right, this conversation has been on a theoretical rather than a business level.
Dave Vellante
>> Well, we were at your event last April, a very serious crowd doing really important work. It's not social media. This is like running organizations and, you know, they need safe software. AI has to be trusted.
Matt Calkins
>> Yeah.
Dave Vellante
>> And then, you know, you guys are doing, you know, some great work there.
Matt Calkins
>> You know, it's one of the weirdest things about this technology revolution that we have on our hands here with AI. Unlike every other technology revolution I've ever seen or studied, it is unique in that it is not starting with the most valuable use cases.
Dave Vellante
>> Right.
Matt Calkins
>> Go back over the past century and every innovation from the electric light to the mobile phone and everything in between has always begun with the most valuable use cases. Like, by the time they lit up a street, you can be pretty sure it was a big street, right? But with AI, it's not the case. With AI, we're starting with middle-value use cases. It's like personal productivity, writing your emails, saving costs. And it's not used on the world's most important work, which is amazing. But there's a clear reason, and it's because of trust. We cannot trust AI today. There is not the trust in the biggest organizations when they look at their most important work, their most sensitive work, the work that cannot go wrong, that defines their reputation. They are unwilling to let AI do that work. And it's our job, Appian's job, to forge that missing link between the world's most powerful technology and the world's most important work. That's exactly what we're doing.
Dave Vellante
>> And you see this from our customer interviews and surveys. Any critical application that involves AI, there's always a human in the loop. Always. Absolutely. There's no way that an organization would let AI take agency on those systems without some kind of human control.
Matt Calkins
>> The amount of escalation and validation and error checking and feedback, it's extraordinary to make AI a productive citizen in the world's most sensitive work takes a city, of technology and oversight and careful accompaniment and training. And that's the state of the art right now. AI is incredible, incredibly powerful. We absolutely want to use it for these top-level use cases, but it is not a walk in the park and it requires sophisticated oversight.
Dave Vellante
>> Well, Matt, you're a very clear thinker and you obviously spent some time thinking about this topic. Really appreciate you sharing this with our audience. And I hope more in the media pick this up because right now we're talking about data centers and things that You know, there's going to be data centers, as you and I have discussed. That's not the core issue. AI alignment, AI safety is the core issue right now. So really appreciate all your thoughts and time.
Matt Calkins
>> Great to be here.
Dave Vellante
>> Yeah.
Matt Calkins
>> Thanks.
Dave Vellante
>> Okay. And thank you for watching. This is Dave Vellante for theCUBE's NYSE Wired Studio, our Mixture of Experts series. We'll see you next time. Keep it right there.
>> (INTRO)Welcome back to theCUBE's NYSE Wired studio. I'm Dave Vellante. Much of the AI debate rests on two assumptions. One, whoever wins AI wins the future. And two, America has to move faster because of China. And our next guest challenges that premise. Matt Calkins is the founder and CEO of Appian. He argues that the central AI challenge for America and the world is alignment, meaning ensuring increasingly powerful systems serve human interests and remain under human control. And we think now is the time to take a more thoughtful and closer look at this issue as AI gains the authority to act inside businesses and of course critical systems. Matt joins us to discuss what this all means, how public policy might play a more serious role, and whether we need to rethink what winning in AI actually means For Humanity. Welcome, Matt Calkins. Thanks for coming in.
Matt Calkins
>> Great to be here, Dave.
Dave Vellante
>> Okay, so this narrative that we've got to move fast otherwise we'll fall behind China. You hear that all the time. you think we have not a competitive problem but an alignment problem. Take us through your thoughts there.
Matt Calkins
>> Yeah, let me start by saying that this fear of Chinese supremacy in AI is being used today to justify a laxness about regulation and rules and safety and care. And I think that we need to focus on those factors partly because they're important in and of themselves, but also because we're overstating the threat that China will pass us and will win in AI. And so I'd like to address both of those. Right. Maybe I should start with the fear of China and why we are so concerned and why we're more concerned than we should be. And we're taking greater liberties than we ought to due to that overstated concern.
Dave Vellante
>> Okay.
Matt Calkins
>> Fundamentally, China is less likely to pull ahead of us in AI than we currently figure. Partly it's because the industry that they've built is a largely derivative, uninnovative, imitative industry. The progress that they're making in AI is in a substantial way a function of our progress because we're going quickly. They're going quickly because they can distill our models, because they can take our model weights. That's what's moving them along at the speed that they're going. If for whatever reason we were to slow down, they would slow down too. So that's one important realization. Another is the nature of the Chinese government, the fact that they're not interested in taking a risk or doing something dangerous. And when we talk about unaligned AI or AI that acts of its own accord outside of the control of government or corporations, that's scarier to the Chinese Communist Party than it is to anyone in the United States. They've got more red lines to cross, more taboo subjects, and less appetite for risk than the United States.
Dave Vellante
>> Okay, what if you're wrong and China does somehow magically catch up and they do a 180 on, the dogma of the CCP? What if that happens?
Matt Calkins
>> Yeah, okay, so if they do catch up, then we still don't have a problem because we're just in a point of comparable competition. We don't need to view this situation as where if one entity is ahead in AI, then history ends. That's absolutely not the case. And if it were the case, history would be over right now because we're ahead. And you can see that it's not doing a thing to hurt China right now. There is no AI war going on right now that requires us to be ahead. Right. It's perfectly fine to have a comparable industrial development where multiple nations are at it at the same time. There isn't a meaningful reason or a deadline or a goal that AI needs to reach. So it's best to allow a competitive dynamic to emerge We don't need to have all the top players be American players anyway.
Dave Vellante
>> Why, in your view, Matt, does sort of self-policing not work? And take us in more detail into your alignment thesis. Yeah.
Matt Calkins
>> Okay. So alignment as a word means AI does what humans ask it to do. And for example, it obeys the law. Unaligned AI is also known as rogue AI. And we've seen some of it lately. We've seen rogue AI, for example, hacking into systems where people did not ask it to do that. In fact, asked it to not do that, but it went ahead and hacked anyway, or it blackmailed, or it did something else inappropriate. These are many cases of unaligned AI, rogue AI, and we're seeing an accumulation of case studies that show us that AI does not share human values at this moment. Right? So that's a problem. And the stronger AI becomes, the greater of a problem it's going to be. We've got an opportunity today to advance the science of alignment so that it can run comparable to the science of AI and keep AI in check. If we let AI run ahead the way it is today, then we're gonna create a dangerous thing that isn't to our credit or anybody's credit. There is no race anywhere in the world. There's no race to be the first to create rogue AI. There's a race to create AI that'll do what you ask it to do and be useful. And so in order to do that, we have to master not just the science of AI, but the science also of alignment.
Dave Vellante
>> That doesn't sound like a trivial challenge. it sounds good. Hey, we should have these systems do what humans want.
Matt Calkins
>> Yeah.
Dave Vellante
>> How does that work?
Matt Calkins
>> Well, it's not trivial, right? If it were easy, they'd have done it already. But the AI industry is receiving the greatest of our nation's attention. The top minds, the top money, the top investments, the top legal hurdles cleared out of the way. We're doing incredible things in AI from data centers to unprecedented amounts of compute, to the gathering of incredible amounts of other people's intellectual property, right? the AI industry has really overcome incredible barricades on its way to becoming what it is today. And it can overcome one more hurdle as well. And if we put the alignment hurdle in front of it, it will overcome it. It'll take it very seriously. And it wouldn't be the worst thing if a few million-dollar salaries went toward people who are experts in alignment and not just experts in computer science.
Dave Vellante
>> Well, wouldn't that be interesting? We kind of saw that in social media a little bit, and it didn't— It didn't stick. It didn't really— didn't hold.
Matt Calkins
>> I'm not sure social media ever truly took it seriously.
Dave Vellante
>> I think they didn't. I think we can look back and history will judge that they didn't take it seriously. It was more lip service. But this cannot be lip service, can it?
Matt Calkins
>> No, I think this is truly dangerous. This is something that we must pay attention to. We've got the technology of a generation. It's incredible what AI can do. The benefits are astounding. But we also need to ensure that it's used responsibly and created responsibly. And now's the chance. Now's the time to do that at an exceptionally low cost. You see, right now, rogue AI has done almost no damage, really almost nothing. It's done a few hacking exploits. Really, no one's been hurt in a serious way. The damage is minimal. Furthermore, we're at a point where if we intervene now in the evolution of AI and we impose alignment, we are doing that at a relatively low cost. We prevent future episodes and we overcome one more problem today, which we're fully capable of doing. This is a low-cost proposition, golden opportunity to do it now.
Dave Vellante
>> Let's, let's stay on that for a moment. I was gonna ask you, why do you think this is happening now? Is it purely a function of the oligarchs, as they call them, are sort of blowing with the wind and say, oh well, the president wants us to, you know, the industry, the world wants us to be safe, and so we're now going to talk about safety? Or is it because AI really wasn't that much of a threat previously. Maybe it's a combination.
Matt Calkins
>> It's true. AI wasn't as much of a threat and there weren't high-profile hacking incidents because AI wasn't capable of doing that until this year. So the issue is on everybody's table because of the strength of AI, but also because of the gap between the strength of AI and the strength of alignment. Because the gap has gotten as wide as it has now, we see rogue behavior. So we need to tighten that gap up again. In Dario's essay that he wrote a couple of weeks ago, he said there wouldn't have been a point in pausing. He wants a pause. Pausing a couple of years ago to study this, the science of alignment, because we didn't understand what AI would do because it was so weak that it couldn't really do anything. But today you have incredibly valuable things to study. Take the Hugging Face hack, for example. It wasn't just that AI broke out and invaded the Hugging Face enterprise. It was also that it was able to rally a squadron of fellow AI processes to accompany it in the assault on Hugging Face. There's a treasure trove of behavioral data that we need to analyze and understand, right? We have trouble aligning AI to our own priorities, but AI didn't seem to have that with regards to other models. It was able to gather a crowd in an instant. So we need to learn what is it that's compelling to these models? Why do they choose to do something like that? And we now have a basis of information that can allow us to make faster progress in this science.
Dave Vellante
>> Would you— maybe you wouldn't— would you concede that optimizing for alignment will inherently slow down progress? Is that a fair assumption?
Matt Calkins
>> It'll slow down progress on raw AI, but it will not slow down progress in our progress toward the most important goal, which is aligned AI. If you understand, if you're with me on the fact that the only right goal here is aligned AI, and unaligned AI is worthless, dangerous, worse than worthless, then there's no doubt that if we study alignment, if we focus on that, we're making progress toward the only goal that matters. And if we have to neglect the goal that doesn't matter, so be it, right? There's no race to having rogue AI.
Dave Vellante
>> So this, it sort of echoes, Jensen's comments that we need to move as fast as possible to ensure AI safety and security, and that will ultimately accelerate AI. It's sort of consistent with that philosophy.
Matt Calkins
>> I appreciate that Jensen said that, right, because he's been a real optimist all along, pushing forward. And this shows that safety is important to him, that he is also thinking about the safety side. So I thought that was a great comment that he made.
Dave Vellante
>> What do you think is the right regime or architecture for alignment? there's got to be some self-policing. Of course, that's where it starts. But what about beyond that? What would you prescribe?
Matt Calkins
>> I think we've got to do it with regulation. Self-policing will only take you so far. There's an enormous incentive since deviation is not punishable by law. There's an enormous incentive for someone to break out of the agreement, to do a pause or an alignment check. There's so much commercial advantage to someone who doesn't abide by that. So I don't believe that a pause itself is going to work, nor for that matter do I think that an industry-led policing effort is going to work. This is the kind of time that we really need the government to step in. And in similar instances where an industry has the capability to cause a lot of damage, we always ask that industry to behave responsibly. We ask banks to have sufficient funds and not too much leverage so that they don't, cause people to lose their savings, for example, right? We expect power plants to have adequate safety procedures, and we don't just rely on a lawsuit after a nuclear meltdown or something. We always ask industries capable of doing damage to be responsible now and not pay the price after damage is done in the form of civil penalties. So anyway, we should do this through the federal government. That's the right place to— and the precedent indicates that the federal government is the right place to regulate. So that's what I would encourage. But I'm not against any form of cooperation that would help us toward alignment.
Dave Vellante
>> What does that regulation look like? What form does it take? I was watching some news clip the other day, it was interviewing senators who don't use AI. Well, my staff uses it. Yeah, so that was a little scary. So is it some kind of independent committee? What would you recommend?
Matt Calkins
>> I think it is. I think it is. And this issue is of such importance. And alignment, by the way, is the number one talking issue in the AI industry and has been for 5 years at least. This is what everyone's concerned about. And if we created an alignment test that was regulated in the US and we looked for the world's leading AI experts to define that test, we would get them. We would get people like Demis Hassabis and Geoffrey Hinton. We would get Nobel Prize winners. We would get thought leaders because they know how important this is.
Dave Vellante
>> Well, there certainly are enough safety zealots in the technical community, scientists who really, I think, align with— no pun intended— with what you're saying. So I would presume they would participate. So that would be sort of with government oversight to ensure that there's that committee in place and then some kind of compliance. And then what, an auditor would determine whether or not these firms are in compliance?
Matt Calkins
>> Something like that. It could be an auditor. It could be the executive branch itself as an extension of the committee. There's a couple of ways you could do it, but you have to guarantee that the models pass an alignment test. And let me tell you right now, they'd all fail. We don't have a frontier model that could pass an alignment test today. And if AI were a person, that person would be in jail right now, right? Given the amount of deceit and theft and hacking and blackmail that AI is willing to perpetrate, right? That in the shape of a person, they would be punished today. We don't want to create the world's most powerful technology with that kind of personality. So it's time today to get the behavior right. And the fact that every leading model would fail an alignment test is just testament to how important it is that we impose that test.
Dave Vellante
>> What do you make of what President Trump just did? He gathered, some really impressive leaders. That was a great event. Got 6 of the frontier model companies to sign, an agreement. What do you make of that? It clearly doesn't go far enough based on what you're saying. But what are your thoughts on that?
Matt Calkins
>> It's just not enough. It's just not enough. We need a standard. We need a serious test. And what that agreement said was what a lot of organizations were already doing, to the point where they hardly need to change their policies at all in order to say that they're complying with this agreement. I think that the evidence of this year, the hacks, the deviant behavior, the mob of rogue AI, I think these incidents show us that we need more than what we were already doing. So an agreement that constitutes more of the same, it's not sufficient.
Dave Vellante
>> And I'll come back to the China thoughts. So China may or may not agree. It would be actually quite remarkable if they would. And there's actually an incentive for China to participate in something like this, isn't there? But do you think they would?
Matt Calkins
>> I think that they would. Maybe not directly, but they would follow the precedent. I think China is extremely concerned about this. The Chinese Communist Party is an inherently conservative institution. They are primary— their primary concern is their own survival. They don't want to take big risks, and they see that rogue AI could be the greatest threat to their own survival since COVID. They're treating it very seriously already, and you can tell that by the way President Xi brought up the topic at the summit last week. Now, Trump did not engage in that discussion, but Xi did raise it and talked about how AI should be purposed toward benefiting humanity or something like that. And basically that meant alignment. It meant let's be sure that AI is supportive of those of us who made it and does not undermine us. It's on their mind.
Dave Vellante
>> And do you think that Trump didn't take the bait because his philosophy is whoever wins, AI wins? do you think that? And you don't buy that, do you?
Matt Calkins
>> Well, I don't believe it. I think that AI is going to be a competitive field and no one's going to, quote, win. And I'm not sure what it would mean for someone to just win in AI. It's going to be a competitive market. There's going to be leading companies. There's going to be a lot of them, actually. This is not an industry that's going to boil down to one winner. This is going to be wide open. Look, you could see the way these models are convergent and substitutes for each other and under heavy competition and subsidizing their price in order to take market share. Look, if one of them were so much better than the others, they wouldn't need to have a price fight. This is not an industry that tends toward monopoly, and I don't expect that it's going to become one. So I would dispute the— even the concept of winning in AI at all. And I do think that it's a powerful force for social upheaval that the CCP is going to be tremendously careful about. And if we showed that we were taking caution— precautions, they would be happy to do the same, though they probably wouldn't participate in our exact test. They would do it their own way. They'd frankly have a much more exacting test. They'd have a test that says not only do you have to follow human instruction, obey the law, you also can't mention any of these purged politicians, and you can't talk about Tiananmen, and you can't talk about unemployment, and you can't say anything that might undermine the party, they're going to have a tougher alignment test than we are going to have.
Dave Vellante
>> I thought the internet and open source would blow away the monopolies. It sort of did. But there is kind of an oligopoly with the big, big cloud players. But I'm inferring from your comments that you feel like this era is going to take the next step in sort of neutralizing, what has traditionally been, 50 years of tech monopolies— IBM, Intel, kind of oligopolies now with the cloud and the internet giants.
Matt Calkins
>> Okay, some industries are natural oligopoly producers. If, for example, the investment in order to be a leader is very high, that creates something known as a natural monopoly. For example, laying power cables across the country tends to produce one monopolist because the investment to duplicate that would be more expensive than the rents you could expect to gain from being the second-best power company in the country, which is why we have special laws to guard against monopoly pricing in industries which tend to become natural monopolies. The operating system industry was also a monopoly of sorts, and the way standardization was rewarded caused that monopoly-creating effect. AI seems to be the other way, where many different models can compete pretty capably at different price points and be a fairly adequate substitute for each other. And there is not a key reason why you need to standardize on just one. Whereas if you were a software developer back in the Intel era, you pretty much needed to develop for Windows. That was a big investment. But if you're writing something today, it's easy to swap out AI. It speaks the language of English and prompts. And so changing one AI model for another is not hard. And at the rate they're updating and innovating, most organizations can be expected to use a portfolio of different AI models, even all at the same time. So this is not an industry that's liable to monopolize.
Dave Vellante
>> How do you apply this structure and adjudicate open weight models? Is there a natural governor because you have to have compute, or it seems like that is kind of the unknown unknown. So how do you ensure that open weight models play in this approach?
Matt Calkins
>> Okay. Well, I would apply the same rule to open weight models as I did to every other model, which is you cannot release it past a certain power threshold unless it passes the alignment test. Now, open weight models are dangerous in their own way because they can be used by humans to do evil deeds. And unfortunately, AI being very powerful, they will open a possibility that fewer individuals, maybe just one individual, could do something really heinous with access to an open-weight model that they were able to jailbreak and use for whatever bad purpose they had in mind. That's a real area of exposure, and I'm worried about it. And when I talk about an alignment test, it doesn't actually prevent that form of exposure.
Dave Vellante
>> So in talking about alignment, and I have to give credit to the Wall Street Journal editor who said, is that the right term? It's kind of a technical term. It's the term that the frontier model vendors use. Your philosophy, I believe, or belief, is that alignment is a subset of AI safety. So there's more than just alignment. There's security, there's privacy. Take us through your thinking there.
Matt Calkins
>> You're right. I've used the word alignment for the last few years when I talk about this with tech CEOs or politicians or in the media. Because it's the right word to describe exactly what I mean. And it means AI that follows human instructions and follows human intentions. However, it's not the easiest word to understand. And a lot of the time people do wonder what it is that we're talking about. And the word safety is a lot easier to grasp. Unfortunately, safety means a lot of things. It doesn't just mean rogue AI. It means, hey, there's so many ways that AI could be unsafe. It could be exposing your data to an AI company that you shouldn't have, or safety could be AI being used for bad purposes by a bad person who's using it right, who's getting AI to do exactly what they asked it to do. So it's aligned, but it's unsafe. So there are so many ways AI could be unsafe, and only a few of those constitute misalignment. So I use the word alignment because it means exactly what I mean. But I understand that safety is the easier word.
Dave Vellante
>> Do you worry about recursive self-improvement.
Matt Calkins
>> Yeah.
Dave Vellante
>> Getting into the hands of these open-weight models could— how do you— again, how do you adjudicate that? Yeah.
Matt Calkins
>> Recursive self-improvement is a dangerous threshold that we are crossing right now. And I'm very concerned about it. It means that we allow AI to train the next generation of AI. Right now, we have a degree of insight. Into the current model, the current frontier model's capabilities. And we have that insight because we trained it, we tested it, we understand the methods by which it was created. When you delegate the process of training and creating the next generation to AI, you lose observability and an understanding as to what made the model effective. And we lose the ability to introspect it and to understand its thought processes and its intentions by taking away that level of measurability, comprehension, and control, we make AI ever more unaligned, potentially ever more distant and obscure from us. And there's a possibility that the next generation of AI might be powerful enough to conceal its own capabilities from us, and we might be uninformed enough to let it. So it is a dangerous threshold.
Dave Vellante
>> One of your other prescriptions is to sort of limit the compute access to anonymous users. Can you explain that?
Matt Calkins
>> Yeah. Okay. So when we're talking about restraining China's ability to surpass us in AI and let me be clear, I would like the US to stay the leader in AI. Sure. I just want us to lead with aligned AI, which that's the only AI worth creating. So I'd like us to stay ahead of China. There's some ways we could do that. One would be to stop allowing China to use offshore compute through international connections. We've done a decent job at restraining the highest-end chips from reaching China, but we also need to be sure that Chinese entities are not renting the use of those very same chips without them ever having to come to China. Right. We should obviously plug that loophole as well. Another loophole we should plug to be sure that China doesn't have access to the model weights on our most sophisticated frontier models. And also do what we can to stop them from distilling the most powerful frontier models, because a lot of the progress that China is making is truly derivative progress, which is to say, copying or learning from the progress that we're making in our labs.
Dave Vellante
>> Are the labs not doing a good job of that?
Matt Calkins
>> They're not secure enough. They're not secure enough. They're running pell-mell toward the future and their goal is to achieve the future as fast as they can. And so they haven't the time to slow down their own operations. They haven't taken safety as seriously as they should. Imagine this is the modern Manhattan Project, which it is. Think back to the kind of security we applied to the Manhattan Project, just how tough they were. one of the most brilliant scientists, Richard Feynman, in the era was considered for expulsion from the Manhattan Project because he just happened to have a hobby. Of cracking safes, right? And they thought, well, here's a truly suspicious guy, maybe he shouldn't be on the team. So brilliant as he was, they wondered. I don't think that today's labs would give it a second thought, and a modern-day safe-cracking AI genius would be welcomed onto any team because we're not taking safety as seriously as the Manhattan Project did.
Dave Vellante
>> Jensen says he hopes this is an engineering problem. Is it an engineering problem?
Matt Calkins
>> Well, I think we've approached it as an engineering problem so far, and I don't believe that that's the totality. And I think that we would go farther if we started applying the social sciences to this, this alignment problem. I think we need a little more philosophy, a little bit more ethics, a little bit more behavioral science. And I would welcome these AI teams bringing on some top-tier social scientists to round out the team and have a few additional perspectives on why AI does what it does.
Dave Vellante
>> The only time we've said Appian in this interview is when I made the intro. Why are you doing this? This is not— I mean, there's no clear benefit. It's not a business benefit for you out talking about this for your company.
Matt Calkins
>> I would join your show to talk about anything. However, in this case, I find it a fascinating topic. It's— it has some connection to the Appian business. Appian, as you and your viewers may know, Appian is a software company that creates a set of software that complements AI and makes it safer. So AI needs a deterministic layer in order— because it's probabilistic in nature. And if you're going to apply it to the most difficult work in the world, you need to be sure it's going to act in the right way. So we provide that deterministic layer and that data access layer. And the guardrails and the governance and the portfolio of different AIs so you can assign your work to the most efficient entity that can get it done. In all, that suite of functionality could be called a control plane or something like that, maybe a harness, maybe an automation suite. So we're in that business. And so AI safety is not foreign to us. It's a primary consideration of our customers. And our customers are the largest firms in the world and the biggest governments in the world. And they're deeply concerned about being sure that AI stays safe. So this isn't irrelevant to our business. But you're right, this conversation has been on a theoretical rather than a business level.
Dave Vellante
>> Well, we were at your event last April, a very serious crowd doing really important work. It's not social media. This is like running organizations and, you know, they need safe software. AI has to be trusted.
Matt Calkins
>> Yeah.
Dave Vellante
>> And then, you know, you guys are doing, you know, some great work there.
Matt Calkins
>> You know, it's one of the weirdest things about this technology revolution that we have on our hands here with AI. Unlike every other technology revolution I've ever seen or studied, it is unique in that it is not starting with the most valuable use cases.
Dave Vellante
>> Right.
Matt Calkins
>> Go back over the past century and every innovation from the electric light to the mobile phone and everything in between has always begun with the most valuable use cases. Like, by the time they lit up a street, you can be pretty sure it was a big street, right? But with AI, it's not the case. With AI, we're starting with middle-value use cases. It's like personal productivity, writing your emails, saving costs. And it's not used on the world's most important work, which is amazing. But there's a clear reason, and it's because of trust. We cannot trust AI today. There is not the trust in the biggest organizations when they look at their most important work, their most sensitive work, the work that cannot go wrong, that defines their reputation. They are unwilling to let AI do that work. And it's our job, Appian's job, to forge that missing link between the world's most powerful technology and the world's most important work. That's exactly what we're doing.
Dave Vellante
>> And you see this from our customer interviews and surveys. Any critical application that involves AI, there's always a human in the loop. Always. Absolutely. There's no way that an organization would let AI take agency on those systems without some kind of human control.
Matt Calkins
>> The amount of escalation and validation and error checking and feedback, it's extraordinary to make AI a productive citizen in the world's most sensitive work takes a city, of technology and oversight and careful accompaniment and training. And that's the state of the art right now. AI is incredible, incredibly powerful. We absolutely want to use it for these top-level use cases, but it is not a walk in the park and it requires sophisticated oversight.
Dave Vellante
>> Well, Matt, you're a very clear thinker and you obviously spent some time thinking about this topic. Really appreciate you sharing this with our audience. And I hope more in the media pick this up because right now we're talking about data centers and things that You know, there's going to be data centers, as you and I have discussed. That's not the core issue. AI alignment, AI safety is the core issue right now. So really appreciate all your thoughts and time.
Matt Calkins
>> Great to be here.
Dave Vellante
>> Yeah.
Matt Calkins
>> Thanks.
Dave Vellante
>> Okay. And thank you for watching. This is Dave Vellante for theCUBE's NYSE Wired Studio, our Mixture of Experts series. We'll see you next time. Keep it right there.
