Cloud AWS re:Invent Coverage | Rand Fitzpatrick, Salesforce

Clips
News
More from Cloud AWS re:Invent Coverage

Rand Fitzpatrick

Senior Director, Product Management

Salesforce

Salesforce’s Heroku emerges as a valuable developer tool for AI app deployment

As part of the Salesforce Inc. family of offerings, Heroku has become a useful tool for developers to build, run and operate applications entirely in ...

play_circle_outline Heroku's approach to AI applications and managed inference

play_circle_outline Unifying Critical Data, Code, and AI Services: Seamless AI Model Access on Heroku

play_circle_outline Heroku's collaboration with Agentforce for extending application logic

Info
Transcript

Rand Fitzpatrick, Salesforce

Rand Fitzpatrick

Senior Director, Product Management Salesforce

Rand Fitzpatrick, senior director, product management at Salesforce, discusses AI applications and managed inference on Heroku. The goal is to make AI more accessible and graceful for developing meaningful applications. Heroku unifies core business data, application code, and AI services within a secure environment. They offer AI models seamlessly, allowing developers to focus on business value without managing infrastructure. Challenges include integrating AI into workflows and ensuring security. Heroku collaborates with Agentforce to extend application logi... Read more

explore Keep Exploring

What is the current approach of Heroku towards making AI applications more accessible and graceful for companies? add

What is the approach Salesforce is taking to unify core business data, application code, and AI services within their trust envelope in the Heroku ecosystem? add

What is Agentforce and how can it be used to apply cutting edge AI to critical business value flows within Salesforce? add

bolt Powered by CUBE AI

Rand Fitzpatrick, Salesforce

search

Rebecca Knight

>> Hello everyone and welcome back to theCUBE Cloud AWS re:Invent Coverage. I'm your host, Rebecca Knight. Right now we are joined by Rand Fitzpatrick. He is the senior director, product management at Salesforce. Thank you so much for coming on the show, Rand.

Rand Fitzpatrick

>> My pleasure. Thanks for having me.

Rebecca Knight

>> So let's get right in. AI is everywhere. It dominates a lot of conversations that we're having here on theCUBE and it feels as though lots of companies are jumping on the AI bandwagon. What is the latest from Heroku regarding AI applications and managed inference? I'd love to hear more about that.

Rand Fitzpatrick

>> To contextualize it a little bit, I've been working in AI on and off since the late '90s and have seen a lot of trends come and go. I could be framed as a little bit of an AI curmudgeon. I think that this wave is different in that there are actually real useful, compelling applications so long as one can get past a bunch of the hyperbole and voice. And Heroku is well known for taking things that are complicated and valuable and making them graceful and accessible. So when I take a look at what's going on in the AI landscape right now, we're seeing a lot of things that are really compelling for a lot of companies to do with AI, but a lot of the moving parts can be pretty hard to operate, manage and scale. So what we're doing at Heroku is making those things a lot more accessible and graceful for the actual construction of meaningful applications that will drive value without you getting lost in the noise.

Rebecca Knight

>> I want to get into all of that, and I love your description as a self-styled AI curmudgeon. From your perspective as a platform, as a service provider, how are you approaching AI and what is the game plan here to make sure you are putting out the noise?

Rand Fitzpatrick

>> Right. So there are a number of interesting pieces to consider when you're trying to get value out of AI. Most of the time to get value out of AI, you need a lot of your data, interesting and valuable data accessible to the AI service that you're going to be working with. Now that might be used for retrieval , in which case you're creating a lot of embeddings and doing additional calls per query. That might be for fine-tuning, that might be full training. In any of these cases, making sure that your core critical business data is actually going to be accessible within the context of those AI systems is crucial. When you take that the next step further and want to do that in the context of your applications, now you're dealing with three really crucial sensitive bits of data, your core business data, your application code, and the AI services themselves. A lot of companies are forced to spread these things out across a broad range of providers in order to actually make these things work. We're offering a way to unify all of these within the trust envelope that Salesforce provides within the Heroku ecosystem. So our dynamic compute environments have great isolated networking compute facilities that offer you great geographic location, great network isolation, great security around your compute, same with your data. And now we're unifying AI within that so that we're giving you a really comprehensive solution to build and scale these things within that secure and trusted environment. We're going further than that though. There's a lot of noise out there and we're taking a very opinionated approach to making the AI models that are going to be most valuable for most use cases available in a very seamless way. So within Heroku, we take a lot of pride in our developer and operator experiences. If you've ever instantiated a Postgres database or similar on the platform, it is often as simple as Heroku add-ons, create Postgres from within the CLI, and we automatically provide you with the environment variables that make it seamless to call these things directly from your application code. We're doing the same with AI to say, Heroku AI model's free, specify the model that you would like and we'll instantiate secure access to that model for use within your application. We also provide a lot of nice developer affordances so that you can make it really nice to be able to debug, test, understand, and develop against these things. So it's a pretty comprehensive service that gives you a lot of nice affordances that your developers can focus on the business value of your application and the utility of your application without having to manage all of the infrastructure, all of the tuning, all of the nuance of these things, and to do so in a unified way.

Rebecca Knight

>> You mentioned the developer experience. I want to dig in there a little bit because there's a lot of growing expectations that AI will be integrated into workflows and applications. What do you make of these challenges and what are some of the biggest challenges that you think developers are confronting right now?

Rand Fitzpatrick

>> Goodness, there are so many different ways to interact with so many of these different models, whether that be the system prompts that are involved with them or the context windows that they can be used with or the differences in API structure that they can tolerate. The variety is quite complicated as you might want to switch between models or use many models at once. Unifying that into a more consistent and approachable interface that is predictable and extensible for developers has been a goal of ours, as well as making sure that the tools that they might want to use alongside AI are secure and simple in their use. So in a lot of agentic workflows from a developer perspective, this involves using compute, memory and tools. And tools in this context can be databases, APIs, web services, application and points, things like that. And being able to actually give these references these tools to your AI can be complicated in a lot of scenarios. And managing the security there can also be quite complex because there's some things that you certainly don't want to pass off to a third party inference service.

Rebecca Knight

>> Well, exactly. And I want to get into that a little bit too. What do you see, or how do you see these changes affecting operations and security teams? Because as you said, security is critical.

Rand Fitzpatrick

>> Yeah, making sure that you have very explicit control on what data leaves your environment. What data is passed to an AI and what code is executed that might be generated by an AI is crucial for security. And those are just on the application environments of the world. On the AI model side of the world itself, looking at data provenance, making sure that your models are well vetted, making sure that you've got the appropriate safeguards in place for toxicity, misinformation, and other common concerns are all super, super important. So there's a broad range of security concerns across the operational application environment as well as the use of AI themselves, and we're baking in a lot of the best practices so that we have really good transparency and great controls for developers without having to reinvent the wheel and build it all themselves each time.

Rebecca Knight

>> So baking in best practices, what's Heroku's take on these challenges and how are you addressing them?

Rand Fitzpatrick

>> So we work very closely with a bunch of the leading model providers to make sure that all of the models are extremely well hardened, tested, and safeguarded. And we also work closely with our partners at AWS to make sure that all of the information is secure when we pass things in between the various resources in our environment. As you know, all of Heroku runs on AWS and we've got great shared practices in terms of security, isolation, boundaries and governance so that we make sure that all of the data exchange is visible, auditable, and transparent for use cases. So between the management of those guardrails and hardening of the AI models as well as the provenance and tracking and audibility of all the data flows, all of that should be out of the box, predictable, visible and trustable for our users.

Rebecca Knight

>> Okay. So as you started this conversation, you mentioned you've been working in AI since the '90s, which makes you a bit of an AI veteran. How are you seeing through, as you said, the noise in the complexity of this landscape and how is Heroku proposing to deliver value to customers? How do you describe Heroku's value proposition?

Rand Fitzpatrick

>> Heroku's value proposition is, and it has always been taking what is recently possible but complex and making it more graceful and accessible so that people can focus on the actual value. At the of Heroku, this was deploying Rails applications. It was great for developers in terms of experience and flow, super challenging to actually develop and urge to deploy and scale. We made that a lot easier and a lot more accessible to more people without them having to adopt additional expertise. With AI, we're doing much the same. We are trying to make it so that people can quickly understand from a curated and opinionated set of model options, what the best practices are to achieve their actual use case goals as opposed to chasing just the newest shiniest model that happens to pop up on Hugging Face or Hacker News. And then to be able to actually have the right tuning for that model for their use case and then to make it a clear and consistent operational part of their application. They don't need to worry about wrangling GPUs or managing complexity or tuning the KV cache or modulating the temperature, which is one of those hard to explain parameters on an AI model. Each time they want to use it, we actually set up best practices and help them auto-tune that for their use case. Then we make it very, very seamlessly accessible within their application so that they can program against it and build either applications that use AI directly or that construct AI agentic workflows within their application structure in ways that allow them to compose and extend to these things in a developer-native code-first way.

Rebecca Knight

>> We saw Salesforce launch Agentforce at Dreamforce. A lot of forces in there, but how does Heroku collaborate with Agentforce and what does that mean for customers?

Rand Fitzpatrick

>> So Agentforce is amazing. It is a beautiful system to apply some of the best cutting edge AI out there to super critical business value flows. And there's an amazing set of videos that I will not try to replicate here in terms of being able to construct these agentic workflows within Salesforce and compose them to drive a lot of your core business processes. There's this amazing capability with Heroku and Salesforce to extend any application logic on Heroku as actions that you can actually invoke within Agentforce. So if you have a need for access to your custom application logic that you are already running on Heroku, or if you need custom compute that doesn't naturally fit within some of the existing flows within Agentforce, we are an amazing extension point for that. So anytime you want a high code, high fidelity intense data processing in a variety of languages, Heroku becomes that natural seamless extension point. Now that can be done with anything on Heroku to extend Agentforce. The fact that we have a parallel developer-centric agentic play is a parallel track that can be used together. So if you wanted to be able to drive dynamic joining of unstructured, unspecified data via AI on the Heroku side, you can do that with or without Agentforce. If you want to be able to call that from within Agentforce, you can do that as well. These are really complimentary services that give developers and businesses a ton of optionality.

Rebecca Knight

>> And before I let you go, I want to ask you about your best advice to customers right now because you are someone with a lot of experience in AI and your commentary at the very start of our conversation about how a lot of customers... There is a lot of noise right now and a lot of customers are seeking the latest and greatest shiny next new thing. How do you talk to customers about this moment in time in AI and what is your advice to them as they are seeking to integrate AI into their companies and workflows?

Rand Fitzpatrick

>> This probably won't come as a surprise as someone who sits on the product management side of the world, but always focus on what problem you're trying to solve first. AI is an amazing tool and there has been a step function change in utilities since the advent of the transformer architecture, but it itself is not a reason to do something. If you've got an awesome opportunity and the value proposition in terms of ROI makes sense with AI, then it should be considered. And if you can make it past that hurdle, think really carefully about the domain that you're actually trying to model and make sure that you're using the right type of model in terms of size, modality, complexity, license, structure, speed for that use case. And I would typically advise people to use several smaller models that are custom suited to their domain cases as opposed to trying to use one giant model for all cases. This gives you a lot more control, a lot more flexibility and more ability to tune the ROI for your specific business cases without having to change everything all at once. So just like you would with any other technical service, thinking about how to make them modular, scalable, and built for purpose, there is no one size fits all solution.

Rebecca Knight

>> Excellent advice. Rand Fitzpatrick, thank you so much for coming on theCUBE.

Rand Fitzpatrick

>> Thank you.

Rebecca Knight

>> And thank you for watching. We will be right back with more of our coverage of AWS re:Invent.