Alex Gallego & Tyler Akidau, Redpanda
In this interview from the theCUBE + NYSE Wired: Mixture of Experts series, James White, CTO at CalypsoAI, joins theCUBE’s John Furrier to unpack CalypsoAI’s newly launched Security Index – the first comprehensive safety ranking of major generative AI models. White explains how the weekly updated leaderboard and the CASI (CalypsoAI Security Index) score enable apples-to-apples comparisons that blend quality and security, helping enterprises move beyond POC purgatory and toward ROI. The discussion connects model selection and risk posture to enterprise strategy at the intersection of tech and finance – where governance, vendor constraints and performance/latency considerations shape deployment choices at scale. White details CalypsoAI’s Red-Team product and three attack lenses: signature attacks, operational attacks (e.g., overwhelming outputs that mimic denial-of-service) and “agentic warfare,” which uses autonomous agents to probe for jailbreaks and prompt-injection gaps. He breaks down CASI’s inputs across severity, complexity, decay of older tactics (like DAN variants) and defensive breaking points, alongside an average performance column so teams can weigh capability vs. security. Highlights include Anthropic models leading the safety pack (with Microsoft among the leaders), Claude 3.5 scoring 96.25, Claude 3.7 trending into the #2 slot with different security trade-offs, DeepSeek-R1 landing mid-table and GPT-3.5 Turbo dropping from the top 12. White also previews a human-in-the-loop Purple-Team approach, and shares guidance for continuous testing in CI/CD, model family choices across cloud stacks and real-world implications for POCs, benchmarks and production hardening.
Share this session