The next evolution of enterprise software is here—and it’s agentic.
While only 1% of SaaS applications today are powered by agentic AI, that’s expected to change rapidly. Agentic AI, a fast-rising field drawing intense interest and search traffic, is reshaping how enterprise software operates. Often called the “fifth wave of compute,” this technology is positioned to fundamentally alter how SaaS apps function, automate tasks, and interface with users.
Unlike AI assistants that respond to commands, agentic AI goes further by autonomously pursuing goals and interacting with other systems. Where robotic process automation (RPA) follows rigid scripts, agentic systems are dynamic, adaptive, and goal-driven.
Gartner predicts that within four years, a third of all SaaS applications will be integrated with agentic services, forming a new “app/AI ecosystem.” That shift represents a dramatic transformation in how enterprise software is designed, deployed, and experienced.
Scaling Beyond Today’s Limits
The agentic shift will push current system architectures to their limits. Traditional SaaS platforms operate at roughly 10,000 transactions per second (TPS). But in the agentic future—where each user may be aided by multiple AI agents—that could skyrocket to 1 million TPS or more.
This surge in demand requires moving from stateless, transactional models to stateful, conversational architectures. Unlike classic CRUD operations backed by relational databases, agentic systems maintain internal state and track every interaction to inform future actions.
One technical hurdle is that large language models (LLMs), which power many of these agents, are inherently stateless. To simulate memory, developers must maintain detailed “conversation journals” and send them with every request—until token limits are reached. Managing this context efficiently becomes critical as systems scale.
From Text to Multimodal Intelligence
Agentic systems are expanding beyond simple chatbots. They’re integrating with IoT sensors, video streams, and other real-time data sources. These multimodal inputs need to be processed and routed efficiently to avoid overwhelming slower LLM components. A delicate balance must be struck between real-time responsiveness and the heavy compute demands of deeper AI processing.
The Performance-Cost Tradeoff
Agentic applications are vastly different from traditional software in terms of performance and cost. While database queries return results in milliseconds, LLM-based transactions can be 100 times slower—and up to 850,000 times more expensive per request.
That said, LLM costs have dropped by around 90% annually for the past three years. If the trend continues, it may become more feasible to use agentic services broadly. Still, high-performance use cases—like real-time transactions—will continue to rely on traditional SaaS infrastructures.
Why Production Remains Hard
Despite the hype, many organizations struggle to implement agentic AI. According to Gartner, 52% of projects fail to get out of the lab. Common issues include poor data quality, high costs, unsuitable tools, and a lack of readiness for 24/7, concurrent environments.
Organizations are experimenting with various deployment strategies, from cloud-native platforms (62%) to self-hosted Kubernetes clusters (around 50%). Many pursue hybrid strategies to balance flexibility, control, and compliance.
Augmentation, Not Replacement
Agentic AI won’t replace SaaS systems—it will augment them. Because of their latency and compute needs, agents are ill-suited for high-frequency, low-latency interactions. Instead, SaaS platforms will continue to provide scalable interfaces, while agentic systems layer on intelligence and automation.
Interfaces will become increasingly multimodal, blending voice, video, text, and sensor data. Depending on the use case, some applications may become 70–80% agentic in function.
Organizations evaluating agentic AI must factor in costs across LLM hosting, compute, memory, vector databases, and agent orchestration platforms. A major ongoing challenge will be explainability—tracing why a model made a decision as these systems grow more complex.
Akka’s Blueprint for Agentic Services
To accelerate adoption, platform provider Akka has developed a reference architecture for building scalable, production-ready agentic systems. Their blueprint includes:
- Streaming endpoints for real-time multimodal data.
- Connectivity adapters to integrate LLMs, vector databases, and third-party systems.
- Agent orchestration for managing workflows, human-in-the-loop inputs, and concurrency.
- Context databases that combine in-memory and durable storage for conversation history.
- Lifecycle management to ensure governance, security, and observability.
Akka treats LLMs as event-driven rather than batch-based systems. Their nonblocking adapters with back-pressure support allow efficient streaming of responses from LLMs into other workflows or external endpoints.
A standout feature is their unified compute approach—standard APIs and agentic workflows run on the same infrastructure, reducing inefficiencies and integration challenges.
Akka has already powered real-world successes:
- SMILE, an open-source ML project used by companies like Amazon and Google
- Swiggy, which cut latency by 90%
- Horn, an AI video call assistant
- Coho AI, which reportedly reached market 75% faster
Looking ahead, Akka aims to bridge LLMs and enterprise data through rich metadata, allowing agents not only to interpret queries but to act on business systems directly.
The Road Ahead
Agentic AI won’t be a silver bullet, but it’s a clear next step in enterprise tech evolution. As businesses experiment and iterate, the winners will be those who effectively manage performance, cost, and user experience while scaling to meet this new computational paradigm.
