Bangalore – Akamai (NASDAQ: AKAM), the cybersecurity and cloud computing company that powers and protects business online, today unveiled Akamai Cloud Inference, to usher in a faster, more efficient wave of innovation for organizations looking to
turn predictive and Large Language Models (LLMs) into real world action. Akamai Cloud Inference runs on Akamai Cloud, the world’s most distributed network, to address escalating limitations of centralized cloud models.
“Getting AI data closer to users and devices is hard and it’s where legacy clouds struggle,” said Adam Karon, Chief Operating Officer and General Manager, Cloud Technology Group, at Akamai. “While the heavy lifting of training LLMs will continue to happen in big hyperscale datacenters, the actionable work of inferencing will take place at the edge where the network Akamai has built over the past two and a half decades becomes vital for the future of AI and sets us apart from every other cloud provider in the market.”
AI Inference on Akamai Cloud
Akamai’s new solution provides tools for platform engineers and developers to build and run AI applications and data-intensive workloads closer to end-users, delivering 3x better throughput while reducing latency up to 2.5x. Using Akamai’s solution, businesses can save up to 86% on AI inference and agentic AI workloads compared to traditional hyperscaler infrastructure. Akamai Cloud Inference includes:
● Compute: Akamai Cloud offers a versatile compute arsenal, from classic CPUs for fine tuned inference, to powerful accelerated-compute options in GPUs, and tailored ASIC VPUs to provide the right horsepower for a spectrum of AI inference challenges. Akamai integrates with Nvidia’s AI Enterprise ecosystem, leveraging Triton, Tao Toolkit, TensorRT, and NvFlare to optimize performance of AI inference on Nvidia GPUs.
● Data management: Akamai enables customers to unlock the full potential of AI inference with a cutting-edge data fabric purpose-built for modern AI workloads. Akamai has partnered with VAST Data to provide streamlined access to real-time data to accelerate inference-related tasks, essential to delivering relevant results and a responsive experience. This is complemented by highly scalable object storage to manage the volume and variety of datasets critical to AI applications, and integration with leading vector database vendors, including Aiven and Milvus, to enable retrieval augmented generation (RAG). With this data management stack, Akamai securely stores fine-tuned model data and training artifacts to deliver low latency AI inference at global scale.
● Containerization: Containerizing AI workloads enables demand-based autoscaling, improved application resilience, and hybrid/multicloud portability, while optimizing both performance and cost. With Kubernetes, Akamai delivers faster, cheaper, and more secure AI inference at petabyte-scale performance. Underpinned by Linode Kubernetes Engine – Enterprise, a new enterprise edition of Akamai Cloud’s Kubernetes orchestration platform designed specifically for large-scale enterprise workloads, and the recently announced Akamai App Platform, Akamai Cloud Inference is able to quickly deploy an AI-ready platform of open source Kubernetes projects, including Kserve, KubeFlow, and SpinKube, seamlessly integrated to streamline the deployment of AI models for inference.
● Edge compute: To simplify how developers build AI-powered applications, Akamai AI Inference includes WebAssembly (WASM) capabilities. Working with WASM providers like Fermyon, Akamai enables developers to execute inferencing for LLMs directly from serverless apps, enabling customers to execute lightweight code at the edge to enable latency-sensitive applications.
Together, these tools create a powerful platform for low-latency, AI-powered applications that allows companies to deliver the experience their users demand. Akamai Cloud Inference runs on the company’s massively distributed network capable of consistently delivering over one petabyte per second of throughput for data-intensive workloads. Comprising more than 4,100 points of presence across greater than 1,200 networks in over 130 countries worldwide, Akamai
Cloud makes compute resources available from cloud to edge while accelerating application performance and increasing scalability.
The Shift from Training to Inference
As AI adoption matures, enterprises are recognizing that the hype around LLMs has created a distraction, drawing focus away from practical AI solutions better suited to solve specific business problems. LLMs excel at general-purpose tasks like summarization, translation, and customer service. These are very large models that are expensive and time-consuming to train. Many enterprises have found themselves constrained by architectural and cost requirements including datacenter and computational power, well-structured, secure, and scalable data systems, and the challenges that location and security requirements place on decision latency. Lightweight AI models, designed to address specific business problems, can be optimized for individual industries, leveraging proprietary data to create measurable outcomes, and represent a better return on investment for enterprises today.
AI Inference Needs a More Distributed Cloud
Gartner predicts that in 2025, 75% of data will be generated outside of centralized data centers or cloud regions. This shift is driving demand for AI solutions that leverage data generation closer to the point of origin. This fundamentally reshapes infrastructure needs as enterprises move beyond building and training LLMs, toward leveraging data for faster, smarter decisions and investing in more personalized experiences. Enterprises recognize that they can generate more value by leveraging AI to manage and improve their business operations and processes. Distributed cloud and edge architectures are emerging as preferable for operational intelligence use cases because they can provide real-time, actionable insights across distributed assets even in remote environments. Early customer examples on Akamai Cloud include in-car voice assistance, AI-powered crop management, image optimization for consumer product marketplaces, virtual garment visualization shopping experiences, automated product description generators, and customer feedback sentiment analyzers.
“Training a LLM is like creating a map – requiring you to gather data, analyze terrain, and plot routes. It’s slow and resource-intensive, but once built, it’s highly useful. AI inference is like using a GPS, instantly applying that knowledge, recalculating in real time, and adapting to changes to get you where you need to go,” explained Karon. “Inference is the next frontier for AI.”
