Usage-based metering for bidirectional streaming solves a software billing problem for real-time speech AI. Customers deploying streaming models on Amazon SageMaker AI bidirectional streaming pay for the endpoint infrastructure that hosts the model.
What changed
Separately, model providers listing on AWS Marketplace need a way to charge for what customers consume during a live stream. Today, we’re announcing usage-based metering for Amazon SageMaker bidirectional (bidi) streaming inference, so model providers can report actual usage for the AWS Marketplace software charge. Amazon SageMaker AI endpoint infrastructure charges still apply.
Price and bundle value
Deepgram , the real-time speech AI company, is the launch partner for this capability, bringing usage-based Speech-to-Text (STT) and Text-to-Speech (TTS) billing to AWS Marketplace. This post builds on a series of innovations between Deepgram and AWS. First, we introduced bidirectional streaming for real-time inference on Amazon SageMaker AI , which supports long-lived bidirectional streams between clients and model servers. Then, Deepgram enhanced Amazon SageMaker AI support with AWS Identity and Access Management (AWS IAM) temporary delegation , simplifying secure authentication for enterprise deployments, and expanded observability with enhanced metrics .
AI features that change the value
Bidirectional streaming is different. A client streams audio and receives transcriptions through the SageMaker AI API using an HTTP/2 event stream. SageMaker AI connects to the model server over WebSockets. A session can last 30 seconds or 30 minutes. Usage accumulates as the session runs, and the final total isn’t known until it ends. A response header alone can’t report periodic usage during that live session.
Who should pay attention
Hourly software pricing can charge customers while a model is available, including quiet periods. With usage-based metering, providers can align the AWS Marketplace software charge with reported usage. SageMaker AI endpoint infrastructure charges remain separate and continue to apply while the endpoint is provisioned.
Patrick Tech Media take
The solution introduces a second WebSocket connection alongside the existing inference stream. When a bidirectional streaming session starts, SageMaker AI establishes two connections to the model server:.
Latest comments
0No comments yet. You can start the conversation.