SageMaker AI introduces usage-based billing for bidirectional streaming inference

AWS has introduced usage-based billing for Amazon SageMaker AI bidirectional streaming inference, allowing model providers to report actual usage in real time streams to AWS Marketplace through a metadata connection that accompanies the WebSocket. Deepgram is the launch partner, bringing usage-based speech-to-text (STT) and text-to-speech (TTS) services to AWS Marketplace. The metadata stream is limited to one message per second per connection, with a maximum of 512 bytes per message; if the metadata stream fails to establish or disconnects midway, inference continues by default because X-Amzn-SageMaker-Metadata-Stream-Required defaults to false. No client configuration is required. SageMaker AI endpoint infrastructure costs are still billed separately.

AI overview based on source reports · Updated 3 days ago