iCreatAI News
Skip to content

Technology areas

Deployment engineering Latest updates

Engineering practices for getting models running: inference optimization, VRAM and cost, serving architecture, and infrastructure selection.

85 条精选Past 30 days: 82 updatesTotal: 1,177 updates

Updated

精选归档 · 第 5 页

Oct 20Tue 81–85 updates
  1. Mistral AI80

    Voxtral 3

    Mistral introduces Voxtral 3, a family of speech-to-text and text-to-speech models plus a LiveKit-based voice agent SDK that combines listening, reasoning, tool calling and speaking in one real-time streaming pipeline. It includes Voxtral Small Transcribe 3 for multilingual transcription with diarization and context biasing, Voxtral Mini TTS 3 for expressive speech in 35+ languages and 50+ dialects, and Pro Voice adaptation from 30 minutes of audio. It is available via Mistral's EU-hosted API or self-deployment, with pricing from $0.0035/min for batch transcription to $13 per 1M characters for TTS.

    Why it matters: The announcement details concrete model capabilities, benchmarks and pricing for teams evaluating real-time voice agents and sovereign speech deployment.

  2. Hugging Face Blog65

    Hugging Face ❤️ TorchTPU: TPUs, Natively

    TorchTPU adds TPU as a native PyTorch backend, letting Transformers, Diffusers and Kernels run on TPU by sending models to "tpu" instead of going through torch_xla or JAX, and Optimum TPU is no longer required. The post shows loading a model with device_map="tpu", running Diffusers pipelines with pipe.to("tpu") plus optional torch.compile, and using Pallas kernels from the Kernels Hub, with reported speedups of 21.5× on Qwen3-30B-A3B and 12.8× on Gemma 4 26B-A4B at batch size 64. It also reports that 96.5% of tests run on TPU pass, with 34 issues filed on the TorchTPU repo and 19 already fixed.

    Why it matters: It documents a new native PyTorch TPU backend and concrete usage patterns for the Hugging Face stack, useful for readers weighing TPU support.

Oct 15Thu
  1. Mistral AI67

    Introducing Timestral

    Mistral releases Timestral, its first family of time-series foundation models in four sizes from 5M to 209M parameters, with the largest ranking first on GIFT-Eval, TIME and fev-bench among pre-trained zero-shot models. Timestral produces probabilistic multivariate forecasts, supports past-only and known-future covariates, and runs on CPU without a GPU. It combines synthetic-first pre-training, covariate-aware attention and decoupled horizon forecasting, and can be fine-tuned on proprietary data through Mistral Forge.

    Why it matters: Useful for teams weighing time-series foundation models, since it reports benchmark standings, a size-versus-accuracy trade-off and CPU latency figures alongside the adaptation options.

Oct 8Thu
  1. Microsoft Research71

    Agent Lightning v1.0: A 3,500-Line Lightweight Agentic RL Framework for Training Agents with Real Harnesses

    Microsoft Research Asia introduces Harnessed Agentic RL, a paradigm in which the same agent harness used in deployment participates directly in reinforcement learning, and open-sources Agent Lightning v1.0, a roughly 3,500-line framework with an API Gateway, Rollout Controller, and Customized Trainer built on verl. Agents run as standard Kubernetes jobs without commercial sandbox services, and Collocated Async RL shares GPUs between rollout and updates, achieving about a 2x end-to-end speedup over synchronous RL. An end-to-end coding agent pipeline raised Qwen3.5-9B from 41.8% to 56.4% Pass@1 on SWE-bench Verified using about 6,000 training samples.

    Why it matters: It offers a concrete, reproducible recipe for training deployed agents in place, with code size, infrastructure, and benchmark numbers stated explicitly.

Jul 29Wed
  1. Berkeley AI Research60

    From CUDA to MLX: How K-Search Brings Decades of Kernel Expertise to Apple Silicon

    IBM Research extended K-Search, an evolutionary LLM-driven kernel search framework from UC Berkeley Sky Lab, with a native MLX backend and a structured CUDA-to-MLX translation layer that maps CUDA primitives, constraints and optimization patterns onto Apple Silicon equivalents. On Apple Silicon the approach reached 0.97x the speed of the native MLX Attention kernel and up to a 20x prefill speedup over the community mlx-lm implementation on the Mamba SSM kernel, attributed largely to adding a parallel scan. The authors say the bottleneck was not the LLM's ability to write Metal code but the quality of context and constraints supplied to it.

    Why it matters: Describes a concrete method for transferring CUDA kernel expertise to non-NVIDIA hardware, with reported benchmark numbers and an open-source reproduction path.