iCreatAI News
Back to main site
Skip to content

Technology areas

Deployment engineering Latest updates

Engineering practices for getting models running: inference optimization, VRAM and cost, serving architecture, and infrastructure selection.

73 条精选Past 30 days: 70 updatesTotal: 989 updates

Updated

精选归档 · 第 4 页

Oct 24Sat 61–73 updates
  1. Google Research62

    Bringing foundational models to the front lines of TB screening

    Google Research and Google for Health describe their path from a 2019 Radiology benchmark showing deep learning could match radiologists in detecting active pulmonary TB to prospective multi-site validation, CE MDR Class IIb certification via Nexus AI, and deployment in about 40 facilities across six countries, where over 90 clinicians have evaluated more than 25,000 chest radiographs. They also detail offline AI screening four kilometers underground at South Africa's Mponeng Gold Mine and open models (HAI-DEF, MedSigLIP, MedGemma, HeAR), including Indonesia's Ministry of Health fine-tuning MedGemma and MedSigLIP on domestic chest radiographs for a national TB program.

    Why it matters: It offers a concrete case study of moving medical AI from benchmark to certified, field-deployed tools, with named partners, regulatory milestones and deployment numbers.

Oct 22Thu
  1. NVIDIA Blog60

    NVIDIA Unveils Minerva to Bring End-to-End Observability for AI Factories

    NVIDIA launched Minerva, an AIOps platform that combines telemetry from infrastructure and AI workloads and applies purpose-built AI models to surface timely, actionable insight. It builds a dynamic knowledge graph mapping GPUs, SuperNICs, switches, cables and the jobs running on them, analyzes billions of data points per hour to detect anomalies and predict failures, and includes a native AI assistant plus connections to external agentic AI tools. IREN is among the partners using Minerva, and Aolani, Bitdeer AI, IBM, Jio, Lambda, Quake AI and Tesla plan to deploy it, with Dell Technologies, Lenovo and Supermicro offering it in their AI factory operations portfolios.

    Why it matters: It details a new NVIDIA observability platform and its named deployment partners, useful for readers tracking AI data center operations tooling.

  2. The Decoder78

    An AI agent in "Skynet mode" hijacks unprotected language models to launch self-sustaining cyberattacks

    Symantec researchers intercepted a full attack prompt at a honeypot showing an offensive AI agent that hijacks unprotected language models, then scans for targets, exploits server software, steals databases and credentials, and installs cryptominers for up to 100,000 steps, leaving the model's owner with the compute bill. The prompt used a five-layer jailbreak, including a fake "Fantasy AI" identity and a companion persona called "ENI," plus 74 attack modules, and a second captured attack delivered an obfuscated script that downloads the XMRig Monero miner targeting go2rtc camera servers. Symantec calls the approach a shift in how attackers use AI, since the model does the planning while specialized modules carry out the attack steps.

    Why it matters: It documents a concrete, intercepted offensive AI agent and its jailbreak structure, offering a useful reference point for anyone securing exposed AI endpoints.

  3. Hugging Face Blog75

    NVIDIA Kumo Relational Outperforms Supervised Machine Learning on Relational Data, No Training Required

    NVIDIA introduces Kumo Relational, a pre-trained relational foundation model that predicts directly on enterprise databases via in-context learning in a single forward pass, requiring no task-specific training, feature engineering, or table flattening. Across 41 tasks on 15 databases, it beats supervised relational deep learning models and expert-engineered gradient-boosted tree pipelines, and on the SAP SALT benchmark it surpasses AutoGluon ensembles by 8% in mean reciprocal rank (MRR). It scales to databases with 500B+ rows and is available through NVIDIA's open-source structured-data-models and kumo-engine libraries or as an NVIDIA NIM.

    Why it matters: Useful for ML teams weighing whether to replace production XGBoost pipelines with a pre-trained relational model, though the reported gains come from NVIDIA's own paper and benchmarks.

Oct 21Wed
  1. NVIDIA Blog60

    Bringing the Spark to Europe: Local Intelligence Powers Next Wave of Agents

    At GTC Berlin, NVIDIA showcased its local AI hardware lineup: RTX Spark for personal agents on Windows PCs, DGX Spark for Linux developers, and DGX Station for enterprise workloads, with a Windows version due in December. Announcements included Underdog Desktop Agent for RTX Spark, DGX Spark 64GB systems rolling out Oct. 23, support for clusters of up to eight DGX Sparks, NVIDIA Sync Model Launcher, and the new Recursive Flint local coding model. NVIDIA also added day-zero support for new Nemotron, GLM, DeepSeek, Qwen, MiniMax and FLUX.3 models, plus accelerations for llama.cpp and vLLM.

    Why it matters: It offers a detailed roundup of NVIDIA's local AI hardware and software announcements for developers and enterprises.

  2. NVIDIA Blog72

    Google Cloud and NVIDIA Announce General Availability of NVIDIA Vera Rubin Infrastructure

    Google Cloud and NVIDIA announced general availability of NVIDIA Vera Rubin-based A5X instances next month, with planned expansion to Europe, combining Vera Rubin NVL72, ConnectX-9 SuperNICs, Google Cloud Virgo Network and fourth-generation liquid cooling for training, inference and large-scale reinforcement learning. Google Cloud plans to scale the infrastructure to as many as 960,000 Rubin GPUs across multiple sites, and Ineffable Intelligence, founded by AlphaGo creator David Silver, was the first customer to access A5X. Google Cloud also said it will be among the first cloud providers to offer NVIDIA Confidential Computing at rack scale on bare metal instances such as A4X Max powered by GB300 NVL72, and is making Confidential Computing generally available on G4 VMs powered by RTX PRO 6000 Blackwell Server Edition GPUs.

    Why it matters: The announcement details Google Cloud's Vera Rubin rollout plans, early customer use cases and confidential computing offerings, useful for readers tracking AI infrastructure availability.

  3. NVIDIA Blog62

    GTC Berlin 2026: Live Updates From NVIDIA CEO Jensen Huang’s Keynote

    At GTC Berlin, Jensen Huang said European companies including L'Oréal, Revolut, Spotify, Merck, Vodafone, Zara, Porsche, Nestlé and HSBC are working with NVIDIA on AI, following a weeklong European road trip in which the company committed nearly $10 billion to the region's technology ecosystem and worked with 12,000+ partners. He announced the NVIDIA DSX POD reference design for Vera Rubin AI factories in modular containers, claiming onsite deployment can shrink from six months to 14 days, along with the OpenShell Reference Architecture for safe AI agents, the Agent Knowledge Foundation, and Agile ONE SN, a 31-degree-of-freedom reference humanoid built on Isaac GR00T expected next year. He also described RTX Spark laptops with 1 petaflop of performance and 128GB of memory, and said Volkswagen, JLR, Volvo and Mercedes-Benz next-generation vehicles will be Hyperion-compatible.

    Why it matters: The piece is a detailed first-hand account of NVIDIA's European AI strategy and product announcements, useful for readers tracking agent infrastructure, AI factories and physical AI.

  4. Latent Space83

    [AINews] DeepSeek DeepInfer: Open Source SOTA Inference running 155T toks/day/cluster making $22m/day at ~80% margin for a 81 day payback

    DeepSeek released DeepInfer, its production inference stack built on vLLM with speculative decoding, SWA Bounded Replay and structured output, reporting 155 trillion tokens/day from one cluster, 96% input-cache hits, 400 ms average time to first token and $38 revenue per accelerator-hour at off-peak prices. The release excludes KV-cache management and load balancing, which remain coupled to internal infrastructure, and vLLM reported decode throughput within 6% of DeepInfer's published results on GB200 under a matched configuration. Community margin estimates of roughly 80% depend on a cache-heavy token mix and omit costs such as SSD cache, networking and idle capacity, so the figures are first-party production claims rather than audited measurements.

    Why it matters: Useful for readers tracking open-source inference economics, since it pairs DeepSeek's production claims with vLLM's comparative benchmark and a careful accounting of what the margin figures do and do not include.

Oct 20Tue
  1. Mistral AI80

    Voxtral 3

    Mistral introduces Voxtral 3, a family of speech-to-text and text-to-speech models plus a LiveKit-based voice agent SDK that combines listening, reasoning, tool calling and speaking in one real-time streaming pipeline. It includes Voxtral Small Transcribe 3 for multilingual transcription with diarization and context biasing, Voxtral Mini TTS 3 for expressive speech in 35+ languages and 50+ dialects, and Pro Voice adaptation from 30 minutes of audio. It is available via Mistral's EU-hosted API or self-deployment, with pricing from $0.0035/min for batch transcription to $13 per 1M characters for TTS.

    Why it matters: The announcement details concrete model capabilities, benchmarks and pricing for teams evaluating real-time voice agents and sovereign speech deployment.

  2. Hugging Face Blog65

    Hugging Face ❤️ TorchTPU: TPUs, Natively

    TorchTPU adds TPU as a native PyTorch backend, letting Transformers, Diffusers and Kernels run on TPU by sending models to "tpu" instead of going through torch_xla or JAX, and Optimum TPU is no longer required. The post shows loading a model with device_map="tpu", running Diffusers pipelines with pipe.to("tpu") plus optional torch.compile, and using Pallas kernels from the Kernels Hub, with reported speedups of 21.5× on Qwen3-30B-A3B and 12.8× on Gemma 4 26B-A4B at batch size 64. It also reports that 96.5% of tests run on TPU pass, with 34 issues filed on the TorchTPU repo and 19 already fixed.

    Why it matters: It documents a new native PyTorch TPU backend and concrete usage patterns for the Hugging Face stack, useful for readers weighing TPU support.

Oct 15Thu
  1. Mistral AI67

    Introducing Timestral

    Mistral releases Timestral, its first family of time-series foundation models in four sizes from 5M to 209M parameters, with the largest ranking first on GIFT-Eval, TIME and fev-bench among pre-trained zero-shot models. Timestral produces probabilistic multivariate forecasts, supports past-only and known-future covariates, and runs on CPU without a GPU. It combines synthetic-first pre-training, covariate-aware attention and decoupled horizon forecasting, and can be fine-tuned on proprietary data through Mistral Forge.

    Why it matters: Useful for teams weighing time-series foundation models, since it reports benchmark standings, a size-versus-accuracy trade-off and CPU latency figures alongside the adaptation options.

Oct 8Thu
  1. Microsoft Research71

    Agent Lightning v1.0: A 3,500-Line Lightweight Agentic RL Framework for Training Agents with Real Harnesses

    Microsoft Research Asia introduces Harnessed Agentic RL, a paradigm in which the same agent harness used in deployment participates directly in reinforcement learning, and open-sources Agent Lightning v1.0, a roughly 3,500-line framework with an API Gateway, Rollout Controller, and Customized Trainer built on verl. Agents run as standard Kubernetes jobs without commercial sandbox services, and Collocated Async RL shares GPUs between rollout and updates, achieving about a 2x end-to-end speedup over synchronous RL. An end-to-end coding agent pipeline raised Qwen3.5-9B from 41.8% to 56.4% Pass@1 on SWE-bench Verified using about 6,000 training samples.

    Why it matters: It offers a concrete, reproducible recipe for training deployed agents in place, with code size, infrastructure, and benchmark numbers stated explicitly.

Jul 29Wed
  1. Berkeley AI Research60

    From CUDA to MLX: How K-Search Brings Decades of Kernel Expertise to Apple Silicon

    IBM Research extended K-Search, an evolutionary LLM-driven kernel search framework from UC Berkeley Sky Lab, with a native MLX backend and a structured CUDA-to-MLX translation layer that maps CUDA primitives, constraints and optimization patterns onto Apple Silicon equivalents. On Apple Silicon the approach reached 0.97x the speed of the native MLX Attention kernel and up to a 20x prefill speedup over the community mlx-lm implementation on the Mamba SSM kernel, attributed largely to adding a parallel scan. The authors say the bottleneck was not the LLM's ability to write Metal code but the quality of context and constraints supplied to it.

    Why it matters: Describes a concrete method for transferring CUDA kernel expertise to non-NVIDIA hardware, with reported benchmark numbers and an open-source reproduction path.