Sign in
Pending benchmarksTracking since Nov 3, 2026

Is WorldSonus state of the art?Pending benchmarks.Announced Oct 6, 2026; no independent results yet.

Noiz AI and HKUST's streaming video-to-audio model for world models and video: it generates 48 kHz stereo audio in 100 ms chunks from past and current frames, with optional text instructions that can change mid-stream. Open weights for non-commercial use (CC BY-NC 4.0).

Announced facts

The details so far

Lab
Noiz AI
Announced
Oct 6, 2026
API
Not yet · None announced
Open weights
Published · Hugging Face FF2416/WorldSonus (linked from NoizAI's GitHub README), not gated; inference code at NoizAI/WorldSonus on GitHub
Type
Causal autoregressive diffusion (rectified-flow) video-to-audio model; each chunk sees only past and current frames
Output
Stereo 48 kHz audio in 100 ms chunks, as a WAV file or streamed raw PCM
Control
Optional text prompt; prompts can change at chunk boundaries during a stream
Components
Frozen DINOv3 visual encoder, T5Gemma 2 text encoder and a causal stereo audio VAE; DINOv3 and T5Gemma 2 keep their upstream licenses
Training data
1,465 hours of audio: 999 hours of paired stereo video-audio and 466 hours of audio only
License
CC BY-NC 4.0 (non-commercial use with attribution)
Paper
WorldSonus: Bringing Sound to Worlds (arXiv 2610.08760), Noiz AI with HKUST, MetaX and Shanghai Jiao Tong University
Requirements
Python 3.10+, a CUDA GPU, FFmpeg and ffprobe
Parameters
Not stated

Tags marked unsure are not stated on the lab's own pages.

Next steps

What would let us call it

Verdicts use only independent boards we already track. Two consecutive snapshots are required before any crown.

  • Publicly announced with a source
  • Usable by the public (API or weights)
  • Listed on an independent board we track
  • Enough results for a rank in a category
  • Holds across two snapshots (crowns only)
Self-reported

What the lab claims

Self-reported. Shown for context; not used in any verdict.

Lab claim: 41.2 ms of compute per 100 ms audio chunk (real-time factor 0.41) on one NVIDIA H100.

Lab claim: VGGish FAD of 1.73 on 5-second stereo VGGSound clips and 2.03 on 30-second interactive videos, matching or exceeding offline bidirectional video-to-audio baselines on acoustic quality and stereo balance.

Receipts

Sources

PAPERWorldSonus: Bringing Sound to Worlds (arXiv)arxiv.org/abs/2610.08760 · Oct 6, 2026LABWorldSonus project pagenoizai.github.io/WorldSonus/REPONoizAI/WorldSonus on GitHubgithub.com/NoizAI/WorldSonusREPOFF2416/WorldSonus on Hugging Facehuggingface.co/FF2416/WorldSonus

Tracking continues until an independent board lists it.