# Is WorldSonus SOTA?

Pending benchmarks. Announced Oct 6, 2026; no independent results yet.

As of Nov 22, 2026, WorldSonus is pending benchmarks on isitsota: announced Oct 6, 2026, with no independent results on the boards we track yet.

Noiz AI and HKUST's streaming video-to-audio model for world models and video: it generates 48 kHz stereo audio in 100 ms chunks from past and current frames, with optional text instructions that can change mid-stream. Open weights for non-commercial use (CC BY-NC 4.0).

Canonical: https://isitsota.com/m/worldsonus
Last updated: 2026-11-03
Tracking since: 2026-11-03

## Announced facts

| Fact | Value |
| --- | --- |
| Lab | Noiz AI |
| Announced | Oct 6, 2026 |
| API | Not yet · None announced |
| Open weights | Published · Hugging Face FF2416/WorldSonus (linked from NoizAI's GitHub README), not gated; inference code at NoizAI/WorldSonus on GitHub |
| Type | Causal autoregressive diffusion (rectified-flow) video-to-audio model; each chunk sees only past and current frames |
| Output | Stereo 48 kHz audio in 100 ms chunks, as a WAV file or streamed raw PCM |
| Control | Optional text prompt; prompts can change at chunk boundaries during a stream |
| Components | Frozen DINOv3 visual encoder, T5Gemma 2 text encoder and a causal stereo audio VAE; DINOv3 and T5Gemma 2 keep their upstream licenses |
| Training data | 1,465 hours of audio: 999 hours of paired stereo video-audio and 466 hours of audio only |
| License | CC BY-NC 4.0 (non-commercial use with attribution) |
| Paper | WorldSonus: Bringing Sound to Worlds (arXiv 2610.08760), Noiz AI with HKUST, MetaX and Shanghai Jiao Tong University |
| Requirements | Python 3.10+, a CUDA GPU, FFmpeg and ffprobe |
| Parameters | Not stated |

Tags marked unsure are not stated on the lab's own pages.

## What would let us call it

Verdicts use only independent boards we already track. Two consecutive snapshots are required before any crown.

- [x] Publicly announced with a source
- [x] Usable by the public (API or weights)
- [ ] Listed on an independent board we track
- [ ] Enough results for a rank in a category
- [ ] Holds across two snapshots (crowns only)

## What the lab claims

Self-reported. Shown for context; not used in any verdict.

- Lab claim: 41.2 ms of compute per 100 ms audio chunk (real-time factor 0.41) on one NVIDIA H100.
- Lab claim: VGGish FAD of 1.73 on 5-second stereo VGGSound clips and 2.03 on 30-second interactive videos, matching or exceeding offline bidirectional video-to-audio baselines on acoustic quality and stereo balance.

## Sources

- PAPER: [WorldSonus: Bringing Sound to Worlds (arXiv)](https://arxiv.org/abs/2610.08760) (2026-10-06)
- LAB: [WorldSonus project page](https://noizai.github.io/WorldSonus/)
- REPO: [NoizAI/WorldSonus on GitHub](https://github.com/NoizAI/WorldSonus)
- REPO: [FF2416/WorldSonus on Hugging Face](https://huggingface.co/FF2416/WorldSonus)

Tracking continues until an independent board lists it.

## Related

- Lab: [Noiz AI](https://isitsota.com/labs/noiz-ai.md)
- All pending releases: https://isitsota.com/releases.md
- JSON: https://api.isitsota.com/v1/models/worldsonus
