CVE-2026-85670tokenizers BpeBuilder Buffer Overflow via merge token

tokenizers (Hugging Face) is affected by an out-of-bounds buffer access in BpeBuilder::build (tokenizers/src/models/bpe/model.rs). When loading a tokenizer.json via Tokenizer::from_file/from_str, the builder sizes a scratch buffer to the longest vocabulary key, then writes each concatenated merge rule into it. A merge whose concatenated token exceeds the longest vocabulary key overruns the buffer, which Rust turns into a panic that aborts the process in Rust and FFI embeddings. This occurs at load time with no encoding required, so an attacker who supplies a crafted tokenizer.json can cause a denial of service. A secondary defect at the same location can cause a usize underflow (panic in debug, potential memory corruption in release) when continuing_subword_prefix is set and a merge token is shorter than the prefix. Observed in version 0.23.1.

CVSS

6.5 MEDIUM (v3.1, CNA)CVSS:3.1/AV:N/AC:L/PR:N/UI:R/S:U/C:N/I:N/A:H

Weakness (CWE)

CWE-787

Known exploited

Not in CISA's KEV catalog

EPSS (exploitation in the next 30 days)

Not loaded for this CVE

Assigned by

VulnCheck

NVD analysis

Awaiting Analysis

Record last updated

Sep 4, 2026

First seen by InfoSec Signals

Oct 9, 2026

The CVE record on cve.org · NVD's entry

The CVE record is authoritative. Products marked “from the name” were identified from the CNA's free-text names, not a CPE, and are matched less certainly. This product uses the NVD API but is not endorsed or certified by the NVD.