CVE-2026-90783: 
Linux Debian vulnerability analysis and mitigation

CVE-2026-90783 is a heap buffer overflow vulnerability in MKVToolNix through version 101.0, caused by an integer wraparound in 32-bit arithmetic within the bundled avilib library's ODML superindex parser. An attacker can craft a malicious AVI file with oversized entry counts that trigger an undersized heap allocation, leading to a heap buffer overflow when the file is processed by mkvmerge. The vulnerability was published on September 13, 2026, and a patch commit is available. It carries a CVSS v3.1 score of 7.8 (High) and a CVSS v4.0 score of 8.5 (High) (GitHub Advisory).

The root cause is classified as CWE-680 (Integer Overflow to Buffer Overflow): the ODML superindex parser in avilib.c (lines 2552–2570 of the bundled avilib-0.6.10) performs 32-bit arithmetic to calculate the size of a heap allocation based on entry counts supplied in the AVI file. When an attacker provides an oversized entry count, the multiplication wraps around, producing a value smaller than required, resulting in an undersized allocation. Subsequent writes into this buffer overflow the heap. Exploitation requires user interaction — specifically, a victim must open or process the malicious AVI file with mkvmerge — but no special privileges are needed (GitHub Advisory, VulnCheck Advisory).

Successful exploitation allows a local attacker to achieve arbitrary code execution with the privileges of the process running mkvmerge, with high impact to confidentiality, integrity, and availability of the affected system. An attacker could read sensitive files accessible to the process, modify data, or crash the application. The scope is limited to the vulnerable system itself, with no direct subsequent system impact (GitHub Advisory, VulnCheck Advisory).

As of the disclosure date, there is no public proof-of-concept exploit and no evidence of in-the-wild exploitation (GitHub Advisory). The EPSS score is 0.0, indicating a very low probability of exploitation in the near term. The vulnerability is not listed in the CISA Known Exploited Vulnerabilities (KEV) catalog. No threat actor attribution has been reported.

1. Craft a malicious AVI file: Create an AVI file with an ODML superindex chunk containing an oversized entry count value that, when multiplied in 32-bit arithmetic within avilib.c (lines 2552–2570), wraps around to produce a small allocation size.
2. Trigger undersized heap allocation: When mkvmerge parses the ODML superindex, the integer wraparound causes malloc() (or equivalent) to allocate a buffer smaller than the number of entries to be written.
3. Overflow the heap buffer: As mkvmerge reads the superindex entries from the file into the undersized buffer, data is written beyond the allocated region, corrupting adjacent heap metadata or objects.
4. Achieve code execution: By carefully controlling heap layout (heap grooming), an attacker can overwrite function pointers, vtable entries, or other control-flow data to redirect execution to attacker-controlled code, running with the privileges of the mkvmerge process.
5. Deliver the file: Distribute the malicious AVI file via social engineering, a shared folder, or an automated media processing pipeline to ensure a victim or service invokes mkvmerge on it (GitHub Advisory, VulnCheck Advisory).

• File System: Presence of unexpected or suspicious .avi files in directories processed by mkvmerge; newly created files or scripts in directories writable by the mkvmerge process after AVI file processing.
• Process: mkvmerge spawning unexpected child processes (e.g., shells, network utilities like curl, wget, or nc); abnormal termination or crash of mkvmerge with heap corruption signals (e.g., SIGSEGV, SIGABRT).
• Logs: Application crash logs or core dumps referencing avilib.c or ODML superindex parsing; system logs showing mkvmerge executing with unusual arguments or from unexpected parent processes.
• Network: Outbound connections initiated by the mkvmerge process to unknown external IP addresses, which may indicate a reverse shell payload was executed.

Update MKVToolNix to a version newer than 101.0 that includes the fix introduced in commit 1495126138e086080f0163bee27fafbdf956a1d0 (GitHub Advisory, Codeberg Commit). Until patched, avoid processing untrusted or unknown AVI files with mkvmerge. If mkvmerge is used in automated workflows, restrict its access to only trusted input sources and consider sandboxing the process to limit the impact of potential exploitation.