CVE-2020-36120: 
NixOS vulnerability analysis and mitigation

Buffer Overflow vulnerability (CVE-2020-36120) was identified in the 'sixel_encoder_encode_bytes' function of Libsixel version 1.8.6. The vulnerability was discovered and reported on December 30, 2020 (GitHub Issue). This security flaw affects systems running Libsixel v1.8.6, particularly on platforms like Ubuntu 20.04 LTS (NVD).

The vulnerability is a stack-buffer-overflow that occurs in the 'sixel_encoder_encode_bytes' function at encoder.c:1788. The issue manifests during the processing of pixel data, specifically in the get_rgb function within pixelformat.c:50:18. The vulnerability can be triggered when processing certain malformed input data through the sixel_encoder_encode_bytes function (GitHub Issue).

When exploited, this vulnerability allows attackers to cause a Denial of Service (DoS) condition in applications using the affected Libsixel library. The buffer overflow can lead to application crashes and potential system instability (NVD).

A proof-of-concept exploit has been documented that demonstrates the vulnerability. The issue can be triggered through specially crafted input data passed to the sixel_encoder_encode_bytes function. The vulnerability has been confirmed through testing with Address Sanitizer (ASAN) which detected the stack-buffer-overflow condition (GitHub Issue).

The vulnerability affects Debian packages including bullseye (1.8.6-2), bookworm (1.10.3-3), and sid/trixie (1.10.5-1). While the issue is marked as 'vulnerable' in these versions, it is considered a minor issue that requires API modifications for a complete fix (Debian Tracker).