angee.messaging_integrate_imap.parser

Pure MIME parsing — raw RFC 822 bytes into the neutral messaging parse shapes.

The stdlib email package with policy.default owns the wire format (RFC 2047 header decoding, RFC 2231 filenames, charset handling, the multipart walk); this module only maps its EmailMessage onto :class:~angee.messaging.backends .ParsedMessage. The mapping rules that matter downstream:

• external_id is the RFC 5322 Message-ID; a message without one gets a stable sha256: digest of its raw bytes, so ID-less mail never collapses onto one row and never duplicates across re-syncs.
• text/plain bodies split into per-paragraph body/quoted/signature parts. The mail-parser-reply library owns the segmentation — reply boundaries at attribution headers ("On …, X wrote:", Outlook From:-blocks), signatures with or without the RFC 3676 -- delimiter, trailing disclaimers — while quote markers are stripped locally, so a reply's quoted paragraphs content-address to the same Fragment rows as the original body and the quotation graph links them.
• an HTML-only message derives a plain body (so previews and search see text) and keeps the original HTML verbatim under a multipart/alternative container.
• attachments keep disposition and Content-ID, so inline images stay inline + cid and real attachments stay attachment.
• a message the MIME parser cannot handle still lands: :func:fallback_message wraps the raw bytes as an attachment behind a best-effort envelope, so no mail is ever dropped and no retry ledger is needed.

Everything here is pure — no network, no database — so the whole matrix is unit testable from literal byte strings. Null bytes pass through untouched; the messaging managers own that scrub on the write path.

parse_message

def parse_message(raw: bytes,
                  *,
                  mailbox: str,
                  uid: int,
                  uidvalidity: int,
                  flags: tuple[Any, ...] = (),
                  internal_date: datetime | None = None,
                  own_addresses: frozenset[str] = frozenset(),
                  truncated_bytes: int | None = None) -> ParsedMessage

Parse one raw RFC 822 message into a :class:ParsedMessage.

internal_date is the server receipt time (the IMAP INTERNALDATE) and the fallback when the Date header is missing or malformed. own_addresses classifies direction (an owned From is outbound; owned From and only owned recipients is internal). truncated_bytes marks a header-only fetch of an oversized message: the size lands in metadata and the body stays empty rather than the message being dropped.

Raises whatever the MIME layer raises on hopeless input — the caller wraps with :func:fallback_message so a poison message still lands.

fallback_message

def fallback_message(raw: bytes,
                     *,
                     mailbox: str,
                     uid: int,
                     uidvalidity: int,
                     flags: tuple[Any, ...] = (),
                     internal_date: datetime | None = None,
                     error: Exception | None = None) -> ParsedMessage

Wrap an unparseable raw message so it still lands instead of being lost.

The lenient compat32 policy recovers what headers it can for the envelope; the raw bytes ride as a message/rfc822 attachment so nothing is discarded and the message can be re-parsed once the defect is understood.

synthetic_external_id

def synthetic_external_id(raw: bytes) -> str

Return a stable dedup key for a message that carries no Message-ID.

Hashing the raw bytes keys the same message identically in every folder and every re-sync, while two distinct ID-less messages never collapse onto one row.

split_plain_text

def split_plain_text(text: str) -> list[tuple[str, str]]

Split plain text into ordered (role, paragraph) segments.

mail-parser-reply owns the segmentation: it splits the text into replies at the attribution headers it recognizes and detects each reply's signature (the RFC 3676 -- delimiter or a salutation tail like "Best regards,") and trailing corporate disclaimers. Roles are the messaging part vocabulary: body paragraphs from the newest reply, quoted paragraphs for older replies and marker-quoted runs (attribution line included, markers stripped to any depth, so the text content-addresses to the original body's fragments), and signature for signatures and disclaimers alike. Paragraphs are blank-line separated; document order is preserved so the first body paragraph stays the preview.

html_to_text

def html_to_text(html: str) -> str

Extract readable text from an HTML body (script/style dropped).

A deliberately small stdlib extractor for previews, search, and fragment dedup — rendering fidelity stays with the stored HTML part.