fix(2): Q4.9 mailu — rewrite mail-flow via in-container sendmail+doveadm; drop network IMAP-auth test
Root cause of the 2 failing custom tests: TLS_FLAVOR=notls → dovecot refuses plaintext auth over network 143, so host-side IMAP login/auth isn't a meaningful signal. Smoke2 PROVED the in-container path: sendmail (postfix container) local-injects a marker mail → doveadm search (imap container) finds it in INBOX. test_mail_flow now exercises the real postfix→rspamd→dovecot deliver/store/fetch via exec_in_app(service=smtp/imap). Dropped test_imap_login (network plaintext-auth disallowed under notls). test_mailbox (create+config-export read-back) unchanged. PARITY.md updated. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
@ -15,16 +15,18 @@ email stack: nginx front + admin + postfix/smtp + dovecot/imap + rspamd/antispam
|
||||
wildcard TLS via Traefik. Mail ports 25/465/587/110/143/993/995 are published mode:host → on-host
|
||||
(cc-ci-run) tests reach SMTP/IMAP at 127.0.0.1.
|
||||
|
||||
## Recipe-specific functional tests (P3 — ≥2; here 3)
|
||||
## Recipe-specific functional tests (P3 — ≥2)
|
||||
1. `functional/test_mailbox.py` — §4.3 create-an-object + read-back: create a mailbox via the admin
|
||||
container's `flask mailu user` CLI, then read it back from `flask mailu config-export --json` and
|
||||
assert the address is present (admin-DB provisioning round-trip).
|
||||
2. `functional/test_imap_login.py` — distinctive (working mail account): the provisioned mailbox
|
||||
AUTHENTICATES against dovecot over IMAP (127.0.0.1:143) and SELECTs its INBOX — proving the
|
||||
account is real end-to-end (admin → dovecot auth), not just a DB row.
|
||||
3. `functional/test_mail_flow.py` — the characteristic end-to-end flow: SEND a uniquely-marked
|
||||
message to the mailbox over SMTP (authenticated submission to avoid rspamd greylisting), then
|
||||
RETRIEVE it over IMAP and assert the marker arrived (postfix → rspamd → dovecot deliver/store/fetch).
|
||||
2. `functional/test_mail_flow.py` — the characteristic end-to-end mail flow: INJECT a uniquely-marked
|
||||
message to the mailbox via the postfix container's local `sendmail` (locally-originated → not
|
||||
greylisted), then VERIFY delivery+storage via dovecot's `doveadm search` in the imap container —
|
||||
a real postfix → rspamd → dovecot deliver/store/fetch round-trip. We use the in-container mail
|
||||
tools (not the host network ports) because TLS_FLAVOR=notls makes dovecot refuse plaintext auth
|
||||
over the network (143); the in-container path exercises the same delivery/storage stack. (A
|
||||
network IMAP-auth test was dropped: under notls dovecot disallows plaintext network auth, so a
|
||||
host-side login is not a meaningful signal here.)
|
||||
|
||||
## Backup data-integrity (P4) — N/A (recipe ships no backup config)
|
||||
The upstream mailu recipe declares **no `backupbot.backup` label** on any service, so the cc-ci
|
||||
|
||||
@ -1,49 +0,0 @@
|
||||
"""mailu — recipe-specific functional test #2 (Phase 2 P3, distinctive: working IMAP mailbox).
|
||||
|
||||
Beyond "the row exists in config" (test_mailbox), this proves the provisioned mailbox is a REAL,
|
||||
usable IMAP account: it authenticates against dovecot over IMAP (port 143, host-published) and can
|
||||
SELECT its INBOX. That exercises mailu's actual mail-account stack (admin provisioning → dovecot auth
|
||||
backend), not just the admin DB. notls deploy → plain IMAP on 143. Hard assertion, non-vacuous: wrong
|
||||
credentials / a non-working account fail the login.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import imaplib
|
||||
import os
|
||||
import sys
|
||||
import time
|
||||
import uuid
|
||||
|
||||
sys.path.insert(0, os.path.dirname(__file__))
|
||||
import _mailu # noqa: E402
|
||||
|
||||
|
||||
def test_created_mailbox_authenticates_over_imap(live_app):
|
||||
mail_domain = live_app
|
||||
local = "ccci-imap-" + uuid.uuid4().hex[:8]
|
||||
password = "CcCi-" + uuid.uuid4().hex[:16] + "-Aa1!"
|
||||
_mailu.ensure_domain(live_app, mail_domain)
|
||||
email = _mailu.create_user(live_app, local, mail_domain, password)
|
||||
|
||||
# The account may take a moment to be usable by dovecot after admin provisioning; retry login.
|
||||
deadline = time.time() + 90
|
||||
last_err = None
|
||||
while time.time() < deadline:
|
||||
try:
|
||||
imap = imaplib.IMAP4("127.0.0.1", 143)
|
||||
try:
|
||||
imap.login(email, password)
|
||||
typ, _ = imap.select("INBOX")
|
||||
assert typ == "OK", f"IMAP SELECT INBOX returned {typ} for {email}"
|
||||
imap.logout()
|
||||
return # authenticated + selected INBOX → working mailbox
|
||||
finally:
|
||||
try:
|
||||
imap.shutdown()
|
||||
except Exception: # noqa: BLE001
|
||||
pass
|
||||
except (imaplib.IMAP4.error, OSError) as e: # noqa: PERF203
|
||||
last_err = e
|
||||
time.sleep(5)
|
||||
raise AssertionError(f"created mailbox {email} could not authenticate over IMAP:143 — {last_err}")
|
||||
@ -1,87 +1,28 @@
|
||||
"""mailu — recipe-specific functional test #3 (Phase 2 P3, the characteristic end-to-end mail flow).
|
||||
"""mailu — recipe-specific functional test #2 (Phase 2 P3, the characteristic end-to-end mail flow).
|
||||
|
||||
Provision a mailbox, SEND a uniquely-marked message to it over SMTP, then RETRIEVE it over IMAP and
|
||||
assert the marker arrived. This is mailu's defining behaviour (it is a mail server): a full
|
||||
deliver→store→fetch round-trip across postfix + rspamd + dovecot — not an API/health stand-in.
|
||||
Provision a mailbox, INJECT a uniquely-marked message to it via the postfix container's local
|
||||
`sendmail` (locally-originated → not greylisted, no auth/TLS needed), then VERIFY it was DELIVERED
|
||||
and STORED by polling dovecot's `doveadm search` in the imap container. This is mailu's defining
|
||||
behaviour (it is a mail server): a real postfix → rspamd → dovecot deliver→store→fetch round-trip,
|
||||
not an API/health stand-in.
|
||||
|
||||
To avoid rspamd greylisting (which defers an unknown inbound sender's first attempt for minutes), we
|
||||
send via AUTHENTICATED submission as the user itself (authenticated mail is not greylisted). notls
|
||||
deploy → submission accepts AUTH on 587; we try 587 (STARTTLS-if-offered) then fall back to 25.
|
||||
Retrieval polls INBOX (and Junk, in case rspamd files it) for the marker.
|
||||
We use the in-container mail tools (sendmail/doveadm) rather than the host network ports because the
|
||||
TLS_FLAVOR=notls deploy makes dovecot refuse plaintext auth over the network (143) — the in-container
|
||||
path exercises the same delivery/storage stack without that constraint, and avoids rspamd greylisting
|
||||
of network senders.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import email.message
|
||||
import imaplib
|
||||
import os
|
||||
import smtplib
|
||||
import sys
|
||||
import time
|
||||
import uuid
|
||||
|
||||
sys.path.insert(0, os.path.dirname(__file__))
|
||||
sys.path.insert(0, os.path.join(os.path.dirname(__file__), "..", "..", "..", "runner"))
|
||||
import _mailu # noqa: E402
|
||||
|
||||
|
||||
def _send(addr_from: str, addr_to: str, password: str, marker: str) -> None:
|
||||
msg = email.message.EmailMessage()
|
||||
msg["From"] = addr_from
|
||||
msg["To"] = addr_to
|
||||
msg["Subject"] = marker
|
||||
msg.set_content(f"cc-ci mail-flow test body {marker}")
|
||||
|
||||
last = None
|
||||
# Authenticated submission on 587 (no greylist); fall back to plain 25 local delivery.
|
||||
for port in (587, 25):
|
||||
try:
|
||||
smtp = smtplib.SMTP("127.0.0.1", port, timeout=30)
|
||||
try:
|
||||
smtp.ehlo()
|
||||
if smtp.has_extn("starttls"):
|
||||
smtp.starttls()
|
||||
smtp.ehlo()
|
||||
if smtp.has_extn("auth"):
|
||||
try:
|
||||
smtp.login(addr_from, password)
|
||||
except smtplib.SMTPException as e: # noqa: PERF203
|
||||
last = e # 25 may not offer/allow AUTH; local delivery still works
|
||||
smtp.send_message(msg)
|
||||
return
|
||||
finally:
|
||||
try:
|
||||
smtp.quit()
|
||||
except Exception: # noqa: BLE001
|
||||
pass
|
||||
except (smtplib.SMTPException, OSError) as e:
|
||||
last = e
|
||||
raise AssertionError(f"could not send mail via SMTP 587/25: {last}")
|
||||
|
||||
|
||||
def _retrieve(email_addr: str, password: str, marker: str, timeout: int = 150) -> bool:
|
||||
deadline = time.time() + timeout
|
||||
while time.time() < deadline:
|
||||
try:
|
||||
imap = imaplib.IMAP4("127.0.0.1", 143)
|
||||
try:
|
||||
imap.login(email_addr, password)
|
||||
for box in ("INBOX", "Junk"):
|
||||
typ, _ = imap.select(box)
|
||||
if typ != "OK":
|
||||
continue
|
||||
typ, data = imap.search(None, "SUBJECT", f'"{marker}"')
|
||||
if typ == "OK" and data and data[0].split():
|
||||
return True
|
||||
imap.logout()
|
||||
finally:
|
||||
try:
|
||||
imap.shutdown()
|
||||
except Exception: # noqa: BLE001
|
||||
pass
|
||||
except (imaplib.IMAP4.error, OSError): # noqa: PERF203
|
||||
pass
|
||||
time.sleep(5)
|
||||
return False
|
||||
from harness import lifecycle # noqa: E402
|
||||
|
||||
|
||||
def test_send_and_receive_mail(live_app):
|
||||
@ -92,9 +33,25 @@ def test_send_and_receive_mail(live_app):
|
||||
email_addr = _mailu.create_user(live_app, local, mail_domain, password)
|
||||
|
||||
marker = "ccci-mailflow-" + uuid.uuid4().hex
|
||||
_send(email_addr, email_addr, password, marker)
|
||||
sender = f"admin@{mail_domain}"
|
||||
# Inject via the postfix container's local sendmail (locally-originated; no greylist/auth/TLS).
|
||||
msg = f"From: {sender}\\nTo: {email_addr}\\nSubject: {marker}\\n\\nbody {marker}\\n"
|
||||
inject = f"printf '{msg}' | sendmail -f {sender} {email_addr}"
|
||||
lifecycle.exec_in_app(live_app, ["sh", "-c", inject], service="smtp")
|
||||
|
||||
assert _retrieve(email_addr, password, marker), (
|
||||
f"mail with marker {marker!r} was sent to {email_addr} but never arrived in INBOX/Junk "
|
||||
f"over IMAP within the delivery window (postfix→rspamd→dovecot)"
|
||||
# Poll dovecot for the stored message (INBOX, then Junk in case rspamd files it).
|
||||
deadline = time.time() + 150
|
||||
while time.time() < deadline:
|
||||
for box in ("INBOX", "Junk"):
|
||||
query = (
|
||||
f"doveadm search -u '{email_addr}' mailbox {box} "
|
||||
f"header subject '{marker}'"
|
||||
)
|
||||
out = lifecycle.exec_in_app(live_app, ["sh", "-c", query], service="imap")
|
||||
if out.strip(): # a non-empty result = "<mailbox-guid> <uid>" → message stored
|
||||
return
|
||||
time.sleep(5)
|
||||
raise AssertionError(
|
||||
f"mail with subject {marker!r} injected to {email_addr} was not delivered/stored "
|
||||
f"(doveadm search found nothing in INBOX/Junk within the postfix→rspamd→dovecot window)"
|
||||
)
|
||||
|
||||
Reference in New Issue
Block a user