207 lines
8.6 KiB
Markdown
207 lines
8.6 KiB
Markdown
# mailmig
|
||
|
||
Moves mail from a **Stalwart** server to **Simple NixOS Mailserver** (SNM) in
|
||
three stages, in this order:
|
||
|
||
1. **Users** – read accounts, aliases, quotas and mailing lists from Stalwart's
|
||
management API and render them for SNM (Nix data + password hashes), so
|
||
every mailbox exists on the new server before any mail moves. Password
|
||
hashes are carried over where Dovecot can verify them: users keep their
|
||
passwords.
|
||
2. **Mail** – copy every folder over IMAP (master-user login on both sides, no
|
||
user passwords needed). Re-runnable: each run copies only what is missing,
|
||
so you can bulk-copy early and run delta passes after the switch.
|
||
3. **DNS** – compute MX/SPF/DKIM/DMARC/SRV per domain, diff against what is
|
||
published (Cloudflare API), apply with a rollback journal, verify via DoH.
|
||
|
||
Single static binary, Rust, no state besides the files listed below.
|
||
|
||
## Build
|
||
|
||
```sh
|
||
cargo build --release # target/release/mailmig
|
||
nix build # ./result/bin/mailmig
|
||
nix flake check -L # unit tests + VM end-to-end test (needs KVM)
|
||
```
|
||
|
||
## Files
|
||
|
||
| file | written by | contents |
|
||
|---|---|---|
|
||
| `mailmig.toml` | you (`mailmig init`) | servers, logins, folder/DNS settings |
|
||
| `plan.toml` | `inventory`, then you | accounts, aliases, forwards, notes – **review and edit** |
|
||
| `secrets.json` | `inventory` | password hashes (mode 0600; never commit) |
|
||
| `out/` | `users` | `mail-accounts.nix`, `hashes/`, `clan-vars-set.sh`, `dovecot-master-user.nix` |
|
||
| `reports/*.json` | `sync`, `verify` | per-account, per-folder counts and errors |
|
||
| `dns-journal/*.json` | `dns apply` | records before the change + what was applied (for rollback) |
|
||
|
||
All of them are git-ignored.
|
||
|
||
## Runbook
|
||
|
||
### 0. Prepare both servers
|
||
|
||
**Stalwart (source)** – an account that may open any mailbox:
|
||
|
||
* 0.15: master user in the config, login `"<account>%<master>"`:
|
||
```toml
|
||
[authentication.master]
|
||
user = "mailmig"
|
||
secret = "<long random secret>"
|
||
```
|
||
* 0.16+: give a normal account the `impersonate` permission (via a role); the
|
||
password must be its main password, not an app password.
|
||
|
||
and admin credentials (or an API key) for the management API.
|
||
|
||
**SNM (destination)** – a temporary Dovecot master user. `mailmig users`
|
||
writes `out/dovecot-master-user.nix`; import it on the mail host while
|
||
migrating and create the passwd file it points at:
|
||
|
||
```sh
|
||
printf 'mailmig:{CRYPT}%s\n' "$(mkpasswd -m sha-512)" > /var/lib/dovecot-master-users
|
||
```
|
||
|
||
### 1. Inventory → plan
|
||
|
||
```sh
|
||
mailmig init # writes mailmig.toml; edit hosts/logins
|
||
export MAILMIG_STALWART_ADMIN_PASSWORD=...
|
||
mailmig inventory --raw raw.json
|
||
```
|
||
|
||
Prints a summary and a list of notes (accounts without addresses, app
|
||
passwords, TOTP, groups, lists with unknown members, catch-all settings).
|
||
Edit `plan.toml`: drop admin/service accounts, set `migrate_mail = false`
|
||
where only the account must exist, adjust quotas. `--from-raw raw.json`
|
||
rebuilds the plan offline. Stalwart ≤ 0.15 is read over REST
|
||
(`/api/principal`), ≥ 0.16 over JMAP (`x:Account`, `x:MailingList`); `kind = "auto"`
|
||
picks one from `/.well-known/jmap`.
|
||
|
||
### 2. Users on the new server
|
||
|
||
```sh
|
||
mailmig users # add --generate-missing to mint passwords
|
||
```
|
||
|
||
* `out/mail-accounts.nix` – `{ domains; accounts.<login> = { aliases; quota; }; forwards; }`
|
||
* `out/hashes/<login>.hash` – Dovecot-ready hash per account
|
||
* `out/clan-vars-set.sh` – loads those hashes into clan vars
|
||
|
||
Passwords that carry over: `$6$`/`$5$` (sha-crypt), `$2b$`/`$2y$` (bcrypt),
|
||
`$y$` (yescrypt), `$1$`, `{SSHA}`/`{SHA*}`-style schemes, Argon2 (needs Dovecot
|
||
with libsodium), and plaintext secrets (hashed by mailmig). Others (PBKDF2,
|
||
scrypt) and accounts without a password are listed as needing a new one.
|
||
App passwords and TOTP do not exist in SNM.
|
||
|
||
**cnx-network-clan** (`modules/mail.nix`): copy `mail-accounts.nix` next to it
|
||
and merge it into the existing account list, so each migrated address gets
|
||
its clan vars generator like any other mailbox:
|
||
|
||
```nix
|
||
let
|
||
migrated = import ./mail-accounts.nix;
|
||
accounts = [ "postmaster@cnx.email" ] ++ builtins.attrNames migrated.accounts;
|
||
# ... genName / passwdGenerators / loginAccounts unchanged ...
|
||
in
|
||
{
|
||
mailserver.domains = [ "cnx.email" ] ++ migrated.domains;
|
||
mailserver.accounts = lib.recursiveUpdate loginAccounts (
|
||
lib.mapAttrs (_: a: { inherit (a) aliases; quota = a.quota or null; }) migrated.accounts
|
||
);
|
||
mailserver.forwards = migrated.forwards;
|
||
}
|
||
```
|
||
|
||
Then `clan vars generate mx1` (creates the generators), `bash out/clan-vars-set.sh`
|
||
(replaces each generated hash with the Stalwart one, so the user's old
|
||
password keeps working), `clan machines update mx1`. Accounts without a
|
||
carried hash keep the generated passphrase (`clan vars get mx1 mail-passwd-…/passphrase`).
|
||
|
||
Copy the new DKIM keys for the DNS step: `scp 'mx1:/var/dkim/*.txt' dkim/`.
|
||
|
||
### 3. Check logins
|
||
|
||
```sh
|
||
export MAILMIG_SRC_MASTER_PASSWORD=... MAILMIG_DST_MASTER_PASSWORD=...
|
||
mailmig check
|
||
```
|
||
|
||
### 4. Copy mail (old server keeps running)
|
||
|
||
```sh
|
||
mailmig sync --dry-run # what would be copied
|
||
mailmig sync # bulk copy, 4 accounts in parallel (-j)
|
||
mailmig sync --sync-flags # later passes before the switch: also copy read/flagged state
|
||
```
|
||
|
||
* Folders: special-use folders map by attribute (Stalwart "Sent Items",
|
||
"Deleted Items", "Junk Mail" → SNM "Sent", "Trash", "Junk"); everything else
|
||
keeps its path with the destination's separator (`Projects/2024` →
|
||
`Projects.2024`; a `.` inside a name becomes `_`). Override with
|
||
`[folders.map]`, skip with `[folders.exclude]`. Shared/other-users
|
||
namespaces are skipped.
|
||
* Each message keeps its flags (minus `\Recent`) and INTERNALDATE.
|
||
Messages flagged `\Deleted` are not copied (`skip_deleted`).
|
||
* De-duplication is by Message-ID (or Date/From/Subject digest without one),
|
||
counted, so legitimate duplicates are preserved. Nothing is ever deleted on
|
||
either side.
|
||
* Exit code 1 if any message failed (quota, size limit); details in the report.
|
||
|
||
### 5. DNS
|
||
|
||
```sh
|
||
export CLOUDFLARE_API_TOKEN=... # Zone:DNS:Edit on the mail zones
|
||
mailmig dns plan # diff per domain, warnings
|
||
mailmig dns apply --yes # journal in dns-journal/
|
||
mailmig dns verify # public view via DNS-over-HTTPS
|
||
mailmig dns rollback dns-journal/<domain>-<time>.json --yes
|
||
mailmig dns zone # zone-file lines for Knot-served zones
|
||
```
|
||
|
||
Per domain mailmig manages exactly: MX, the SPF TXT (`v=spf1…`), `_dmarc`,
|
||
`<selector>._domainkey` and the RFC 6186 SRV records (`_imaps`,
|
||
`_submission(s)`). Mail SRV records that point at an old host and that SNM
|
||
does not offer (`_pop3s`, `_jmap`, `_caldavs`, …) are deleted; everything
|
||
else is left alone, but records still pointing at the old host (e.g.
|
||
`autoconfig` CNAMEs) are reported. Per-domain SPF/DMARC/MX overrides:
|
||
`[dns.overrides."domain"]` (keep `include:` for other senders such as a
|
||
website or newsletter tool).
|
||
|
||
**MTA-STS**: if a domain publishes `_mta-sts` with an `enforce` policy that
|
||
lists only the old MX, senders that cached it refuse to deliver to the new
|
||
MX until `max_age` runs out. `dns plan` shows the live policy. Fix that
|
||
first (policy listing both MX hosts or `mode: testing`, bump the id, wait
|
||
`max_age`), then switch MX.
|
||
|
||
### 6. After the switch
|
||
|
||
* Run `mailmig sync` (without `--sync-flags`) for a few days: mail that
|
||
still reaches the old server through cached MX records is copied over.
|
||
* `mailmig verify` exits 0 when nothing is missing.
|
||
* Mail clients: point IMAP/SMTP at the new host (`client_host`), or give the
|
||
new server a certificate for the old hostname and repoint that name.
|
||
* Remove `dovecot-master-user.nix` and `/var/lib/dovecot-master-users`, and
|
||
the Stalwart master user.
|
||
|
||
## Not migrated
|
||
|
||
Sieve filters and vacation replies, contacts and calendars (SNM has no
|
||
CardDAV/CalDAV), app passwords, TOTP, shared-folder ACLs, catch-all rules
|
||
(reported as notes; SNM uses `catchAll` per account).
|
||
|
||
## Tests
|
||
|
||
`cargo test`: IMAP parser, modified UTF-7, folder mapping, de-duplication,
|
||
hash classification, Stalwart REST/JMAP shapes, Nix rendering, DNS diff and
|
||
zone output, Cloudflare API against a mock server (apply + rollback).
|
||
|
||
`nix flake check` (`checks.e2e`): three VMs – Stalwart 0.15.5, SNM
|
||
(nixos-26.05, Dovecot 2.4) importing the golden `tests/expected-mail-accounts.nix`
|
||
and `tests/expected-dovecot-master-user.nix`, and a client running mailmig
|
||
through the whole runbook: inventory over REST, hash carry-over (sha512-crypt,
|
||
bcrypt, plaintext, none → generated), STARTTLS master login on Stalwart,
|
||
implicit-TLS master login on Dovecot, folder mapping, flags/dates/duplicates/
|
||
`\Deleted`/3 MB message, idempotent re-run, delta + flag pass, and a zone file
|
||
from SNM's real DKIM keys.
|