Self-Hosting Your Email: Postfix, Dovecot, Stalwart, and the Deliverability Gauntlet

Running your own mail server is the final frontier of self-hosting. You can containerize every other workload, own your DNS, proxy everything through Traefik, and still depend on Google or Microsoft for the service that arguably matters most. Email is the identity layer of the internet — it’s how you reset passwords, receive invoices, and communicate with people who aren’t on your preferred platform. Owning that layer completely is a compelling idea.

It is also genuinely hard. Not “hard to understand” hard — the protocols are well-documented and the software is mature. Hard in the specific sense that the major email providers have constructed an obstacle course of reputation requirements, cryptographic authentication standards, and opaque filtering systems that can silently drop your carefully crafted messages into a spam folder with no explanation. Self-hosting email means fighting this system every day, not just at setup time.

This guide is written for technically advanced homelabbers who understand what they’re getting into. It covers the full stack — protocols, Postfix+Dovecot+Rspamd, the modern Stalwart alternative, complete DNS configuration, deliverability strategy, and a pragmatic discussion of when full self-hosting is and isn’t the right answer.

Part 1: Why Self-Host — and Why You Might Not Want To

The Case For

Privacy and sovereignty. When you run your own mail server, your messages live on your hardware and transit through infrastructure you control. No Google employee, algorithm, or government subpoena to a third party can read your mail without going through you first. For privacy absolutists, this is non-negotiable.

No vendor lock-in. Your domain is yours. Your mail files are yours. If you want to migrate to a different stack next year, you can. You’re not locked into Gmail’s UI, Google’s storage pricing, or Apple’s ecosystem.

Cost at scale. A single VPS running a mail server can handle dozens of users and domains for roughly $6–10/month — less than a single seat of Google Workspace. For organizations with many users, the economics shift significantly in favor of self-hosting.

Learning. SMTP, IMAP, DKIM, DMARC, and SPF are foundational internet protocols. Actually operating a mail server teaches you things that reading documentation never will. If you work in infrastructure, this knowledge pays off repeatedly.

The Honest Case Against

Let’s be direct about the difficulties, because most tutorials skip this part.

Deliverability is genuinely hard. Gmail and Outlook use reputation systems that treat new mail servers as guilty until proven innocent. Even with perfect DNS configuration, your first emails to Gmail users may land in spam or be rejected outright. This isn’t a configuration problem you can fix in an afternoon — it requires time, volume, and ongoing attention.

Residential IPs are almost always blocked. If you’re thinking about running a mail server on your home connection, don’t. Every major residential ISP blocks or heavily penalizes outbound port 25. Your home IP is probably already on multiple blacklists just by virtue of being residential. You need a clean VPS IP.

Your server being down means mail bounces. Unlike a web server where downtime means a temporary error page, an unreachable mail server causes senders to receive bounce messages or retry queues to accumulate. There’s no “I’ll fix it in the morning” — if your server is down during the retry window, mail is lost.

Maintenance burden is real and ongoing. Spam filtering requires tuning. Security patches need applying promptly (mail servers are high-value targets). Blacklists need monitoring. DKIM keys need rotating. TLS certificates need renewing. This isn’t a set-and-forget service.

The 99% solution exists. Fastmail costs $3/month per user. Proton Mail’s free tier is genuinely private. Migadu charges a flat rate for unlimited users. These services give you privacy-respecting email on your own domain without any of the maintenance overhead. If your goal is “use my own domain with reasonable privacy,” these services are the right answer for most people.

Who Should Self-Host Email

• Privacy absolutists who are unwilling to trust any third party with their messages, regardless of cost
• Developers and sysadmins who want to deeply understand email infrastructure
• Organizations that have specific compliance or control requirements
• Homelabbers who treat operational complexity as a feature, not a bug

Realistic Expectations

Receiving mail is easy. Any properly configured server with correct MX records will receive mail reliably within minutes of setup. The hard part is sending mail, specifically sending to Gmail and Outlook users without hitting spam filters. Expect to spend significant time on IP reputation, warming your sending IP, and ongoing monitoring. Even then, you may occasionally have deliverability problems that require debugging.

Part 2: How Email Works — The Protocol Stack

Before you configure anything, you need a clear mental model of how email actually moves.

The Core Protocols

SMTP (Simple Mail Transfer Protocol) handles everything related to sending and transferring mail between servers. It operates on three ports:

• Port 25: Server-to-server relay. When Gmail delivers mail to your server, it connects on port 25. Your server sends outbound mail by connecting to recipient servers on port 25. This is the port that matters most for deliverability and the one many providers block.
• Port 587: Mail submission. Your email client (Thunderbird, Apple Mail) connects here to hand off messages for delivery. Requires STARTTLS and authentication.
• Port 465: SMTPS (SMTP over TLS). Historically deprecated, then re-standardized. Some clients prefer it. Support it if you can.

IMAP (Internet Message Access Protocol) is how your email client reads mail stored on the server. Messages stay on the server; the client downloads headers and bodies on demand. Port 993 with TLS is the standard. IMAP supports folders, flags (read/unread/starred), and multiple clients staying in sync.

POP3 is the older retrieval protocol that downloads and deletes messages from the server. Mostly obsolete for anything other than simple single-device setups. You’ll configure it if clients ask, but don’t prioritize it.

Mail Flow End-to-End

Sending a message:

MUA (Thunderbird)
  → port 587 → MSA (your Postfix, submission port)
  → port 25 → recipient's MTA (Gmail's inbound server)
  → MDA (Gmail stores the message)
  → IMAP → recipient's MUA

Receiving a message:

Sender's MTA
  → port 25 → your MTA (Postfix)
  → LMTP or pipe → MDA (Dovecot or Postfix virtual delivery)
  → Maildir on disk
  → port 993 → your IMAP client

Key Components

DNS Records That Matter for Email

Email has five categories of DNS records you need to understand before you touch a config file:

• MX: Points to your mail server. The record that makes receiving work.
• A/AAAA: Maps your mail server hostname to its IP address.
• SPF (TXT): Declares which IP addresses are authorized to send mail for your domain.
• DKIM (TXT): Holds the public half of the cryptographic key pair used to sign outgoing mail.
• DMARC (TXT): Policy specifying what to do when SPF or DKIM fail.
• PTR (reverse DNS): Maps your IP address back to your hostname. Set at your VPS provider, not your DNS registrar.

Part 3: Prerequisites and Infrastructure Requirements

A Clean VPS IP Address

This is the most critical requirement and the one most likely to cause you problems. You need:

1. An IP that is not on blacklists. Before buying a VPS for mail, check the IP against MXToolbox Blacklist Check. Reputable VPS providers rotate IPs from canceled accounts, and some previous tenant may have been a spammer.

2. Port 25 unblocked. Many VPS providers block port 25 by default to prevent spam. Check before purchasing, or check immediately after provisioning.

Providers with port 25 available (some may require a support ticket):

Providers that block port 25 (avoid for mail):

If you’re on a provider that blocks port 25, you have two options: switch providers, or use a transactional relay service for outbound (covered in Part 8).

Reverse DNS (PTR Record)

Your VPS IP address must resolve back to your mail server’s hostname. This is called a PTR or reverse DNS record, and it’s set through your VPS provider’s control panel — not your DNS registrar.

If your mail server is mail.yourdomain.com with IP 1.2.3.4, then:

host 1.2.3.4
# Should return: 4.3.2.1.in-addr.arpa domain name pointer mail.yourdomain.com

Gmail, Outlook, and most spam filters reject or heavily penalize mail from IPs without a matching PTR record. This is non-negotiable.

Other Requirements

• A domain you own — ideally one that’s at least a few months old. Brand-new domains have no reputation and face extra scrutiny.
• Static IP — dynamic IPs get blacklisted automatically by many spam databases.
• SSL/TLS certificate — Let’s Encrypt works perfectly. Certbot is the standard tool.
• Firewall rules — open these ports, block everything else:

1
2
3
4
5
6
7
8

# Inbound
ufw allow 25/tcp    # SMTP (server-to-server)
ufw allow 587/tcp   # Submission (client to server)
ufw allow 465/tcp   # SMTPS
ufw allow 993/tcp   # IMAP over TLS
ufw allow 80/tcp    # HTTP (for Let's Encrypt)
ufw allow 443/tcp   # HTTPS (webmail)
ufw allow 22/tcp    # SSH (restrict to your IP in production)

Part 4: Option A — The Classic Stack: Postfix + Dovecot + Rspamd

The Postfix+Dovecot combination has been the dominant self-hosted mail stack for twenty years. It is battle-tested, extensively documented, and flexible enough to handle everything from a single-user setup to a large multi-domain organization. Rspamd is the modern spam filter that replaced the slower SpamAssassin as the community default.

Postfix (MTA)

Postfix handles all SMTP: receiving inbound mail on port 25, accepting submissions from clients on port 587, and relaying outbound mail to recipient servers.

Installation:

1
2

apt update && apt install postfix postfix-mysql -y
# During install, select "Internet Site" and enter your domain name

Core configuration file — /etc/postfix/main.cf:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53

# Identity
myhostname = mail.yourdomain.com
mydomain = yourdomain.com
myorigin = $mydomain

# Network
inet_interfaces = all
inet_protocols = all

# What to deliver locally
mydestination = $myhostname, localhost.$mydomain, localhost

# Virtual mailboxes (users not in /etc/passwd)
virtual_mailbox_domains = yourdomain.com
virtual_mailbox_base = /var/mail/vhosts
virtual_mailbox_maps = hash:/etc/postfix/vmailbox
virtual_minimum_uid = 100
virtual_uid_maps = static:5000
virtual_gid_maps = static:5000

# TLS for inbound connections (smtpd = server mode)
smtpd_tls_cert_file = /etc/letsencrypt/live/mail.yourdomain.com/fullchain.pem
smtpd_tls_key_file = /etc/letsencrypt/live/mail.yourdomain.com/privkey.pem
smtpd_use_tls = yes
smtpd_tls_security_level = may
smtpd_tls_auth_only = yes
smtpd_tls_protocols = !SSLv2, !SSLv3, !TLSv1, !TLSv1.1
smtpd_tls_loglevel = 1

# TLS for outbound connections (smtp = client mode)
smtp_tls_security_level = may
smtp_tls_loglevel = 1

# SASL authentication (clients submitting mail)
smtpd_sasl_type = dovecot
smtpd_sasl_path = private/auth
smtpd_sasl_auth_enable = yes
smtpd_recipient_restrictions =
    permit_sasl_authenticated,
    permit_mynetworks,
    reject_unauth_destination

# Relay restrictions
smtpd_relay_restrictions =
    permit_mynetworks,
    permit_sasl_authenticated,
    defer_unauth_destination

# Milter for Rspamd
milter_protocol = 6
milter_default_action = accept
smtpd_milters = inet:localhost:11332
non_smtpd_milters = inet:localhost:11332

Virtual mailbox map — /etc/postfix/vmailbox:

admin@yourdomain.com     yourdomain.com/admin/
user@yourdomain.com      yourdomain.com/user/

After editing, compile it: postmap /etc/postfix/vmailbox

Submission port — /etc/postfix/master.cf:

Uncomment and configure the submission stanza:

submission inet n       -       y       -       -       smtpd
  -o syslog_name=postfix/submission
  -o smtpd_tls_security_level=encrypt
  -o smtpd_sasl_auth_enable=yes
  -o smtpd_tls_auth_only=yes
  -o smtpd_reject_unlisted_recipient=no
  -o smtpd_recipient_restrictions=permit_sasl_authenticated,reject
  -o milter_macro_daemon_name=ORIGINATING

smtps     inet  n       -       y       -       -       smtpd
  -o syslog_name=postfix/smtps
  -o smtpd_tls_wrappermode=yes
  -o smtpd_sasl_auth_enable=yes
  -o smtpd_recipient_restrictions=permit_sasl_authenticated,reject

Create the mailbox directories:

1
2
3
4

mkdir -p /var/mail/vhosts/yourdomain.com
groupadd -g 5000 vmail
useradd -g vmail -u 5000 vmail -d /var/mail/vhosts -s /sbin/nologin
chown -R vmail:vmail /var/mail/vhosts

Dovecot (IMAP/MDA)

Dovecot serves your mail to IMAP clients and handles the final delivery from Postfix to the Maildir on disk.

Installation:

1

apt install dovecot-core dovecot-imapd dovecot-lmtpd -y

Primary configuration — /etc/dovecot/dovecot.conf:

protocols = imap lmtp
mail_location = maildir:/var/mail/vhosts/%d/%n/

Authentication — /etc/dovecot/conf.d/10-auth.conf:

disable_plaintext_auth = yes
auth_mechanisms = plain login

!include auth-passwdfile.conf.ext

Passwd-file authentication — /etc/dovecot/conf.d/auth-passwdfile.conf.ext:

passdb {
  driver = passwd-file
  args = scheme=SHA512-CRYPT username_format=%u /etc/dovecot/users
}

userdb {
  driver = passwd-file
  args = username_format=%u /etc/dovecot/users
  default_fields = uid=vmail gid=vmail home=/var/mail/vhosts/%d/%n
}

Create users with doveadm pw to generate password hashes:

1
2
3
4
5
6

doveadm pw -s SHA512-CRYPT
# Enter password, copy the hash

# /etc/dovecot/users format:
# user@domain:{SHA512-CRYPT}$6$...hash...
echo "admin@yourdomain.com:{SHA512-CRYPT}\$6\$yourhashhere" >> /etc/dovecot/users

TLS — /etc/dovecot/conf.d/10-ssl.conf:

ssl = required
ssl_cert = </etc/letsencrypt/live/mail.yourdomain.com/fullchain.pem
ssl_key = </etc/letsencrypt/live/mail.yourdomain.com/privkey.pem
ssl_min_protocol = TLSv1.2
ssl_cipher_list = ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:...
ssl_prefer_server_ciphers = yes

LMTP socket for Postfix — /etc/dovecot/conf.d/10-master.conf:

service lmtp {
  unix_listener /var/spool/postfix/private/dovecot-lmtp {
    mode = 0600
    user = postfix
    group = postfix
  }
}

service auth {
  unix_listener /var/spool/postfix/private/auth {
    mode = 0666
    user = postfix
    group = postfix
  }
}

Add to Postfix main.cf to use LMTP:

1

virtual_transport = lmtp:unix:private/dovecot-lmtp

Rspamd (Spam Filter)

Rspamd is a modern, fast, and highly accurate spam filter written in C. It integrates with Postfix via the milter protocol, can sign outgoing mail with DKIM, and provides a web UI for monitoring and training.

Installation:

1
2
3
4
5

# Add the Rspamd repository
curl https://rspamd.com/apt-stable/gpg.key | sudo apt-key add -
echo "deb https://rspamd.com/apt-stable/ $(lsb_release -cs) main" \
    > /etc/apt/sources.list.d/rspamd.list
apt update && apt install rspamd -y

Enable the milter worker — /etc/rspamd/local.d/worker-proxy.inc:

1
2
3
4
5
6

bind_socket = "localhost:11332";
milter = yes;
upstream "local" {
    default = yes;
    self_scan = yes;
}

DKIM signing in Rspamd — /etc/rspamd/local.d/dkim_signing.conf:

1
2
3
4
5
6
7
8
9

enabled = true;
allow_username_mismatch = true;

domain {
    yourdomain.com {
        selector = "mail";
        path = "/etc/rspamd/dkim/yourdomain.com.mail.key";
    }
}

Generate the DKIM key pair:

1
2
3
4
5
6
7

mkdir -p /etc/rspamd/dkim
rspamadm dkim_keygen -s mail -d yourdomain.com \
    -k /etc/rspamd/dkim/yourdomain.com.mail.key > /tmp/dkim_public.txt
cat /tmp/dkim_public.txt
# Copy the TXT record value — add it to your DNS as mail._domainkey.yourdomain.com
chown -R _rspamd:_rspamd /etc/rspamd/dkim
chmod 440 /etc/rspamd/dkim/*.key

Training the Bayesian filter:

1
2
3
4
5
6
7
8

# Mark a message as spam
rspamc learn_spam < suspicious-message.eml

# Mark a message as ham (legitimate)
rspamc learn_ham < good-message.eml

# Check classification statistics
rspamc stat

Access the Rspamd web UI (bind to localhost, proxy via Nginx/Traefik):

1
2
3
4
5

# Set a password for the web UI
rspamadm pw
# Add to /etc/rspamd/local.d/worker-controller.inc:
# password = "$2$yourhashedpassword";
# bind_socket = "localhost:11334";

The docker-mailserver Shortcut

If the above feels like a lot of moving parts, docker-mailserver packages Postfix, Dovecot, Rspamd (or Amavis), and supporting tools into a single Docker Compose stack with a management script. It’s an excellent starting point.

docker-compose.yml:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27

services:
  mailserver:
    image: ghcr.io/docker-mailserver/docker-mailserver:latest
    container_name: mailserver
    hostname: mail.yourdomain.com
    env_file: mailserver.env
    ports:
      - "25:25"
      - "465:465"
      - "587:587"
      - "993:993"
    volumes:
      - ./docker-data/dms/mail-data/:/var/mail/
      - ./docker-data/dms/mail-state/:/var/mail-state/
      - ./docker-data/dms/mail-logs/:/var/log/mail/
      - ./docker-data/dms/config/:/tmp/docker-mailserver/
      - /etc/letsencrypt:/etc/letsencrypt:ro
    restart: always
    stop_grace_period: 1m
    cap_add:
      - NET_ADMIN
    healthcheck:
      test: "ss --listening --tcp | grep -P 'LISTEN.+:smtp' || exit 1"
      timeout: 3s
      retries: 0

volumes: {}

mailserver.env (key settings):

1
2
3
4
5
6
7
8
9

OVERRIDE_HOSTNAME=mail.yourdomain.com
DOMAINNAME=yourdomain.com
SSL_TYPE=letsencrypt
ENABLE_RSPAMD=1
ENABLE_CLAMAV=0
ENABLE_FAIL2BAN=1
SPOOF_PROTECTION=1
ENABLE_UPDATE_CHECK=1
POSTMASTER_ADDRESS=admin@yourdomain.com

User management:

1
2
3
4
5
6
7
8

# Add a user
./setup.sh email add user@yourdomain.com password123

# Add a DKIM key
./setup.sh config dkim keysize 2048 domain yourdomain.com

# List users
./setup.sh email list

Part 5: Option B — Stalwart, the Modern All-in-One

Stalwart Mail Server is a complete rewrite of the mail server concept in Rust. Instead of assembling Postfix, Dovecot, Rspamd, and supporting daemons, Stalwart is a single binary that handles SMTP, IMAP, JMAP, and built-in spam filtering. It ships with a web-based admin UI and is under active development.

Why Stalwart Is Compelling

• Single binary or single container. No juggling multiple services and their interactions.
• JMAP support. JMAP (JSON Meta Application Protocol) is the modern successor to IMAP — push-based instead of poll-based, faster for mobile clients, and much easier to implement correctly. Stalwart is one of the few production mail servers with full JMAP support.
• Web admin UI out of the box. Manage domains, users, aliases, and spam thresholds through a browser.
• Active development. Stalwart releases frequently with meaningful improvements, unlike some of the more mature but slower-moving alternatives.
• Built-in spam filtering. Stalwart includes statistical spam filtering without requiring a separate milter daemon.

Docker Deployment

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21

services:
  stalwart:
    image: stalwartlabs/mail-server:latest
    container_name: stalwart
    restart: unless-stopped
    ports:
      - "25:25"      # SMTP (inbound)
      - "465:465"    # SMTPS
      - "587:587"    # Submission
      - "993:993"    # IMAPS
      - "4190:4190"  # ManageSieve (optional, for server-side filters)
      - "8080:8080"  # Web admin UI
    volumes:
      - stalwart_data:/opt/stalwart-mail
    cap_add:
      - NET_BIND_SERVICE
    environment:
      - TZ=UTC

volumes:
  stalwart_data:

Initial setup:

1
2
3
4

docker compose up -d

# Initialize the server and get the admin password
docker exec -it stalwart /usr/local/bin/stalwart-mail --init

The init command will output a random admin password and a setup URL. Navigate to http://yourserver:8080 to log in.

Post-init configuration via the web UI:

1. Add your domain under Domains → Add Domain
2. Configure TLS — paste your Let’s Encrypt certificate and key paths, or enable ACME (Stalwart can manage certificates automatically)
3. Add users under Accounts → Add Account
4. Configure DKIM — Stalwart generates and manages DKIM keys per domain; the UI shows you the DNS TXT record to add
5. Review spam thresholds under Settings → Spam Filter — the defaults are reasonable but you’ll tune them after observing real traffic

Accessing Stalwart’s JMAP endpoint:

Clients that support JMAP (like Mimestream on macOS or the Stalwart webmail) connect to https://mail.yourdomain.com/jmap. Standard IMAP clients use port 993 as usual — Stalwart speaks both.

Stalwart vs. Postfix+Dovecot — When to Choose Each

Choose Postfix+Dovecot if you need proven reliability, have complex routing requirements, want maximum community support, or are deploying for an organization where stability is paramount.

Choose Stalwart if you want the simplest possible setup, care about JMAP, prefer managing things through a UI, and are comfortable being an early adopter. It’s also an excellent choice for a personal single-user setup.

Part 6: DNS Configuration — The Deliverability Foundation

DNS is where most self-hosted mail servers fail. Not because of missing records, but because of misconfigured or incomplete ones. Every record here is required for good deliverability.

MX Record

Tells the world where to deliver mail for your domain.

yourdomain.com.  300  IN  MX  10  mail.yourdomain.com.

The 10 is priority — lower numbers are preferred. If you have a backup MX, give it a higher number (e.g., 20).

SPF — Sender Policy Framework

A TXT record that declares which IP addresses are authorized to send email for your domain. Receiving servers check this against the IP that connected to them.

For a self-hosted-only setup (recommended):

yourdomain.com.  300  IN  TXT  "v=spf1 ip4:1.2.3.4 -all"

• ip4:1.2.3.4 — your server’s IP
• -all — hard fail: any server not listed is unauthorized. Use this for self-hosted-only setups.
• ~all — soft fail: any server not listed is suspicious. Use this if you also send via a relay service.

If you also send via Google (e.g., forwarding through Gmail):

yourdomain.com.  300  IN  TXT  "v=spf1 ip4:1.2.3.4 include:_spf.google.com ~all"

SPF has a 10 DNS lookup limit. Keep your record simple.

DKIM — DomainKeys Identified Mail

DKIM cryptographically signs every outgoing message using a private key you hold. Receiving servers verify the signature against the public key in your DNS. A valid DKIM signature proves the message genuinely came from your server and wasn’t modified in transit.

If using Rspamd, the key is generated as shown in Part 4. The DNS record looks like:

mail._domainkey.yourdomain.com.  300  IN  TXT  "v=DKIM1; k=rsa; p=MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEA..."

The mail before ._domainkey is the selector — it matches the s=mail value in the DKIM-Signature header of your outgoing mail.

If using opendkim standalone:

1
2
3
4
5
6

mkdir -p /etc/opendkim/keys/yourdomain.com
opendkim-genkey -s mail -d yourdomain.com \
    --directory /etc/opendkim/keys/yourdomain.com
# Private key: /etc/opendkim/keys/yourdomain.com/mail.private
# Public key DNS record: /etc/opendkim/keys/yourdomain.com/mail.txt
cat /etc/opendkim/keys/yourdomain.com/mail.txt

DKIM key rotation: Rotate your DKIM keys annually. Add the new key as a new selector (e.g., mail2026._domainkey), update your signing configuration, wait for DNS propagation, then remove the old selector.

DMARC — Domain-based Message Authentication Reporting and Conformance

DMARC ties SPF and DKIM together and tells receiving servers what to do when they fail. It also provides a reporting mechanism so you can monitor authentication results across the internet.

Start with monitoring only:

_dmarc.yourdomain.com.  300  IN  TXT  "v=DMARC1; p=none; rua=mailto:dmarc@yourdomain.com"

• p=none: monitor only, take no action. Use this first to understand your mail streams.
• rua=: aggregate report recipients. You’ll receive daily XML summaries from major providers.

After reviewing reports, tighten to quarantine:

_dmarc.yourdomain.com.  300  IN  TXT  "v=DMARC1; p=quarantine; pct=50; rua=mailto:dmarc@yourdomain.com; ruf=mailto:dmarc@yourdomain.com"

• p=quarantine: failing mail goes to spam
• pct=50: apply the policy to 50% of failing mail (gradual rollout)

Final state — reject:

_dmarc.yourdomain.com.  300  IN  TXT  "v=DMARC1; p=reject; rua=mailto:dmarc@yourdomain.com"

p=reject is the strongest protection and the best signal to Gmail that your domain is managed carefully. Don’t rush to this — wait until reports show 100% of your mail is passing SPF and DKIM.

Processing DMARC reports: The reports are aggregate XML files sent to your rua address. Tools like Parsedmarc or the web service DMARC Analyzer can parse and visualize them.

PTR Record (Reverse DNS)

Set this in your VPS provider’s control panel, not your DNS registrar. Find the “Reverse DNS” or “PTR” setting for your IP.

1.2.3.4  →  mail.yourdomain.com

Verify it’s propagated:

1
2
3
4
5

host 1.2.3.4
# Should return: 4.3.2.1.in-addr.arpa domain name pointer mail.yourdomain.com

dig -x 1.2.3.4 +short
# Should return: mail.yourdomain.com.

Testing Your DNS Records

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17

# Check MX records
dig MX yourdomain.com

# Check SPF
dig TXT yourdomain.com | grep spf

# Check DKIM
dig TXT mail._domainkey.yourdomain.com

# Check DMARC
dig TXT _dmarc.yourdomain.com

# Verify PTR
dig -x 1.2.3.4

# Full email health report
# Visit: https://mxtoolbox.com/emailhealth/yourdomain.com

BIMI — Brand Indicators for Message Identification

An optional record that causes Gmail and Apple Mail to display your logo next to messages. Requires p=quarantine or p=reject DMARC with full alignment, a publicly accessible SVG logo in the correct format, and (for Gmail) a Verified Mark Certificate (VMC) from a CA.

default._bimi.yourdomain.com.  TXT  "v=BIMI1; l=https://yourdomain.com/logo.svg"

For most self-hosters this is a nice-to-have rather than a priority. Get everything else right first.

Part 7: Deliverability — Getting Into the Inbox

This is the section you’ll return to the most. Everything up to here is configuration; deliverability is operations.

Why IP Reputation Is Everything

Gmail’s spam filtering doesn’t primarily evaluate message content — it evaluates sender reputation. A new IP with no reputation is treated like a stranger trying to walk into your house. Even with perfect DNS, perfect TLS, and perfect message content, a brand-new IP sending to Gmail has a high probability of hitting spam folders.

Reputation is built over time by:

• Sending mail that recipients interact with (open, reply, not mark as spam)
• Maintaining consistent sending volume
• Avoiding sudden volume spikes
• Having clean bounce handling
• Not appearing on any blacklists

IP Warming

When you start sending from a new IP, warm it gradually:

For a personal mail server where you’re not doing bulk sending, this happens organically over weeks of normal use. The key is to not send a large batch of mail immediately after setup — that pattern is exactly what spammers do.

Checking Blacklists

Before you do anything else, check your IP:

• MXToolbox Blacklist Check — checks 100+ blacklists simultaneously
• Spamhaus Blocklist Lookup — Spamhaus is the most impactful list
• Barracuda Central — Barracuda is widely used by enterprise mail

Common blacklists and removal:

If you’re on a blocklist before you’ve sent a single message, the previous owner of your IP was the problem. Request a new IP from your provider or contact the blacklist for removal.

Gmail Postmaster Tools

Register your domain at postmaster.google.com. This gives you:

• Domain reputation — High/Medium/Low/Bad, updated daily
• IP reputation — The reputation of the specific IPs sending your mail
• Spam rate — Percentage of your mail being marked as spam by Gmail users
• Authentication — SPF, DKIM, and DMARC pass rates
• Delivery errors — Breakdown of SMTP errors from Gmail’s inbound servers

The data appears once you’re sending enough mail to Gmail accounts. Even small volumes show up within a few days. This dashboard is your most important window into how Gmail sees you.

If your spam rate climbs above 0.1%, Google will start throttling you. Above 0.3%, you’ll be blocked. These thresholds are low.

Testing Your Setup

mail-tester.com — Send a test message to the address shown on the site, get a score out of 10 with detailed breakdown. A score of 9 or 10 means your technical configuration is solid.

1
2

# Send a test message
echo "Test message body" | mail -s "Test subject" test-xxxxx@mail-tester.com

MXToolbox Email Health — https://mxtoolbox.com/emailhealth/yourdomain.com — comprehensive DNS and configuration check.

Google Check MX — https://toolbox.googleapps.com/apps/checkmx/ — validates your configuration from Google’s perspective.

Sending a test to yourself on Gmail: The simplest real-world test. Send a message and check whether it lands in the inbox or spam. If it’s in spam, check the headers — there will be an X-Spam-Status or similar header with the reason.

The Gmail Problem

Even with a mail-tester.com score of 10/10, Gmail may spam-folder your mail initially. This is normal for new IPs. Strategies to build reputation with Gmail:

1. Exchange mail with Gmail users you know. Ask them to respond, and to move your mail to the inbox if it lands in spam. This positive engagement signal is the most effective reputation builder.
2. Have Gmail users add you to their contacts. Contacts rarely get spam-folded.
3. Don’t use spam trigger words during your warmup period. Avoid words like “free,” “win,” “click here,” “unsubscribe” in test messages.
4. Monitor Postmaster Tools obsessively during the first month.
5. Be patient. Reputation takes weeks to build and seconds to destroy.

The Microsoft 365 / Outlook Problem

Microsoft is often stricter than Gmail and provides less feedback. Key resources:

• SNDS (Smart Network Data Services) — sendersupport.olc.protection.outlook.com/snds: Check your IP’s reputation with Microsoft. Shows trap hit rates and complaint data.
• Junk Mail Reporting Program (JMRP) — Sign up to receive spam complaint reports when Outlook users click “Junk” on your mail.
• Microsoft Sender Support — If you’re blocked, submit a delisting request at https://sender.office.com.

If you’re sending to a corporate Outlook/Exchange environment, the receiving organization may have additional filtering (Defender for Office 365, third-party gateways) that you can’t influence from outside.

The SMTP Relay Option

The pragmatic solution for deliverability: keep your server for receiving mail, and route outbound through a reputable transactional email service.

# In Postfix main.cf — relay outbound through a service
relayhost = [smtp.mailgun.org]:587
smtp_sasl_auth_enable = yes
smtp_sasl_password_maps = hash:/etc/postfix/sasl_passwd
smtp_sasl_security_options = noanonymous
smtp_tls_security_level = encrypt

# /etc/postfix/sasl_passwd
[smtp.mailgun.org]:587    your_username:your_api_key

1
2

postmap /etc/postfix/sasl_passwd
chmod 600 /etc/postfix/sasl_passwd /etc/postfix/sasl_passwd.db

Transactional relay services:

With a relay, your domain’s reputation is backed by the relay service’s IP reputation, which is already warm. You lose some control but gain significantly better deliverability.

Part 8: Webmail

Most email clients (Thunderbird, Apple Mail, mobile apps) connect directly via IMAP. But a webmail interface is useful for access from shared computers and for users who prefer a browser-based workflow.

Roundcube

The classic, full-featured PHP webmail. Mature, well-maintained, and supports everything from basic mail reading to PGP encryption with plugins.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25

services:
  roundcube:
    image: roundcube/roundcubemail:latest
    container_name: roundcube
    restart: unless-stopped
    environment:
      - ROUNDCUBEMAIL_DEFAULT_HOST=ssl://mail.yourdomain.com
      - ROUNDCUBEMAIL_DEFAULT_PORT=993
      - ROUNDCUBEMAIL_SMTP_SERVER=tls://mail.yourdomain.com
      - ROUNDCUBEMAIL_SMTP_PORT=587
      - ROUNDCUBEMAIL_DB_TYPE=sqlite
      - ROUNDCUBEMAIL_DB_USER=roundcube
      - ROUNDCUBEMAIL_DB_PASSWORD=changeme
    volumes:
      - roundcube_db:/var/roundcube/db
      - roundcube_config:/var/roundcube/config
    labels:
      - "traefik.enable=true"
      - "traefik.http.routers.roundcube.rule=Host(`mail.yourdomain.com`)"
      - "traefik.http.routers.roundcube.tls.certresolver=letsencrypt"
      - "traefik.http.services.roundcube.loadbalancer.server.port=80"

volumes:
  roundcube_db:
  roundcube_config:

Snappymail

A modern, lightweight PHP webmail and spiritual successor to RainLoop. Faster and more visually polished than Roundcube, with lower resource usage.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14

services:
  snappymail:
    image: djmaze/snappymail:latest
    container_name: snappymail
    restart: unless-stopped
    volumes:
      - snappymail_data:/var/lib/snappymail
    labels:
      - "traefik.enable=true"
      - "traefik.http.routers.snappymail.rule=Host(`webmail.yourdomain.com`)"
      - "traefik.http.routers.snappymail.tls.certresolver=letsencrypt"

volumes:
  snappymail_data:

Configure the IMAP and SMTP server connections through Snappymail’s admin panel at https://webmail.yourdomain.com/?admin.

Stalwart’s Built-in Webmail

Stalwart ships with a basic webmail interface accessible at the same port as the admin UI. It’s functional and zero-configuration, but lacks the polish of Roundcube. Fine for personal use; deploy Roundcube or Snappymail if you have multiple users with varying needs.

Part 9: Security and Maintenance

A mail server that isn’t maintained actively becomes a liability. Here’s what ongoing operations actually looks like.

Fail2ban

Fail2ban watches your logs and bans IPs that attempt to brute-force authentication. Essential for any public mail server.

1

apt install fail2ban -y

/etc/fail2ban/jail.local:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23

[DEFAULT]
bantime = 3600
findtime = 600
maxretry = 5
backend = systemd

[postfix]
enabled = true
port    = smtp,465,submission
logpath = /var/log/mail.log
maxretry = 3

[postfix-sasl]
enabled = true
port    = smtp,465,submission,imap,imaps,pop3,pop3s
logpath = /var/log/mail.log
maxretry = 5

[dovecot]
enabled = true
port    = imaps,submission
logpath = /var/log/mail.log
maxretry = 5

1
2
3

systemctl enable fail2ban && systemctl start fail2ban
fail2ban-client status            # View all active jails
fail2ban-client status postfix    # View banned IPs for Postfix

TLS Only — Disable Unencrypted Connections

In Postfix, require STARTTLS for authenticated clients:

1
2
3

# main.cf
smtpd_tls_security_level = may        # Accept unauthenticated inbound (required for server-to-server)
smtpd_tls_auth_only = yes             # Require TLS before AUTH

In Dovecot:

# /etc/dovecot/conf.d/10-auth.conf
disable_plaintext_auth = yes

Monitoring

Your most important log:

1
2
3
4
5
6
7

tail -f /var/log/mail.log

# Common patterns to grep for:
grep "status=bounced" /var/log/mail.log     # Bounced messages
grep "status=deferred" /var/log/mail.log    # Messages in retry queue
grep "reject:" /var/log/mail.log            # Rejected inbound mail
grep "NOQUEUE: reject" /var/log/mail.log    # Rejected before queue

Check the outbound queue:

1
2
3
4

mailq                    # Show queue contents
postqueue -p             # Same as mailq
postqueue -f             # Force retry of deferred queue
postsuper -d ALL deferred  # Delete all deferred mail (use with care)

Prometheus metrics:

The postfix_exporter parses /var/log/mail.log and exposes Postfix metrics for Prometheus. Add to your existing Prometheus+Grafana stack:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10

services:
  postfix-exporter:
    image: prometheuscommunity/postfix-exporter:latest
    container_name: postfix-exporter
    command:
      - --postfix.logfile_path=/var/log/mail.log
    volumes:
      - /var/log/mail.log:/var/log/mail.log:ro
    ports:
      - "9154:9154"

Alert when queue size exceeds threshold:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12

# prometheus/alerts/mail.yml
groups:
  - name: mail
    rules:
      - alert: PostfixQueueSize
        expr: postfix_queue_size{queue="deferred"} > 50
        for: 15m
        labels:
          severity: warning
        annotations:
          summary: "Postfix deferred queue is growing"
          description: "{{ $value }} messages in deferred queue"

Backup Strategy

Mail data is irreplaceable. Back it up like you mean it.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12

# Back up Maildir with Restic to Backblaze B2
restic -r b2:bucket-name:mailbackup \
    --password-file /etc/restic/password \
    backup /var/mail/vhosts

# Back up configuration
restic -r b2:bucket-name:mailconfig \
    --password-file /etc/restic/password \
    backup /etc/postfix /etc/dovecot /etc/rspamd

# Verify backup integrity monthly
restic -r b2:bucket-name:mailbackup check

IMAP export for client-side backup:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11

# imapsync: copy one IMAP account to another or to local
apt install imapsync
imapsync \
    --host1 mail.yourdomain.com --ssl1 \
    --user1 admin@yourdomain.com --password1 yourpassword \
    --host2 backup.yourdomain.com --ssl2 \
    --user2 admin@yourdomain.com --password2 yourpassword

# offlineimap: sync IMAP to local Maildir
apt install offlineimap
# Then configure ~/.offlineimaprc with your account details

Managing Spam

Check the Rspamd web UI regularly during the first few months. Look for:

• False positives: Legitimate mail being flagged. Add to whitelist or train as ham.
• False negatives: Spam getting through. Train as spam.
• Score distribution: Understand where your thresholds are.

1
2
3
4
5
6

# Rspamd training from command line
rspamc learn_spam /path/to/spam.eml
rspamc learn_ham /path/to/legitimate.eml

# View Rspamd stats
rspamc stat

Part 10: The Pragmatic Middle Ground

Full self-hosting isn’t the only option. Here’s the complete decision matrix.

Option 1: Full Self-Hosted

Everything on your server. Maximum control, maximum maintenance burden.

Best for: Privacy absolutists, developers learning the stack, organizations with control requirements, homelabbers who enjoy operational complexity.

Effort: High setup, ongoing maintenance. Plan for ~2 hours/month of maintenance time.

Option 2: Self-Hosted Inbound + Relay Outbound

Your server receives mail (easy, reliable). An SMTP relay service like Mailgun, Brevo, or Amazon SES sends outbound mail (excellent deliverability, minimal cost).

Best for: Most people who want to self-host. You own your incoming mail, your domain, and your infrastructure. You outsource the hard deliverability problem to experts.

Effort: Moderate setup, minimal maintenance. Best cost/benefit ratio for self-hosting.

Option 3: Custom Domain on a Privacy-Respecting Provider

Use your own domain with Fastmail ($3/month), Migadu ($4/month flat rate for unlimited users), or Proton Mail (€8/month). Your domain, their infrastructure.

Best for: Users who want the benefits of their own domain — portability, no vendor lock-in on the domain itself — without any operational responsibility.

Effort: Minimal. Point your MX records, configure SPF/DKIM in their UI, done.

Option 4: Cloudflare Email Routing + Fastmail

Use Cloudflare Email Routing (free) to receive mail for your domain and forward it to a Fastmail address. Use Fastmail to send as your domain.

Best for: Zero-cost inbound routing with excellent reliability, plus a professional sending setup.

Effort: Very low. No servers to manage.

Recommendation Matrix

Wrapping Up

Self-hosting email is one of the most educational and rewarding infrastructure projects you can undertake. It will teach you more about how the internet actually works than almost anything else. The protocols are fascinating, the security considerations are real, and the satisfaction of watching a message you sent arrive at its destination via your own infrastructure is genuine.

It will also, at some point, put your mail in Gmail’s spam folder for reasons that are opaque, frustrating, and entirely outside your control. That’s not a reason not to do it — it’s just the reality you’re signing up for.

The path of least regret for most technically capable people is Option 2: self-hosted inbound with a relay for outbound. You own your mail, you control your domain, you maintain your server — and you outsource the part that’s hardest to get right on your own. If that feels like a compromise too far, Option 1 with docker-mailserver or Stalwart is a reasonable weekend project that will serve you well with ongoing attention.

Whatever you choose: get your DNS right first, check your IP before you invest time configuring it, and watch your logs.

Related posts: VPS Setup Best Practices, DNS Deep Dive, Traefik Complete Guide, Backup Strategy, WireGuard VPN