How to Build a Bulletproof Email Server with Proxmox Mail Gateway

Author:

 


What Proxmox Mail Gateway Is (and Isn’t)

Proxmox Mail Gateway is an open‑source email security gateway that filters incoming and outgoing email traffic before it reaches your actual mail server. It focuses on:

Spam and virus protection✔ Anti‑phishing and malware filtering
SMTP policy enforcement (SPF/DKIM/DMARC, DNSBL, greylisting)
Detailed message tracking and logging
High availability and clustering for resilience (Proxmox)

Important: PMG is not a mail store or IMAP/POP3 server — it acts as a gateway/security proxy in front of a real mail server (like Postfix, Mailcow, Exchange, etc.) (Proxmox)


1. Plan Your Email Architecture (Foundation)

A secure and bulletproof email setup generally uses:

Network Flow

Internet → Firewall → Proxmox Mail Gateway → Internal Mail Server → Users
  • Firewall filters ports before PMG (only allow SMTP/SMTPS etc.)
  • PMG handles all SMTP traffic first (spam/virus/DMARC/SPF)
  • Internal mail server (e.g., Dovecot/Postfix, Mailcow, Exchange) stores/serves mail

This setup ensures malicious traffic never touches your internal mail system. (TomskSoft)


2. Install Proxmox Mail Gateway

System Requirements

  • 64‑bit CPU (Intel VT/AMD‑V)
  • Minimum 4 GB RAM
  • 10 GB+ disk space (SSD preferred)
  • Network interface for incoming SMTP traffic (Unihost)

Installation Steps

  1. Download the ISO from Proxmox. (Proxmox)
  2. Boot the PMG installer on a dedicated server or VM.
  3. Follow the installer wizard — choose disk, timezone, hostname, etc. (Proxmox Mail Gateway)
  4. After install, access the web UI at https://<PMG-IP>:8006. (Proxmox Mail Gateway)
  5. Upload your subscription key (optional but recommended for stable updates). (Proxmox Mail Gateway)

 Alternative: Install on Debian

Advanced users can install via apt on a Debian host — useful in complex environments. (Unihost)


3. Configure Core Mail Proxying

Mail Proxy Setup

  • In the web UI: Configuration → Mail Proxy
  • Default Relay: your internal mail server (e.g., mail.yourdomain.com)
  • Relay Domains: specify which domains PMG should accept for (Proxmox Mail Gateway)

This makes PMG the SMTP endpoint for your domains.

Reddit tip: If you have multiple PMG instances (e.g., MX1/MX2), set them in a cluster and add their IPs to trusted networks to avoid SPF issues. (Reddit)


4. Harden Filtering & Security Features

PMG has powerful built‑in protections:

Anti‑Spam & Virus

  • Uses SpamAssassin + DNSBL + SURBL + greylisting to reject spam before delivery. (Proxmox)
  • ClamAV scans for viruses and malware. (Proxmox)

Spam Hardening Tips

Enable greylisting to block many spam sources before reaching your mail server.
Use public DNSBLs (e.g., Spamhaus, Barracuda).
Tune SpamAssassin thresholds to balance false positives.
Create whitelists/blacklists for trusted senders or unwanted domains. (Proxmox)


5. Enforce Email Authentication (SPF/DKIM/DMARC)

SPF

  • Ensure your domain’s DNS has a proper SPF record.
  • PMG will check incoming mail against it and reject spoofed senders. (Proxmox)

DKIM Signing

  • PMG can sign outgoing mail with DKIM keys to improve reputation. (Proxmox)

DMARC

  • Use DMARC enforcement in DNS to tell receivers how to treat mail that fails checks.

These policies protect against spoofing and phishing.


6. Encrypt SMTP with TLS

Configure TLS policies so that SMTP sessions are encrypted:

  • PMG supports STARTTLS by default.
  • You can enforce per‑destination TLS policies for stronger security. (Proxmox Mail Gateway)

This protects mail in transit and avoids retrograde plaintext delivery.


7. Logging, Monitoring & Tracking

Message Tracking Center

PMG’s built‑in Message Tracking lets you search logs, view processing steps, and diagnose delivery problems — critical for debugging and compliance. (Proxmox)


8. High Availability (HA) & Clustering

For bulletproof uptime and failover:

  • Create an HA cluster with multiple PMG nodes.
  • Use round‑robin MX records pointing to two or more PMG servers.
  • Configure synchronized rules and logs across nodes. (Proxmox)

This setup ensures your gateway stays up even if a node fails.


9. Harden the Server & OS

 Basic Hardening

Use strong passwords for admin accounts. (Proxmox Mail Gateway)
Restrict access to the admin web UI (firewall, VPN, IP allow list).
Ensure only necessary ports (SMTP/SMTPS, web UI) are exposed.
Regularly update via the enterprise repository (subscriptons help). (Proxmox Mail Gateway)

 Regular Backups

  • Backup PMG configs and rule sets to recover quickly from failure.

10. Real‑World Tips & Pitfalls (Community Observations)

Integration quirks

  • Users note that correct SMTP port mapping and internal relay configurations are crucial — misconfigurations can lead to delivery issues. (Reddit)

Trusted Networks

  • If you have multiple gateways (for multiple domains), add all your PMG and mail server IPs to trusted networks to avoid SPF mismatches. (Reddit)

Spam Training

  • Some admins report that manual marking of quarantined mail and regular tuning gives the best results over time for filtering accuracy. (Reddit)

What Proxmox Mail Gateway Does Not Replace

While PMG is powerful, you still need:

A Mailbox server (Postfix/Dovecot, Exchange, Mailcow) – PMG doesn’t store or serve mail to users. (Reddit)
A User authentication & access layer (IMAP/POP3, SSO).
DNS records (MX, SPF, DKIM, DMARC) configured properly for your domains.


Summary Checklist: Bulletproof Email with PMG

Install PMG on dedicated hardware or VM.
Position it before your mail server in SMTP flow.
Configure mail proxy relays correctly.
Enable SPF/DKIM/DMARC enforcement.
Tune spam/virus filters and DNSBLs.
Enable TLS policies for encrypted SMTP.
Use message tracking and logging for oversight.
Harden the system and secure the admin UI.
Build HA clusters and MX redundancy for resilience.


Here’s a case‑study and commentary‑rich overview of how organisations and administrators are using Proxmox Mail Gateway (PMG) to build robust, secure email infrastructures, including real deployments, lessons learned, pros/cons, and community feedback.


What PMG Does

Proxmox Mail Gateway is an open‑source email security solution deployed between the internet/firewall and your internal mail server. It acts as an SMTP proxy that filters spam, viruses, phishing, and other threats before they reach your mail store, and helps ensure outbound email complies with authentication standards like SPF/DKIM/DMARC. (Proxmox)


Case Study 1 — Bin Drai Enterprises (Enterprise Deployment)

Challenge

Bin Drai Enterprises, a real estate firm with 100+ users, struggled with high volumes of spam, phishing, malware, and ransomware delivered to their internal mail server (Zimbra). (Proxmox)

Implementation

  • Deployed PMG as a VM in front of their existing mail server.
  • Configured SMTP relay domains, DNS (MX and reverse DNS), TLS encryption, and spam/virus filtering rules via the PMG web interface.
  • Changed public DNS MX records to point to the PMG instance so only filtered email reached Zimbra. (Proxmox)

Results

Spam and malicious messages dropped significantly before reaching users.
Real‑time threat protection improved operational efficiency.
The web UI made monitoring and custom rule creation accessible.
Ongoing updates helped reduce spam over time. (Proxmox)

Commentary: The IT manager highlighted that PMG delivered enterprise‑class filtering without licensing costs, and offered stable, scalable protection with full control and logging. (Proxmox)


Case Study 2 — University of Macau (High‑Load Academic Environment)

Before PMG

The university used a legacy setup (Postfix + PolicyD) and was unsure whether PMG could handle high throughput and complex filtering needs. (Proxmox)

After Deployment

  • Deployed three PMG VMs in a cluster for load balancing and high availability.
  • Wanted to handle daily bulletin mail blasts (thousands of messages) and normal campus mail without degradation.
  • PMG’s flexible rule engine allowed custom filtering policies that matched varied traffic types. (Proxmox)

Lessons & Feedback

Easy to integrate with existing infrastructure.
Rule‑based filtering was intuitive, powerful, and flexible.
Performance scaled well with a clustered PMG setup. (Proxmox)

Comment: The Univ. of Macau team appreciated that PMG could be tested in a sandbox before full deployment and offered transparent, granular control over filtering behaviors. (Proxmox)


Community & Admin Commentary

Positive Experiences

  • Many admins report that PMG “catches tonnes of spam out of the box”, even on multi‑domain environments, with minimal tuning. (Reddit)
  • Some use it as an outbound relay to protect IP reputation, helping avoid blacklisting from compromised servers. (Reddit)
  • PMG’s rule engine and filtering modules (SpamAssassin, ClamAV, DNSBL/SURBL) are cited as effective and flexible. (Proxmox)

Challenges & Things to Watch

  • Rule logic can be confusing initially, especially for complex allow/block patterns — e.g., strict whitelisting without bypassing spam checks isn’t always straightforward. (Reddit)
  • Performance quirks can occur (e.g., slow loading in the Message Tracking Center when logs grow large), which may require log housekeeping. (Reddit)
  • DKIM/DMARC handling quirks have been reported when sending autogenerated reports or bounce messages — highlighting the importance of proper SPF/DKIM config on all endpoints. (Reddit)
  • Some admins noted edge cases where rejected/non‑delivery reports behaved differently when SG checks were strict, especially with services like Gmail that enforce authentication. (Reddit)

Lessons & Best Practices from Deployments

Architecture & Clustering

  • For high‑availability or load‑balanced environments, run multiple PMG nodes and configure equal‑priority MX records, so traffic is spread and resilient. (Proxmox Mail Gateway)
  • Clusters can also synchronize configs and ensure continuity if a node fails. (Proxmox Mail Gateway)

Filtering Strategy

  • Use the object‑oriented rule engine to combine sender, recipient, time, and content criteria for targeted actions. (Proxmox Mail Gateway)
  • Implement greylisting, SPF, DNSBL/SURBL, and virus policies at the gateway to stop threats early and reduce load on the mail server. (Proxmox)
  • Tune thresholds gradually; overly aggressive settings may block legitimate mail.

Monitoring & Logging

  • Regularly review the Tracking Center for patterns and anomalies.
  • Archive logs for audit and troubleshooting, especially after mass rule changes.

Integration Tips

  • Ensure correct DNS configuration — particularly MX, PTR (reverse DNS), SPF, and DKIM — so gateways and receiving servers align on authentication. (Proxmox)
  • When deploying with internal mail systems (e.g., Exchange, Zimbra, Postfix), adjust trusted networks and relay settings to maintain deliverability and authentication integrity.

How PMG Helps in Real Deployments

Aspect Benefit in Case Studies / Deployments
Spam & Virus Filtering Significant reduction in unwanted mail early in the flow, reducing load and risk. (Proxmox)
Custom Rule Engine Allows tailored policies for domains, users, and content types. (Proxmox Mail Gateway)
Clustering/HA Enables resilient deployments for universities and enterprises. (Proxmox)
Outbound Protection Helps protect IP reputation by centralizing mail egress (community reports). (Reddit)
Open‑Source & Cost‑Effective Enterprise‑grade capabilities without proprietary licensing costs. (Proxmox)

Overall Commentary & Takeaways

Strengths

  • PMG is a powerful open‑source filter that can rival commercial email gateways for spam and malware protection. (Proxmox)
  • Deploys easily in front of existing mail servers and scales from small businesses to universities. (Proxmox)
  • The flexible rule system and integrated tools (SpamAssassin, ClamAV, DNSBL/SURBL) give admin control and visibility over mail flows. (Proxmox Mail Gateway)

Limitations

  • Some advanced use cases can require careful tuning and understanding of SMTP/relay behavior. (community reports) (Reddit)
  • Wireless or lightweight container setups may require additional troubleshooting for performance or UI access (reddit feedback). (Reddit)

Community Voice

Admins using PMG for years affirm its value — especially as a cost‑effective and configurable security layer in email infrastructure. Some also highlight that while it’s very capable, hybrid approaches (e.g., combined with cloud SMTP services for outbound reputation management) can be optimal for some environments. (Reddit)


In Summary

Proxmox Mail Gateway has been used successfully by organisations of various sizes to secure email traffic, reduce spam, protect internal mailboxes, and improve overall mail hygiene. Real deployments show that:

It can stop threats before they hit mail servers. (Proxmox)
It scales with clustering for resilience. (Proxmox)
Flexible rule systems allow granular filtering. (Proxmox Mail Gateway)
Community feedback highlights both strengths and occasional challenges. (Reddit)