Posted inUbuntu

Building Your Own Mail Server with Postfix and Dovecot on Ubuntu 22.04: Take Full Control of Your Mail Flow

Ubuntu tutorial - IT technology blog
Ubuntu tutorial - IT technology blog

Why do many DevOps still choose to “torture themselves” by building their own Mail Server?

Instead of spending $6/user/month for Google Workspace, a $5/month VPS can handle hundreds of mail accounts for a startup. Previously, while managing systems for a financial firm, I realized that using third-party services causes scale-up costs to skyrocket. Building your own Mail Server not only saves budget but also gives you absolute control over logs and sensitive data flows.

However, this path is not paved with roses. If misconfigured, your server will quickly become a “spam machine” and be blacklisted globally overnight. Therefore, do it right from the very first step.

Choosing the right deployment method

There are three common directions on Ubuntu 22.04 that you can consider:

1. Using a Control Panel (CyberPanel, cPanel)

  • Pros: “Instant” installation, managed via an intuitive web interface.
  • Cons: Resource-heavy (usually requires an extra 500MB-1GB RAM for the panel) and very difficult to debug deeply when mail flow issues occur.

2. Deploying via Docker Mailserver

  • Pros: Excellent environment isolation, easy server migration in 5 minutes.
  • Cons: Docker network configuration is quite complex for those unfamiliar with iptables or container networking.

3. Manual Installation of Postfix + Dovecot (Our Choice)

  • Pros: Optimal performance, gain a deep understanding of how MTA and MDA communicate.
  • Cons: Takes about 30-60 minutes for the initial precise configuration.

We will go with the 3rd method. Once you understand how Postfix talks to Dovecot, you won’t fear any Relay access denied errors anymore.

Prerequisites for a stable system

To prevent this lab from “breaking” halfway, you need to prepare:

  • Ubuntu 22.04 VPS (minimum 2GB RAM to smoothly run virus/spam scanners later).
  • A domain name with an A record pointing to the server’s IP.
  • MX record (e.g., mail.itfromzero.com) pointing to the mail server domain.
  • Open Firewall ports: 25 (SMTP), 587 (Submission), 993 (IMAPS).

Step 1: Installing and Configuring Postfix (MTA)

Postfix acts as the Mail Transfer Agent (the mail carrier). On Ubuntu, the apt package manager includes a built-in wizard that makes the initial setup quite smooth.

sudo apt update
sudo apt install postfix -y

When the selection menu appears, choose Internet Site. For the System mail name, enter your main domain (e.g., itfromzero.com).

Next, open the main configuration file for fine-tuning:

sudo nano /etc/postfix/main.cf

Update the following parameters to ensure Postfix knows where to receive mail and how to authenticate:

myhostname = mail.itfromzero.com
myorigin = /etc/mailname
mydestination = $myhostname, itfromzero.com, localhost.localdomain, localhost
home_mailbox = Maildir/

# Connect to Dovecot for user authentication
smtpd_sasl_type = dovecot
smtpd_sasl_path = private/auth
smtpd_sasl_auth_enable = yes
smtpd_recipient_restrictions = permit_sasl_authenticated,permit_mynetworks,reject_unauth_destination

Step 2: Installing Dovecot (MDA)

If Postfix is the shipper delivering mail between cities, Dovecot is the warehouse manager keeping mail locally. It allows you to retrieve mail to your phone or computer via IMAP/POP3 protocols.

sudo apt install dovecot-imapd dovecot-pop3d -y

We will use the Maildir format. With this format, each email is a separate file, which helps prevent the entire mailbox from being corrupted if a file error occurs.

Edit the file /etc/dovecot/conf.d/10-mail.conf:

mail_location = maildir:~/Maildir

To allow Postfix to “ask” Dovecot for user authentication, open /etc/dovecot/conf.d/10-master.conf and edit the service auth section:

unix_listener /var/spool/postfix/private/auth {
  mode = 0660
  user = postfix
  group = postfix
}

Step 3: Data Encryption with SSL/TLS

Sending unencrypted mail is like sending a public postcard. Anyone on the transmission path can eavesdrop on your content. Use Let’s Encrypt to issue a free certificate.

Once you have the certificate, declare the paths in /etc/dovecot/conf.d/10-ssl.conf:

ssl = yes
ssl_cert = </etc/letsencrypt/live/mail.itfromzero.com/fullchain.pem
ssl_key = </etc/letsencrypt/live/mail.itfromzero.com/privkey.pem

Additionally, apply a similar configuration to Postfix in the main.cf file to enable TLS on port 587.

Step 4: Creating Users and Real-world Testing

This system uses Linux accounts as mail accounts. To add a new user, simply use the standard Ubuntu command:

sudo adduser hoang_devops
sudo systemctl restart postfix dovecot

Now, open Outlook or Thunderbird and enter the following parameters:

  • Incoming: mail.itfromzero.com (Port 993 – SSL/TLS)
  • Outgoing: mail.itfromzero.com (Port 587 – STARTTLS)
  • User/Pass: The account you just created above.

Hard-earned Experience: How to Keep Mail Out of Spam?

Many people set up their server only to find that 100% of outgoing mail ends up in Gmail’s Spam folder. To achieve a high Sender Score, you must configure the “DNS power trio”:

  1. SPF: Declares which IPs are allowed to send mail on behalf of the domain.
  2. DKIM: Digitally signs the mail to ensure content is not modified in transit.
  3. DMARC: Instructs the receiver on what to do if the mail fails SPF/DKIM.

Another important note is the PTR record (Reverse DNS). You need to request your VPS provider to configure the PTR to point the IP back to mail.itfromzero.com. Without this record, major mail servers like Yahoo or Outlook will often reject mail immediately.

Running your own Mail Server is a great experience to level up your system administration skills. Good luck with your configuration, and may you soon own your own “clean” mail system!

Share:
Tags: