Setting up a mail server with OpenSMTPD, Dovecot and Rspamd · poolp.org

[poolpOrg]

[leo-unglaub]

Awesome article, thanks for all your work!

[mchack23]

neat, been waiting for this article for some days now since I read your "mail's not hard" article just when I was trying to set up an openbsd smtpd server myself.
Since I'm still flailing a bit (it is working but I managed to create local mail loops for instance), I'll try and recreate and tweak your setup next :) Hints on how to set up virtual users are most welcome (and how I can relay local user mail to those virtual users)

Anyway: Found a small typo it should say options.inc (not options.conf) in this line: "This is done by adding the dkim_signing filter to the filters key configuration in /etc/rspamd/options.conf:"

When I find more I'll edit this post. :) Thanks again Gilles!

[poolpOrg]

@mchack23 typo fixed

the source markdown for the articles are in this repository, feel free to submit pull requests for typos and such :-)

[myfirstnameispaul]

I have been using Vultr 1GB VPS for about a year with a LEMP server spitting out transactional mail without issue. A few months ago I moved my MiaB over to the same service and haven't had any real issues, never submitted a ticket for any sort of filtering. Are you using a European DC?

Have you found dnswl.org to be useful? I see a lot of the big guys are there.

I look forward to your article on Microsoft because I'm not sure it can be done, as well as the DKIM key size as this is the first I've been aware of this.

While I originally taught myself how to configure my own mail server, I moved to MiaB in order on a simple VPS because as a personal mail server, it's fantastic and trouble-free, as much as a mail server can be.

However, I am wondering if you ever cover capacity limits? For some of the organizations I think you are targeting, they may quickly max out a VPS so capacity planning seems like a very relevant topic. What is considered for CPU limits in a mail server? What is using up memory and how painful is swap? How much storage should I plan on using, especially when forecasting growth rates? Are there some rules of thumb for any of these? Can I share block storage with more than one server? Reverse proxy? Etc...

I have also wondered about IPv6. What happens when spammers can just use a new IP address for every email and blast you a million times a second into eternity? Can I get by with just IPv4?

[benwaffle]

If reading about how you can build such setups is of any interest, I could also write about it to help people build MORE hosting alternatives which is ultimately what I hope for.

Yes, I'm interested.

[superkuh]

The hard part is 5 years later when your current mailserver's OS goes end of life and you have to port everything to a new setup using different versions of things.

[tbtb]

I think, that's an ongoing project which is supposed to adapt to any future s/w changes or updates

[cmouse]

I'd like to suggest incorporating this in your howto:

OpenSMTPD could use LMTP (or execute dovecot-lda, not sure which works in OpenSTMPD) to deliver mail via dovecot, the benefit is that then dovecot updates it's indexes immediately, and you can benefit from sieve filtering for incoming mail. Then you can make dovecot filter e.g. spam messages to Spam folder directly.

[ynakao]

Thanks for the great writeup. As @benwaffle mentioned, I'm also interested in large-scale email server setup especially about virtual users and multiple domain managements.

I will write instructions assuming the upcoming 6.6.0 release which is due in a few weeks now. It brings a ton of improvements and I don’t want the instructions to be obsoleted and need a rewrite in October.

Out of curiosity, which is the OpenSMTPD v6.6.0 specific settings? Will there be similar breaking changes on OpenSMTPD in the future?

[omoerbeek]

PR #18, I think the flag does not match the text.

[poolpOrg]

@myfirstnameispaul

I have been using Vultr 1GB VPS for about a year with a LEMP server spitting out transactional mail without issue. A few months ago I moved my MiaB over to the same service and haven't had any real issues, never submitted a ticket for any sort of filtering. Are you using a European DC?

For my own needs, I'm using multiple DC (Amsterdam, Frankfurt, London, Paris) all in Europe yes.

Have you found dnswl.org to be useful? I see a lot of the big guys are there.

In all honesty no, I know dnswl.org but I can't even recall if I ever subscribed mine, so they may be useful but you can certainly do without.

I'm 100% positive that I didn't subscribe most of the domains I plugged.

However, I am wondering if you ever cover capacity limits? For some of the organizations I think you are targeting, they may quickly max out a VPS so capacity planning seems like a very relevant topic.
What is considered for CPU limits in a mail server?
What is using up memory and how painful is swap?
How much storage should I plan on using, especially when forecasting growth rates?

I don't because it really depends on what you are going after.

I documented a very simple setup which would work fine for a lot of people on the cheapest VPS, because that's how I test these setups and even load them a little, but how people need to handle capacity planning depends on many factors.

On my own setup, Rspamd instances run on machines of their own so I can adjust them depending on CPU and RAM usage, whereas Dovecot runs on a separate machine with decent disks, whereas my OpenSMTPD instances are basically CPU-bound, delivering the messages to a remote machine and not the local disk. My planning here is per-service and since services are per-machine, this is something I'd expect a company to do but not a lambda user, my recipes won't work for them.

Then, even on a simpler setup, some users will want MariaDB to handle their accounts, others will want an elk stack to monitor, others will want a webmail and maybe some fancy log analysers, and depending on where you run these, it changes radically how you do your planning. A machine could run very fine with your mail infra and load like hell just with elastic installed, so you might need to go for a real dedicated server or much higher-frequency CPUs just for one of the components.

I think capacity planning is not very different with mail than with any other service and, as far as I'm concerned, it goes by graphing and monitoring how things evolve in time to adjust, I don't have a secret works-for-all recipe. Some more competent SREs surely have better solutions though.

A very cheap VPS will do as they are more powerful than most mail servers from the last decade and the software have not grown more hungry in resources (only speaking about opensource solutions, not the commercial ones which will plain not work on a VPS).

For Rspamd, I have no idea what the minimal resources are, but I had never had a problem running it on the cheapest VPS I could get my hands on. It mostly does CPU, that's about all.

For Dovecot, this is VERY dependant on how many users you have, how they consume their mail, if they search mailboxes and such, so again I had never had a problem running on the cheapest VPS BUT your mileage may vary and graphing resources consumption to adjust is key. It mostly does CPU and DISK but is also mostly idle if not on a busy server.

For OpenSMTPD, not only the cheapest VPS will do but SMTP servers are mostly idle, so testing the limits is quite difficult. It can run (not just build but actually run) on very cheap machines, people use Raspberry boards, as well as on some ancient or not-so-powerful architectures supported by OpenBSD (https://www.openbsd.org/plat.html). It does CPU and DISK but it can also deliver remotely (what I do) in which case it mostly does CPU when it's not idle, even on a relatively busy server.

A lot of people thing that a lot of CPU is needed for all the TLS handling but this is not correct, I could share stories about that ;-p

I don't observe variations in terms of memory usage across time and on a server that has been running for a while, the memory consumption of just mapping the Go runtime is an order of magnitude higher than the memory consumption of any process from Dovecot or OpenSMTPD for instance.

When looking into how good I'm with resources, my main concern is if I should add disk or not.

Are there some rules of thumb for any of these? Can I share block storage with more than one server? Reverse proxy? Etc...

The sharing of block storage is a tricky part and depends on what system you're using.
If using Maildir backend, you're supposedly safe when sharing block storage because Dovecot keeps its state internal to a Maildir, and OpenSMTPD delivers atomically and lockless inside Maildir.
Then questions arise if you're not only sharing block storage but also clustering / cloning the SMTP instances or the IMAP instances. Both are technically doable, not necessarily in a very tricky way, but I would not say it's easy.

Reverse proxying is simple in both cases, both Dovecot and OpenSMTPD can support proxy-v2 and as a matter of fact I'm using an haproxy myself.

I have also wondered about IPv6. What happens when spammers can just use a new IP address for every email and blast you a million times a second into eternity? Can I get by with just IPv4?

That's a problem.

But since it's not globally solved by Big Mailer Corps, IPv6 remains insignificant when it comes to mail as of today and you can certainly just get by with IPv4.

[poolpOrg]

@cmouse

I'd like to suggest incorporating this in your howto:

OpenSMTPD could use LMTP (or execute dovecot-lda, not sure which works in OpenSTMPD) to deliver mail via dovecot, the benefit is that then dovecot updates it's indexes immediately, and you can benefit from sieve filtering for incoming mail. Then you can make dovecot filter e.g. spam messages to Spam folder directly.

I'd rather see people write such howtos ;-)

[poolpOrg]

@ynakao

Thanks for the great writeup. As @benwaffle mentioned, I'm also interested in large-scale email server setup especially about virtual users and multiple domain managements.

I will write instructions assuming the upcoming 6.6.0 release which is due in a few weeks now. It brings a ton of improvements and I don’t want the instructions to be obsoleted and need a rewrite in October.

Out of curiosity, which is the OpenSMTPD v6.6.0 specific settings?

All the filter stuff comes with v6.6.0 so while you could have rspamd, dkim and such, it was done through a slighly trickier method of "queue reinjection" where you tell OpenSMTPD to relay to an Rspamd proxy or a DKIM proxy and you tell the proxy to re-enqueue in the daemon.

It was not necessarily much more complex in terms of configuration but it was a bit more complex in terms of understanding how it works and far less elegant than just adding a filter keyword to a listen line :-)

Will there be similar breaking changes on OpenSMTPD in the future?

In this case the change is not breaking, the configurations that worked before would still work, but they would be far less elegant and efficient at solving the issue, or would even be missing some of the improvements.

That being said, OpenSMTPD is a very active project with a release cycle of six months and we'll have breaking changes every now and then because we'd rather have users change a keyword than let technical debt accumulate.

We have had a very big breaking change a year ago where the configuration file was completely revamped, this was the first time since the project was first started ten years ago, so a huge breaking change should not happen before another decade :-) but it's likely that within the next year, we'll change a keyword here or there to help improve the project.

[cmouse]

I'd like to suggest incorporating this in your howto:

OpenSMTPD could use LMTP (or execute dovecot-lda, not sure which works in OpenSTMPD) to deliver mail via dovecot, the benefit is that then dovecot updates it's indexes immediately, and you can benefit from sieve filtering for incoming mail. Then you can make dovecot filter e.g. spam messages to Spam folder directly.

I'd rather see people write such howtos ;-)

Perhaps. But I think it would make sense to involve dovecot on the delivery phase always, as it would enable much more functionality in dovecot and would allow dovecot to update it's indexes during delivery.

[jirib]

My only concern for HA mail setup are mails waiting in a queue to be delivered, eg. a mail server in DMZ with paused delivery aka "deffered relaying" (cf 9.2.1 Postponing Mail Delivery in Postfix: Definitive Guide; would that be doable with smtpd -P mta?) and which waits for a manual trigger to try to deliver via SMTP to a mail server in intranet. If the mail server with this setup would fail, the mails will be lost. IIRC 'extras' used to allow Python plugins which could save mails into redis etc. to have a queue "distributed"...

[epsilon-0]

sweet as fuck!

[yf4Yeh4ZLt89lT]

I solved the SPF and DMARC complain by sending e-mail with the command :
"mail -r [email protected] -s test [email protected]"

For information, before, there was no "-r [email protected]".

But the missing DKIM signature is still there :-(

[netstx]

Is your DKIM record correct? If you copy it from the console (cat /etc/mail/dkim/domain.com.pub for example) it will have line breaks, so I'd take a look at that.

[yf4Yeh4ZLt89lT]

I finally understood that the messages are signed only when sent from a an external client that authenticate itself first.

By curiosity, what is the configuration to get DKIM signed messages when sent locally (directly from the server) ?

[yf4Yeh4ZLt89lT]

I tried to ad to the configuration the part "learn to Dovecot to train rspamd" and it does not work.

It seems the pigeonhole is not compatible : on OpenBSD 6.8, once I installed the official package "dovecot-pigeonhole-0.5.11v1" I can not restart dovecot.
I get this error message :

root@mx1 [11:27:17]:~ # dovecot -F
doveconf: Error: Couldn't load plugin /usr/local/lib/dovecot/settings/libmanagesieve_login_settings.so: Module is for different ABI version 2.3.ABIv11(2.3.11.3) (we have 2.3.ABIv13(2.3.13))
doveconf: Error: Couldn't load plugin /usr/local/lib/dovecot/settings/libmanagesieve_settings.so: Module is for different ABI version 2.3.ABIv11(2.3.11.3) (we have 2.3.ABIv13(2.3.13))
doveconf: Error: Couldn't load plugin /usr/local/lib/dovecot/settings/libpigeonhole_settings.so: Module is for different ABI version 2.3.ABIv11(2.3.11.3) (we have 2.3.ABIv13(2.3.13))
doveconf: Warning: NOTE: You can get a new clean config file with: doveconf -Pn > dovecot-new.conf
doveconf: Warning: Obsolete setting in /etc/dovecot/dovecot.conf:25: 'imaps' protocol can no longer be specified (use protocols=imap). to disable non-ssl imap, use service imap-login { inet_listener imap { port=0 } }
doveconf: Fatal: Error in configuration file /etc/dovecot/dovecot.conf: protocols: Unknown protocol: sieve

I can restart Dovecot normally after deleting the package.

Edit on 06th May :

After reinstalled the server with OpenBSD v6.9, the issue disappeared.
I'am waiting incoming spam to check the learning feature does work.

[netstx]

Just wanted to share that I have followed this guide both on 6.8-stable and 6.9-stable and it worked really well.

[ghost]

Weirdly enough, i followed this guide on 6.9-stable, i can log-in from gmail but not send/receive mail.

[ghost]

@hagnert, yeah same for me (mutt login). I can actually receive mail, but I'm unable to send any. Been getting following error in my logs:

SSL failed: error:1408F10B:SSL routines:ssl3_get_record:wrong version number

I configured mutt to use the port 587 for the smtp_url. Everything else end up in a straight "Connection refused". So I assume thats perhaps the right port?

I have to admit that I didn't search for to long, but would be very thankful for any helpful advice.

EDIT:

Perhaps it's some issue related to my client. Since Client is a GNU/Linux Gentoo (OpenSSL) and Server OpenBSD (LibreSSL). I don't really understand to much of the nuances between these SSL standards, but I could add a compliation flag on my clients openSSL, which might resolve the issue:

 - - abi_x86_32    : 32-bit (x86) libraries
 + + asm           : Support assembly hand optimized crypto functions (i.e. faster run time)
 - - bindist       : Disable/Restrict EC algorithms (as they seem to be patented) -- note: changes the ABI
 - - rfc3779       : Enable support for RFC 3779 (X.509 Extensions for IP Addresses and AS Identifiers)
 - - sctp          : Support for Stream Control Transmission Protocol
 - - sslv3         : Support for the old/insecure SSLv3 protocol -- note: not required for TLS/https
 - - static-libs   : Build static versions of dynamic libraries as well
 - - test          : Enable dependencies and/or preparations necessary to run tests (usually controlled by FEATURES=test but can be toggled
                     independently)
 - - tls-heartbeat : Enable the Heartbeat Extension in TLS and DTLS
 - - vanilla       : Do not add extra patches which change default behaviour; DO NOT USE THIS ON A GLOBAL SCALE as the severity of the
                     meaning changes drastically
 + + zlib          : Add support for zlib (de)compression

But since a recompilation of OpenSSL might force me to recompile some of my other programs, I'm searching for another reason in the meantime... Especially since I hope it's just a typo somewhere on the server and I've read that SSLv3 seems to have some security issues (?). Is SSLv3 perhaps demanded if you want to connect to a OpenBSD server?

[hakova]

FWIW, I used this guide to setup my email server on OpenBSD 6.7 I believe and upgraded the system twice since then. Neither of those upgrades broke the email server.

…

On 21/05/22 07:44AM, whateverthisisafasf wrote: @hagnert, yeah same for me (mutt login). Been getting following error in my logs: `SSL failed: error:1408F10B:SSL routines:ssl3_get_record:wrong version number` I have to admit that I didn't search for to long, but would be very thankful for any helpful advice. -- You are receiving this because you were mentioned. Reply to this email directly or view it on GitHub: #17 (comment)

[RennisDitchie]

@poolpOrg

I read some of the previous posts on this thread, and though this might come across as a support issue, I think the following is worthwhile mentioning since its an issue with the midpoint of the blog post itself, so it kind of teeters towards something that the blog author could clarify.

To begin, I wanted to add the following points:

• Thanks for the post itself, seems pretty decent, and was straight forward up to this point.
• I followed the points up to a T, so I am fairly certain I did not miss anything.

Within the "Installing and configuring Rspamd" section, you listed the following command:
mail$ doas pkg_add redis rspamd opensmtpd-filter-rspamd

However, when I have done this, you are forced to utilize the 'doas' configuration file before you're allowed to do this. I would ask if you could update your blog post to reflect this because this wasn't mentioned. This might seem straight forward to someone experienced with this, but keep in mind, most people like myself use Linux for this kind of thing, so even using OpenBSD for this whole process is absolutely new to me.

So, anyway, I basically copied the standard 'doas' configuration file into /etc/ with the following command:
cp doas.conf /etc/doas.conf

I then proceeded onward with the remaining sections of that post.

The part that I'm currently stuck on is "Also make sure the dkim key file is readable by members of the _rspamd group." instruction.

The problem with this directive is that when I use the 'groups' command, I get the following output:
wheel kmem sys tty operator staff guest

As you can see, I don't see an '_rspamd' group at all.

I'm assuming that your 'doas' command was supposed to perhaps install rspamd as its own user or group (not sure, totally new to all this).

I did a bit of research, and if you want to make a file readable by a group, then the related permissions are '040' (https://kb.iu.edu/d/abdb)

However, I'm not sure what exactly to do in this scenario for the '/etc/mail/dkim/(mydomainname).key' file itself's permissions since the '_rspamd' group doesn't clearly exist on the OpenBSD VM I'm running on Vultr.

This is more apparent since when I try to start the 'redis' and 'rspamd' processes with the 'rcctl' commands, I get the following output:

# rcctl start redis
redis(ok)
# rcctl start rspamd
rspamd(failed)

Any help on this?

Thanks!

[tmontney]

Any suggestions on incorporating multi-factor authentication? One of the reasons I moved away from my previous provider was because they charged extra for it.

[ffuentese]

How do I use this setup for more than a domain identical to the mail server? I'm working on a small mail service for friends and me but neither this one nor the Vultr tutorial explain how exactly mutliple domains are managed.

[ffuentese]

Well, I managed to do it creating a table of domains but it'll be great if you could share what you did and leave it documented here. Though I did find some guide, I think it's an old one at prefetch.eu

[mateors]

@ffuentese Unfortunately I don't have any documentation, I did it by myself. If you want we can meet using google meet.

[myfirstnameispaul]

Maybe GitHub isn't the best tool for blog comments, because subscription can only be to every comment, not just to threads, so if you glance up there at the number of people subscribed, you can assume a pretty good percentage of them are getting this in an inbox that probably matters, since GitHub also offers coarse delivery options.

[poolpOrg]

I enabled discussions so feel free to create a dedicated topic 😊

[ffuentese]

@ffuentese Unfortunately I don't have any documentation, I did it by myself. If you want we can meet using google meet.

Just share what you know

[wravoc]

Latest updates for April 2023

With the latest OpenBSD 7.3 and Rspamd 3.4 running on Vultr VPS, with commiserate changes in a hosted environment to /etc/resolv.conf, myname, mygate, and hosts, Rspamd would die unexpectedly.

Being new to all this it took quite some troubleshooting to find primary line producing the error which was:

Change /etc/rspamd/worker-proxy.inc hosts = localhost to

upstream "local" { default = yes; hosts = "mail.hypno.cat"; }

Or when using SSL wildcard or subdomain routing just hypno.cat

This will fix errors:
failed to receive a response from daemon
smtp failed-command command="DATA" result="421 server internal error"
rspamd_inet_address_listen: bind

I hope it helps and if you wanna try this tut on Vultr for only $6/mo you can gain a $100 hosting credit by clicking here

Excellent article, thanks! I will donate https://www.paypal.com/paypalme/poolpOrg

[wravoc]

Oh and make sure pf.conf isn't the popular security config

pass in on lo0 all pass out on lo0 all

but the starter

set skip on lo

[wravoc]

Securing IMAP

Another thing! It's been hours and hours staring this down...

/etc/dovecot/conf.d/10-ssl.conf

Change ssl=no to ssl=required or your BSD account password will go over the internet in plain text!

https://doc.dovecot.org/configuration_manual/dovecot_ssl_configuration/

[lucarinaorg]

Thanky you!!!
Perfectly worked on Vultr OpenBSD 7.4
Can you provide more info in the case of multi domain name?
I found this:
https://undeadly.org/cgi?action=article&sid=20130130081741
and
https://github.com/Excision-Mail/Excision-Mail/blob/develop/roles/smtpd/templates/smtpd.conf.j2

but your touch will be of added value
thank you again

[isniz]

@hakova: You are welcome. I did not adjust rDNS settings of my email server's IP address. It did not create any problems. Domains have an MX records pointing to the email server's FQDN anyway.

[isniz]

Step 5 could be more elegant. Instead of adding a new match from any for domain line, domains could be listed like that:
match from any for domain { "hypno.cat", "abc.de" }

Changed for myself and using since.

[hakova]

@isniz: Thank you for your additional comments. I guess what I was trying to ask was the following:
In the above example, the two domains served are hypno.cat and abc.de. Is the rDNS setting point to hypno.cat or abc.de, or something totally different (say mail.example.com) in your case? Moreover, when you are setting up the email client, what would be the IMAP and SMTP settings you would use for each domain? hypno.cat and abc.de accordingly, or a common mail.example.com for both?
I hope I could explain myself better this time.

[isniz]

@hakova: In my case it is one of two domains that rDNS is pointed to. But it could be something totally different, say mail.example.com. When you are setting up the email client, IMAP and SMTP settings should be pointed to server's FQDN, mail.example.com in our case.

[ozgurkazancci]

@isniz: Thank you for your additional comments. I guess what I was trying to ask was the following: In the above example, the two domains served are hypno.cat and abc.de. Is the rDNS setting point to hypno.cat or abc.de, or something totally different (say mail.example.com) in your case? Moreover, when you are setting up the email client, what would be the IMAP and SMTP settings you would use for each domain? hypno.cat and abc.de accordingly, or a common mail.example.com for both? I hope I could explain myself better this time.

@hakova The rDNS should be the hostname of your mailserver;
IP of your server should point as rDNS to: mail.yourdomain.com

And all your domains should have:
mail.yourdomain.com as MX records of them.

And setting up your email client;
You point SMTP/IMAP/POP server hostname as;
mail.yourdomain.com (Hostname of your mail server)

That's all. Here is how to test your Forward-confirmed Reverse DNS record:

https://www.blacklistmaster.com/reverse-dns-test

[abdulocracy]

I was having very strange issues with the Debian-distributed version of opensmtpd-filter-rspamd (version 0.1.7-1+b7 for bookworm). Outgoing messages from authenticated users were only being DKIM signed if they were sent from the localhost, and not if they were from a foreign IP.

Installing filter-rspamd from master by following the instructions in the repo fixed it entirely. Now rspamd detects authenticated users as it should.

[thyssentishman]

Thank you for an amazing blog post. It's been my preferred reference since the first time I set up a mail server on OpenBSD.

One small suggestion, you may want to remove the login.conf snippet here as a default dovecot login class is now (since 2022 actually) shipped with the package.

[poolpOrg]

Ah ! Hadn't been paying close attention, I'll likely do an update version since this post is a bit dated now, thanks for the hint !