Why you should not use DBMail as your IMAP server
Recently we acquired new servers for our growing needs. Decision was made to move our old mail server as a virtual machine. Mysql seemed a nice place to store mails instead of the old fashioned bunch of files in a disk. Also we had a separate database server and there wasn’t that much disk space in the virtual machine host. We needed that space for other virtual machines more. Strangely though, there seemed to be very little mail softwares able to use SQL for storing the mails itself (on Linux of course). Actually, we only found DBMail (maybe we didn’t look hard enough?).
I started setting up the new server. Postfix and DBMail 2.2, which came with ubuntu, started working quite easy together. I got everything up and working with my thunderbird. Though DBMail 2.2 didn’t support STARTTLS, it seemed no biggie, since SSL was supported easily through stunnel. Every test went ok and the new server worked with my Thunderbird fine (Note to self: don’t rely only on one client software when testing). I ran first sync to our mailboxes and changed DNS records to point to our new server. Next day, when DNS records had started to update, other people started to complain about the mail not working, their clients receiving mail very slowly etc.
First thing I discovered: if you had forced STARTTLS in your mail client and it then did try to use it with DBMail, DBMail hangs. It just hangs. Processes are running, you can connect to it, but it does not answer anymore to any command. So if anyone forces STARTTLS not knowing that server doesn’t support it (which should not be bad), it crashes the whole mail system. Nice. Did I think to test this? No, ofcourse not.
Next problem was that in some weird way, SSL through stunnel didn’t work either with all mail clients (most people used either Thunderbird or Mac OS builtin mail client). Some people could connect ok and some couldn’t. At this point I didn’t even bother to figure out more and decided to upgrade DBMail 2.2 to 3.0.
Manual installation from source code had the normal small problems of missing dev version of libraries etc. Database upgraded nicely to 3.0 format. STARTTLS was also supported. Everyone could even connect to the server. Then, again, the problems started. Saving mail in send-folder example did seem to work about every second time. The other times it just timed out. Fetching new mail was very slow and all in all the server seemed to work very slowly. We do not have that much users nor traffic that it could, in any way, cause that kind of a problem. The database server is so massive, that mysql is not the bottleneck. We tried living with those faults for a while, but after a few days the mail server stopped responding at all. Reason: DBMail 3.0 leaked memory and it had been terminated by the kernel.
That was it, we had a short disgussion about the matter and conclusion was to change back in to old faithful Dovecot. Every now and then you make bad decisions and DBMail was one of them for us. New mail server using Postfix+Dovecot is just now syncing mails from DBMail (god damn slowly might I add). Mails are stored through NFS share to our storage server and indexes are on the local virtual machine disk. No need for big disk space on the virtual machine. STARTTLS and SSL both work without any problems. No stunnel needed. Mysql is only used for user and virtual host information.
Sometimes, or actually many times, don’t try to fix something that isn’t broken. Dovecot had worked fine. I had bad hunch about DBMail. I told it to my boss who did the final decision about the change. I was ok with DBMail, since it was just a hunch.
There isn’t much details and I left out a lot since I wanted to keep this short (atleast on some level). The main point is still there.
“Actually, we only found DBMail (maybe we didn’t look hard enough?)” — there’s a set of patches for c-client to support SQL-based storage, which means that UW-imapd is able to do it. But UW-imapd is generally considered deprecated these days (I believe only Mark Crispin ever knew enough about how it actually worked to maintain it), so that’s probably not useful either.
I’ve been successfully running dbmail for about 10 years now, but all my clients are using POP3, which seems a bit more stable. We have occasional issues (primarily that if disk space usage gets to over 50% mysql stops being able to compact tables after deleting messages, which ends up accelerating disk usage even more until the system runs out of space … and mysql’s behaviour when it runs out of disk space leaves a little to be desired).