While I don't necessarily trust an external company with all my emails, I also don't trust myself to maintain the myriad daemons involved in this setup without doing something subtly wrong that results in my server not sending/receiving all the mail it should -- or, worse, being used for spam.
What would be useful is a pre-assembled virtual machine image or other form of appliance that allows you to deploy and test a mail server within about an hour or so, without having to duct-tape any of this together yourself.
I've been hosting my own mail since 1996. It's actually one of the easier services to self-host:
1) SMTP was developed for unreliable environments. If you have problems with uptime, your incoming email will bounce around for 5 days before it gets dropped. So assuming you can get your SMTP server running one day out of five, you shouldn't be in danger of losing anything.
2) Contemporary daemons like postfix and dovecot have sane defaults, so even a naive default install should be mostly secure. They're also extremely low velocity, so once you set it up there's not a lot of ongoing maintenance.
Hats off to ya. I used to view setting up your own email services as a rite of passage for any UNIX admin worth their salt. SMTP was one of the first services I ever got working on my home Linux box, and I had to do it with Sendmail and the goddamn Bat book. Nowadays it's a bit easier.
It's easier now, in that we have good well-documented software, but the external environment has changed. While other servers used to just accept the email you sent, spam countermeasures have gotten complex enough that if you just follow the postfix installation guide you're going to have a lot of your outbound smtp filtered.
Grab a free account at mailgun.com and configure it as your outgoing SMTP relay.
You'll get an IP address for your outbound traffic which is "clean", monitored and registered with a ton of ESPs. You can also use Mailgun as a proxy for your incoming mails as well, for spam filtering or custom routing purposes.
What about Digital Ocean or another VPS provider? What is to stop them from just handing the NSA a copy of my server image complete with all email history, address book, and authorized PGP keys? I'd have even tagged and indexed all the mail for them!
Yes, that's the point. If you are worried about Google cooperating with the NSA, and you decide to roll-your-own mail solution, but are using US-based services like MailGun, you are doing it wrong. :)
Moving to a self-hosted solution (even a US one) offers you more privacy protection options than Gmail/Hotmail, that's for sure. But since physical access is everything, using a US-based VPS provider means there is only a small speedbump between the government and your mail. Using a US-based service like Mailgun, while extremely cool, removes even this speedbump, since they will presumably be forced to cooperate in the same way that Google or Yahoo do.
The best option would be to host your own mail with a VPS with a very strong privacy record, explicit statements about not cooperating with US inquiries, based and hosted in a country with strong privacy protections.
I have not experienced the troubles you mention with postfix defaults (actually debian's postfix defaults). Can you think of some examples to be on the look out for?
I was the same way, although I preferred NetBSD, but I eventually determined that it wasn't worth the effort to host my own email. That said, I still learned configure it for local only use. I grabbed incoming mail using Fetchmail and sorted it with Procmail.
By the way, I still feel that Sendmail has to have the worst configuration file I have ever seen.
Sure, it takes 30 seconds to install postfix, but in practice, email is stupidly hard to admin because of spam. You may be able to send email easily, but a large percentage will be blocked as spam unless you use a well know relay even if you do everything right. Similarly, you will also be fighting a flood of spam in the other direction.
I host my own mail. I use OpenBSD as my Firewall/Router so it was a simple decision to use the spamd daemon on it for greylisting. The SMTP destination sits on a CentOS machine behind the OpenBSD gateway. I see a spam message maybe once every week or two. I have also used MailScanner to do SpamAssassin and virus scanning of received mail at places I have worked. The spam count went down so far I had people ask me if the mail server was down. A little bit of verification showed we weren't loosing legitimate mail, users were just getting so much spam they thought that was normal.
A smarthost solves your outgoing problem. I can't send direct from my home connection because it is a residential IP and that is going to be blacklisted all over the place. I could use my IPS's SMTP server, hell you could use GMail if you wanted. I however, use a VPS I pay for. My SMTP server logs in (password) over a TLS connection (preshared certificates). It only relays for my mail server because of that.
You don't need to use a well known relay, you just don't relay from a blacklisted IP. Don't leave an open relay and don't send directly from a residential IP block and you probably don't have to worry, unless you start sending spam.
> but a large percentage will be blocked as spam unless you use a well know relay even if you do everything right
That's hardly a rule. You clearly will end up on a blacklist if you host from home or if your IP is already on a list due to previous spam activity, but of you run off a static IP in a decent colo and have a proper PTR record, it is more likely than not to be all OK.
The 90% of inbound spam is still very effectively trimmed off by greylisting, i.e. postgrey if you are in postfix. I've been running my own mail server for close to 10 years now and it's really not a big deal.
SpamAssassin and friends still do a reasonable job for me, and have for years. I delete a handful every day, but like the parent I've had my email address since 1996, and I haven't been too careful about using it. If I had to do it over again today, I'd look at rspamd. Getting people to accept your mail can be a pain in the ass, yes, but it's not impossible.
* Get a "clean" statically allocated IP address.
* Control your forward and reverse DNS and make them match.
* Set up SPF/DKIM.
* Fill out the Yahoo Bulk Email sender form
That last one probably isn't really necessary, depending on how much email you send to Yahoo.com addresses.
I've been running my own mail server since 2006 and have never had a problem with being rejected. I've run a fairly tight ship:
1. Only allow email locally (i.e., via webmail and ssh) - which many would regard as a real pain, but I and the few people who use my server have been OK with. IMAP is very useful with this policy, for batch move/send from outbox folders.
2. Set up TLS to only deliver via SSL
3. Advertise both these facts via SPF (I haven't bothered with DKIM)
Do you have any tips on testing for deliverability? I have a hunch that if I try to send something, say to my gmail or one of my close friends' email, that the spam filter will allow it because it's a familiar name/domain, but I would expect that if I email a stranger's gmail, it might get flagged...
You can use http://www.mxtoolbox.com/ to see if your domain or IP have been blacklisted by any ofthr public lists. Also, if you get blacklisted you should receive a bounceback email telling you who blacklisted you and where to go to get yourself removed.m
That's bollocks. I have my own email server on some random Hetzner IP address and have never had email rejected; and do note my domain TLD is .ro which tends to get higher spam scores.
I've been meaning to finally get around to doing this myself, so I'll shoot you a question: Do you have a fallback MX? I'm thinking of getting a super-cheap VPS running only Postfix as a mail fallback in case my primary host goes down for an extended period. How is accessing mail on the fallback host (when primary is down) usually handled? An IMAP daemon running there as well? Should the fallback just wait for the main one to come back and then send all the waiting mail there?
Usually, the backup MX just gathers email (so that the email does not bounce to the sender) and delivers it to the primary MX when it gets online and then the primary delivers emails to the server that is going to expose it via IMAP or POP3.
But this is just for receiving emails (from the outside world), if your IMAP/POP3 server is on the same server as your primary MX you will not have access to emails on the server while the primary is down. You have to find a way to sync your received mail (maildir/mailbox) to two or more servers.
If you decide to implement a backup MX, try to sync your allowed recipients list from the primary because spammers often try to send emails to an MX with a lower priority, and if it accepts all emails for the domain without checking if that mailbox actually exists you could became a source of back-scatter.
Citadel is another integrated solution that is really easy to setup. Just one application, all written in C. The web interface is the only thing that I think could be better.
"Citadel offers versatile email services with very low administration needed. It provides its own implementations of these server protocols: IMAP, POP3, SMTP, ManageSieve, XMPP, Citadel."
I didn't mention Citadel because there hasn't been a release since November 2011; however on further investigation it looks like development is still active. http://code.citadel.org/?p=citadel.git;a=shortlog
I've used roundcube before, both on a shared hoster and on my own server later on. It's okay, I wasn't impressed with the user interface at the time though that seems to have changed judging from their website. The software is a bunch of php and shell scripts and uses a MySQL, PostgreSQL or SQLite database.
Thanks. It doesn't sound hugely convincing. I guess people do harden PHP+shell scripts, but it sounds like work to me. eitland thinks Kolab can be run without it, at the expense of not having a webmail interface, which would be OK.
E-mail and the extensions and extras that make it go and make it nice /are/ duct-tape. I wouldn't advocate for /anyone/ running their own mailserver unless they're totally aware of what they're doing and on top of spam issues.
When considering if running your own mailserver is practical for you, consider the total cost of ownership; you'll most likely be paying for a dedicated server (or vps, VM, or whatever the cool kids are using these days) which will require staying on top of all the "normal" things; monitoring, backups (and you'll probably want to test them), updates, what blacklists to use, ensuring that /your/ server doesn't end up blacklisted, etc. Unless you've been a sysadmin before and know what this all entails, I wouldn't really recommend it. If you're not part of some bigger org (a hosting co or bigger company), it'll also be harder to get your one-off no-name server removed from blacklists, since this looks just like something Joe Linkfarm would do. I could show you this one really simple trick that a mother found to whiten your server reputation, but...
As far as creating a VM image that sets all this up for you, it's kinda the point that it doesn't exist; mailservers don't work well when run by people who don't really care about them. It's quite far away from a "set and forget" deal. This is why many hackers / sysadmins still use GMail; it's not the best at everything, but it's OK enough at most things that it's the least hassle to use.
That all said, I do agree that it'd be great to have a service which offers both more control for power users (I'd love a mailfilter-style config w/regex support for GMail), better privacy (easy PGP integration, etc), tag support, threading, and normal IMAP support, but I'm not quite holding my breath. This seems like a decent problem for some start-up to solve :) It's not a "sexy" problem, but it's a real one.
> What would be useful is a pre-assembled virtual machine image or other form of appliance that allows you to deploy and test a mail server within about an hour or so, without having to duct-tape any of this together yourself.
There at least a couple Linode stack scripts that will do this, but I haven't given any of them a try since last time I checked they were mostly/all Ubuntu based. Dovecot + postfix isn't all that difficult to set up, and there are literally tons of guides all over the place (Arch wiki, Linode, Dovecot's site, etc.) for dovecot, postfix, courier, and anything else you might want. Perhaps the most dumbfounding thing for a beginner is the certificate step...
What would be useful is a pre-assembled virtual machine image or other form of appliance that allows you to deploy and test a mail server within about an hour or so, without having to duct-tape any of this together yourself.