Categories
Cloud Developer Tips

Sending Email from EC2

A common question about moving applications to EC2 is what to do about sending email. Web applications occasionally send email, to their administrators (“Help! I’m running out of disk space!”) or to their users (“Here is your forgotten password”), and this needs to work when the application is hosted within EC2 also. Unfortunately, it is not simple to send email reliably from within the cloud. Here are some tips about sending reliable email from EC2 instances.

What is the Problem?

One word: spam. Email sent from servers within EC2 is marked as spam by most ISPs and email providers. The cloud offers a very scalable way for anyone to launch email-sending machines, on-demand. Spammers know this. So do the anti-spammers. Luckily (for the anti-spammers), existing anti-spam safeguards are effective against attempts to send spam from within EC2. Unluckily (for you), these same anti-spam measures make your legitimate (I hope!) emails get flagged as spam.

The anti-spam safeguard that trips up your application’s emails is called reverse DNS lookup. This lookup checks that the name and IP address of the mailserver sending the message match the name and IP address specified in DNS records. Here’s how it works:

  1. A machine at mail.yahoo.com receives a message from your mail server, whose IP address is (let’s say) 1.2.3.4.
  2. mail.yahoo.com looks in DNS for a PTR record for the IP address 1.2.3.4. If none exists, the mail is not delivered. Otherwise, the PTR record specifies the FQDN of the machine that should have the address 1.2.3.4. Let’s say this is specified as mail.mydomain.com.
  3. mail.yahoo.com looks in DNS for the IP address of the host specified by the PTR lookup: mail.mydomain.com. If the IP address does not match the address from which the mail was received (1.2.3.4 in our case), the message is not delivered.

The above is a bit of a simplification, but it is enough to explain the difficulty in sending email from within EC2. The problem lies in step 2, when the receiving mail server tries to find the PTR record for the IP address from which the mail was sent. The public IP addresses used by EC2 belong to Amazon, and the DNS lookups for these addresses will therefore look in Amazon’s DNS records. Amazon does not have PTR records pointing to your mailserver’s FQDN in their DNS, so the reverse DNS lookup fails at step 2, and your email message is not delivered.

Amazon, unlike traditional hosting companies, does not have a service that allows you to set a reverse DNS entry (a PTR record) within their DNS records. If they did offer such a service it would solve the reverse DNS lookup problem, and enable your EC2 application to send emails. Of course, such a service would also allow spammers to easily use EC2 for sending spam, so it is clear that Amazon cannot offer a service to allow custom PTR records without instituting some effective safeguards against abuse.

Update 30 October 2009: Amazon now has a new email policy in which outbound SMTP traffic is blocked (beyond miniscule usage). In order to be able to send email directly from EC2 you also need to provision an Elastic IP address for your instance and submit the following form. In return, Amazon will work to keep that Elastic IP of of the common anti-spam lists. More information here. Note that the comments in this article relating to reverse DNS are still applicable: some email providers do not check the PTR records, and Amazon’s new program will help email get through to those providers – but not all.

Update 25 January 2010: Amazon has announced a new private beta where they will set PTR records for your Elastic IP address. To participate in the private beta, join the EC2 developer forums and send a private message to ian@aws.

Update 23 March 2010: Reverse DNS for Elastic IPs is now officially supported!

Sending Email from EC2: What Works

In any case, let us explore how you can send email from within EC2. Here are a number of cases that work:

  • One case that works is sending email to email accounts that are hosted on the same EC2 instance, using sendmail as the mail transfer agent (MTA). This can be appropriate for a corporate setting, where your recipients can be told to configure their email readers accordingly. To do this, set up sendmail on your EC2 instance, then set up a POP3 or IMAP server on the same instance, and then set up your users’ email readers to fetch email from those accounts. Be sure to configure the sendmail and POP/IMAP services to accept connections only from machines you trust. (You’ll probably want to use an Elastic IP and an EBS drive also, to ensure that access to those email boxes can be recovered if the instance goes down.)This case does not have the reverse DNS lookup problem because sendmail will deliver the messages to the account on the local machine directly, bypassing the reverse DNS lookup.One limitation of this approach is that the users must learn to check this “special” email account. Some email readers make this easy by allowing more than one server-side account to be checked.

    A more significant limitation to this approach is that it will not work for sending mail to recipients who have accounts elsewhere. The reverse DNS lookup anti-spam measure described above will prevent those messages from getting through.

  • Sending email from EC2 to an address whose email delivery you control can work also. The application sends email from a known email account (say, “myAppAdminEvents@mydomain.com”) to your application administrators’ email accounts in your domain. In this approach you set up an SMTP server on your EC2 instance. You configure the SMTP installation on EC2 to relay mail directly to your domain’s mail servers. And you need to make sure that the anti-spam tools running on your domain’s mail servers (and in the user’s email reader) are configured to whitelist messages from the known email account.
  • You can use third-party SMTP services to send emails from your EC2 instance. There are many third-party email services available, including the free-but-limited Google Apps Email, and also including paid-but-less-limited solutions. A detailed discussion of how to set up your email to use a third-party SMTP server is beyond the scope of this article (but here’s a good one). However, a useful strategy is to run an SMTP server locally on your EC2 instance which is configured to relay the mail to the third-party server. This relaying setup allows your application to benefit from reduced connection time when sending messages, and relays the messages in a separate process.There are many third-party SMTP service providers available, each with their own pricing structure and Terms & Conditions.

    Google Apps Email’s limitations are an interesting subject, and I plan another blog article on strategies for dealing with these limitations.

The above three methods of sending email from your EC2 instances each have their pros and cons. If you’re only delivering messages to internal recipients, use the first or the second method. If you are also sending mails to arbitrary addresses, go for a third-party SMTP provider.

25 replies on “Sending Email from EC2”

good work Shlomo,

just wanted to share my experience with the same problem: –
I changed the server's IP address, but before changing it, I got 4 elastic IPs & checked them on http://www.spamhaus.org/pbl/index.lasso whether it is listed in the block list or not, 4th ip was not listed in the policy block list, then I released the other ips & assigned the 4th ip to the server, now the mails from this server are not getting rejected by yahoo, rediff & hotmail also the mails are not labelled as spam.

Thanks,
Deepak

My elastic IP is not listed in Spamhaus, SpamCop, nor CBL…and Hotmail (an msn.com address) is putting my instances mail into its Junk folder. I have a full fledged SPF and MX records setup appropriately…no dice. I wonder if there's a way to apply to be white listed…

I've not bitten the bullet yet, but I'll need to, so once again I'm reading your excellent blog with interest.

I'm planning to set up the company domain's MTA to allow relay to clients, which have certificates signed by a local Certificate Authority, which the MTA trusts, and have all servers instantiated in EC2 use TLS for SMTP with client certificates.

For my applications, it is good enough to put the client private key and signed certificate and the necessary set up for SendMail into my basic AMI.

I'm planning to use a dedicated Certificate Authority for this purpose because it is a closed network – i.e. I'll be using the CA Perl script you get with OpenSSL.

My company's MTA is Postfix, so this should be easy enough to configure. If I was in a bigger company I'd probably lobby for a dedicated MTA for a MX subdomain, hosted on a low-cost mini-server with reverse DNS lookup. Here in the UK, there is a non-cloud company I use for that called Memset.

I'll let you know if I get burned 🙂

@rob- wondering whether you have had success with the above setup. I have setup my MTA and my clients as well, however for some reason I am still getting a relay access denied. I believe I am having an issue with the certificates on the client side. I'd be most interested in your experiences.

Cheers!

Michael Klatsky
TnR Global, LLC

Are there any limits on the number of emails I can send from an EC2 instacnce when I am using a third party smtp server ?

Thanks

I did see some stackoverflow comments where in people have reported saying they were using google mail smtp endpoint but still received number of email message limit exceeded email.

I am composing mail successfully using SMTP details on EC2. But the copy of send mail is not get saved in Sent folder. For that I have used method of IMAP to save the send mail copy in Sent folder. But that doen’t work on EC2 machine. The code is working on local machine using IMAP methods.
1. compose mail – SMTP details
2. save copy of mail – IMAP details (Not work).

Please help me.

@vinod,

Gmail saves a copy of every sent message in the Sent folder. Sending the message via Gmail’s SMTP service should do that for you. You shouldn’t have to have any special handling, and no IMAP is needed to support it.

That’s as far as I know about the Sent folder.
If you learn the solution please report back.

I created an EC2 email server, the emails don’t fail and don’t go into spam, but it take over 5 min for emails to be delivered.

I get a Reverse DNS does not match SMTP Banner on MXTOOLBOX.com and been trying to find a way to fix this (assuming it’s causing my slowness issue)

The thing is, I need my to send emails on behalf of someone else… example, I have an application that our customers use to automate their client onboarding process, and to do that, we need to send an email that can’t be sent from our domain name, it has be from ourcustomer@theirdomain.com

Anyone has any suggest on how to resolve this? Is this what’s causing my slowness issue?

@moe L.,

You will need to use an Elastic IP address and configure Reverse DNS for it. See here https://aws.amazon.com/blogs/aws/reverse-dns-for-ec2s-elastic-ip-addresses/ for more details. This will help you for mail sent from your own domain.

However, in order to send on behalf of your customers’ domains, the only spam-friendly way is to get them to add your server’s DNS name (IP address would work, but DNS name is much more robust) to their SPF record in their DNS. It’s not pretty.

My ASP script (domain name abc.com) in EC2 sends mail to to all email address except mails under the domain abc.com. What amy be the issue?
As a temporary solution, I created a gmail account and send script mail to gmail and so gmail will forward copy to the mails to @abc.comm

Explained with example.
1) asp script is under eg: abc.com/contactus.aspx. It’s a contact form
2) This script send mails to vinu@gmail.com, info@mycompanysite.com, test@anotherdomain.com, and receives it successfully.
3) but fails to get/receive mails to vinu@abc.com, info@abc.com
4) alternative solution done – under vinu@gmail.com, I put forwarding option for the mail vinu@abc.com. Now I am getting mails under vinu@abc.com.
Please advice.

Thank you for the link. I shall check this. (actually I could send mails from other php scripts(non – aws), and from my gmail accounts also.)

First, you must check your domain DNS health, Here is a great tool that helps you a lot https://dnschecker.org/domain-health-checker.php, That checks which blacklist services have your A record and MX record IPs in them. Altogether it includes the DNS health test, MX record test, Mail (MX) record blacklist test, domain IP blacklist test, DMARC test, SMTP test for Mail records, and SPF records test. Provide you a better insight, what is the actual problem.

Hi,
I have a strange issue, I have an SMTP application that connects to one of the company’s un-monitored email accounts (hosted at some ISP) and sends out mail to any valid email address…the application works perfectly…but somehow on EC2 it only sends the first two emails then I get a “505 administrative prohibition”, everytime I run the application only two mails go out…I setup Elastic IP (but not reverse DNS, don’t know what that’s supposed to be)…I’m confused…any suggestion ?

Hi ya,

Nice article really liked it. Just a quick question as I can see you have a really good knowledge on EC2. Recently, I had to set up several email parameters through our DB application using Office 365 access details however, as the DB does not support TLS encryption I had to install STunnel as a middle tier to send emails that support TLS. Since I was setting up the connection via Stunnel, the application does not prompt for any errors while sending however, no emails are really sent from the cloud to outside the world. We have checked firewall rules and the inbound/outbound rules on EC2 and PVC and the traffic is allowed. What else can be checked?

Leave a Reply to slitz Cancel reply

Your email address will not be published.