Scale UP Postfix servers

The ever increasing business complexity demands the need of sending bulk emails. Typically the MTA’s available nowadays are not capable of sending emails at rate which a business organization actually needs. There only few exceptions available such as PowerMTA but they will cost you a lot of money.

This article will give you a good picture on sending bulk emails with at a cost effective way by running multiple postfix servers loadbalanced by Haproxy.

Tune your POSTFIX box to increase the email sending rates:

Before we go onto loadbalancing POSTFIX load, lets make sure each of your postfix servers are sending as much as emails it can. These are some interesting parameters which you will find helpful in tuning postfix to send more emails per minute.

Each of these parameters must be calculated using benchmarks. There are some providers whom you can send emails to for benchmarking. They just accept whatever emails you sent and never complains :) Google out to findout such providers who will let you send emails to random email accounts ANYTHING@provider_domain.com.

default_process_limit: This will give you the number of postfix processes. If your servers have good capacity consider increasing this limit,otherwise decrease it. Run benchmarks with values 75, 100, and 125 as values and if the servers appears to be sending more mails when its set to 125, run benchmarks again for 150, 175 and 200 and repeat this until you end up with the right value. On the other hand if you find email rates are high when the value is 75, run benchmarks again for 50, 60 and 60. If your server has latest processers with high speed SSD drives, an increased limit would fit for you.

default_destination_concurrency_limit: This parameter says how many messages may be sent to the same destination at a time. You can increase or decrease this limit based on server capacity. Do trust benchmark results again to come up with the right value for your server.

default_recipient_limit and qmgr_message_recipient_limit: qmgr_message_recipient_limit is the maximum number of recipients held in memory by postfix. Once postfix has assigned the email to respective transport, default_recipient_limit will take priority over qmgr_message_recipient_limit which will upper limit on the number of in-memory recipients. Use benchmark to determine these values.

transport_destination_rate_delay: default_destination_rate_delay is the delay in delivery per transports. By defalt its zero, but maybe increased per transportssuch Some in master.cf providers may block you if they receive too many emails together. In such situations, you dont need to change increase the delay as whole, but add a unique transport for such providers in master.cf and then set a unique transport_destination_rate_delay. Unfortunately, this is one of the hardest part to determine such providers. It can only be achieved by continuel monitoring of your failed emails and email statistics. You might also want to frequently change this value based on your analysis.

With right tuning and proper hardwares, you can achieve upto 20K emails per server per minute. If you want to increase the rate further, you can have a loadbalancer box running haproxy that will distribute the load among postfix servers thereby multiplying the total capacity. With 5 boxes of postfix servers and a loadbalancer, you can reach the rates of upto 100,000 emails per minute.

Loadbalancing Postfix with haproxy:

Install haproxy:

$ add-apt-repository ppa:vbernat/haproxy-1.5
$ apt-get update
$ apt-get install haproxy

Now add te below configurations into your haproxy:

frontend smtp # Frontend block starts here
bind 10.10.10.1:25 # Replace 10.10.10.1 with the IP address of your loadbalancer. Its recommended to use a private network. Make
#sure this IP is added to each of your postfix servers for relay access
mode tcp
rate-limit sessions 5000
no option http-server-close
timeout client 1m
log global
option tcplog
default_backend postfix #Replace “postfix” with the name of the backend. In this exampele, I have given the name “postfix” for backend

backend Postfix #Backend Block starts here
mode tcp
no option http-server-close
log global
option tcplog
option redispatch
option smtpchk HELO hostname #Haproxy run health check on postfix servers and issue the command “HELO hostname”,replace
#hostname with your postfix mail name
timeout server 1m
timeout connect 5s
balance roundrobin
server server1 10.10.10.2:25 check inter 8000 #Backend server 1, run healthcheck on this server every 8000 milliseconds
server server2 10.10.10.3:25 check inter 8000 #Backend Server 2, run healthcheck on this server every 8000 milliseconds
server server3 10.10.10.4:25 check inter 8000 #Backend Server 3, run healthcheck on this server every 8000 milliseconds
server server4 10.10.10.5:25 check inter 8000 #Backend Server 4, run healthcheck on this server every 8000 milliseconds
server server5 10.10.10.6:25 check inter 8000 #Backend Server 5, run healthcheck on this server every 8000 milliseconds

Restart Haproxy using

/etc/init.d/haproxy restart

Run your benchmarks again for bulk emails :)