Scale and load

[jshapiro2]

Hello All,

I am looking to send 200k to 300k transactional email per a day through Postal. Has anyone put this much load on the system? Any suggestions or tips?

[LotisHunters]

My personal record when setting up a single Postal instance and a mailcow server with my Postal sender whitelisted was about 7,000 emails a minute.

I wouldn't say that's something you should expect out of the gate though, because it took a good bit of work to get there.

Before I go into what I've found to be the best settings to tweak for performance, I will preface it by saying that managing your sending reputation and warming up IPs and domains will be critical for being able to send that many emails. If you turn up Postal and blast 300k emails a day off the bat, you will likely run into issues with ESPs.
Also, this is just what I recommend based on my testing and experience playing with Postal. It's not "the definitive" way to configure Postal and may not work the best for you depending on what hardware you have.

That being said, Postal is definitely capable of managing that kind of load as long as you have the CPU, RAM, database, and network to handle it.

On the database, make sure it has plenty of RAM. The most important setting in my experience is the number of connections it allows. There are a bunch of other settings you can tweak for performance, but they're just general settings like memory allocation. Especially when you start scaling up Postal workers, it may consume a few thousand connections. (This may be alleviated with connection pooling but I haven't explored that)

On the Postal side, you will likely want plenty of workers available. Supposedly you can configure this in the config file, but it doesn't seem to work for me, so what I do is '''sudo postal start --scale worker=10'''
(You can increase and decrease that number without restarting postal by running the command again but with a different number)
The more workers you configure, the more database connections you will end up consuming. It will also use a ton of RAM so don't just do a crazy high number. You'll likely hit job queueing bottlenecks/limits by doing that if not run out of database connections.
You should be able to handle that many emails a day with somewhere between 10-50 workers in my experience, but definitely play with that number.

I hope that's helpful and can get you off the ground.
If you have more specifics about what hardware you're using, I may be able to provide more specific advice.

[jshapiro2]

Thank you so much for the feedback and suggestions. I am not to worried about the ESPs as I have lots of experience with whitelisting and have already taken care of all requirements.

This environment is running in a private cloud on VMware. The Postal server is currently setup with 2 cores and 16GB of memory. Ubuntu 22.04.3 is the current OS and Docker 25.0.1. It is a 10GB connection to the MySQL server. Not worried about the DB cause I can scale up as needed.

Any other suggestions or documentation you recommend?

[LotisHunters]

16 gigs of RAM should be fine.
The CPU might be a bit light, but you'll be able to see that pretty clearly if it is an issue.
The main frustration I have when trying to scale Postal is when I hit a limit and system usage doesn't show any issue. My guess is that a lot of throughput gains could be had through rabbitmq tweaks, but I don't know enough about it to fully understand where to even start.
One thing I noticed is that if a lot of messages soft fail, they will fill up the queue and keep retrying a lot. The default is 18 attempts. Once the soft fails in the queue start piling up, I find that throughput starts slowing down a LOT until they clear out. It ends up being better to pause sending messages to Postal in order to let it clear out so that it doesn't choke. I'm not fully sure what is causing the issue there but I think it's probably job queueing.

I'd say start with what I recommended and maybe 10-15 workers and see what throughput you're able to handle or if it causes system load issues.

[jshapiro2]

Thanks, I have the maximum_delivery_attempts set at 10. The good thing is all my emails are transactional and email is validated at time of send by the recipient.

I have read about the the rabbitmq concern and looking at setting up the RabbitMQ management plugin.

[arshiyaaara]

Thanks, I have the maximum_delivery_attempts set at 10. The good thing is all my emails are transactional and email is validated at time of send by the recipient.

I have read about the the rabbitmq concern and looking at setting up the RabbitMQ management plugin.

how did u set this ?