This block means that either the deliveries have been paused in the interface or none of the available servers are currently accepting emails. MailerQ is not interested in the messages in the parked state because every path that the message can take is blocked. These are the queues like to:, from:ip-pool to:, etc., and can also be seen during pauses from the management console. If the domain is currently not accepting emails or there is some other reason the message should wait, it is moved to the parked state, meaning that it currently resides in (or is being moved to) RabbitMQ and is waiting for a queue there until it is consumed again by MailerQ. This rule slightly decreases throughput per connection but has a much more stable pattern, so overall performance and reliability increase. Usually, not too many messages can be in this state since there may only ever be one message scheduled per actual connection. Every message in this state is somewhere in between being scheduled inside MailerQ and about to be sent it may have even already been partly sent! These are the messages that are practically out the door and are waiting for the receiving MTA's response. If everything goes well and the email may be sent directly or within a very short amount of time, the message will move to the assigned state. Then, depending on the state of deliveries to the domain and previous attempts, MailerQ will either assign the message if it can be sent directly or it will park the message if there is no more space in the in-memory queues. After the message is loaded in memory, MailerQ will append the message to its internal memory queues if there is still space, depending on the email throttle settings and the maximum number of messages that may be in memory simultaneously. When the message is first consumed from the outbox, the message will be in memory. You might have seen it in the frontend already, but there are three different stages for messages. We'll dive a bit into how MailerQ handles messages internally. Overall, this greatly increases the throughput, but this can potentially cause some extra problems. Making it even more complicated is that MailerQ is not managing a simple IP but, in some cases, many thousands of IPs! Each with its throttles, its state, and each MTA IP can also have multiple connections to the same server. There are a plethora of things that can go wrong in this process. Maybe some specific mailboxes do not exist. Perhaps the other server does not want to accept the email right now, greylisting it to be tried again later from the same IP. For one, maybe the connection cannot be opened, which means we should try again later. Although this is conceptually very simple, numerous practical problems are making this difficult. Create an email, resolve the recipient's domain, open a connection to the server, and finally deliver it to the other server. Sending an email in its essence is pretty simple. For this edition, our lead C++ engineer Michael van der Werve is going to talk about a recently introduced optimization for queueing messages on three different levels: per IP, IP Pool, and domain. In this new series, we ask our developers to provide insights into features and optimizations that are not very visible on the surface but have a lot of impact under the hood.
0 Comments
Leave a Reply. |
Details
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |