AspEmail.NET Manual Chapter 4: Message Queuing
Contents
4.1 Message Queuing: Overview
4.1.1 AspEmail.NET's Message Queuing Mode
When the method objMail.Send() is called, AspEmail.NET opens a session with the SMTP server and sends message information to it for further processing and delivery. Depending on the message size and number of recipients, an SMTP session may take a long time to complete. Sending messages to hundreds or thousands of recipients may put a heavy load on IIS for an extended period of time and affect the overall Web server performance.
AspEmail.NET offers the Boolean property Queue which, if set to True, instructs the Send method to place a message in a queue and return immediately, without opening an SMTP session. Your message-sending script no longer has to wait until a message is sent, which means a better client-response time and less load on IIS.
AspEmail.NET's queue is simply a folder on the server's hard drive where each individual message is represented by a text file with a unique name. The messages are retrieved from the queue and sent out by EmailAgent.NET, a separate executable running in the background as a Windows service.
EmailAgent.NET is a multi-threaded process capable of opening multiple parallel SMTP sessions for better throughput. If a message cannot be delivered for any reason, it is returned back to the queue to be re-sent later.
EmailAgent.NET supports multiple queues. To send your message to the right queue, the queue path must be explicitly specified via the QueuePath property, as follows:
objMail.Queue = true;
objMail.QueuePath = @"c:\Queue2";
objMail.Send();
NOTE: For compatibility with the classic AspEmail component, if the Send method is called without an argument, AspEmail.NET will attempt to send the message to the queue pertaining to the classic version of EmailAgent as specified in the system registry.
The Queue property is a premium feature.
4.1.2 Deferred Processing
EmailAgent.NET only picks those message files from the queue whose Last-Modified information is set to a timestamp prior to the current system time. When a message fails to be sent for any reason, its Last-Modified time is automatically changed forward by an amount specified in the configuration panel (see below) so that EmailAgent.NET would try to send it again after that amount of time.
The Last-Modified timestamp of a message file can be explicitly specified in your script via the property Timestamp to schedule the delivery of the message at a specific time. For example, the following code sample schedules the message to be delivered in 5 hours:
objMail.Queue = true;
objMail.QueuePath = @"c:\Queue2";
objMail.Timestamp = DateTime.Now.AddHours(5);
objMail.Send();
4.1.3 DNS Relay
Normally, EmailAgent.NET emails messages using your local SMTP relay server. Alternatively, each message can be emailed directly to the recipient's own SMTP server by looking up the DNS records for the recipient's email domain. The DNS relay mode is slower but makes it possible to send messages without access to an SMTP relay server.
4.1.4 Bounced Message Handling
In addition to SMTP functionality, EmailAgent.NET is also capable of retrieving bounced email from a POP3 account to help remove email addresses that are no longer valid from your database of subscribers. This functionality is described in detail in Subsection 4.3.4 of this chapter.
4.2 What's New in EmailAgent.NET
EmailAgent.NET is similar to EmailAgent included with the classic AspEmail component and described in Chapter 7 of the AspEmail User Manual.
Compared to EmailAgent, EmailAgent.NET offers the following new features:
- Multiple queues. The old version only supported a single queue.
- Up to 256 simultaneous threads per queue vs. 64 in the old version.
- Support for Transport Layer Security (TLS). The old version could not work with SMTP servers requiring TLS.
- A new log file is created hourly, daily, weekly or monthly. In the old version, a single log file kept growing indefinitely.
- The configuration panel is a regular executable. In the old version, it was a Control Panel applet.
- EmailAgent.NET is 100% managed code. The old version was a standard Windows application.
AspEmail/EmailAgent and AspEmail.NET/EmailAgent.NET are cross-compatible. In other words, message files generated by AspEmail can be processed by EmailAgent.NET, and message files generated by AspEmail.NET can be processed by EmailAgent.
4.3 Manual Installation
The EmailAgent.NET service consists of three files:
- EmailAgentNet.exe: The actual Windows service;
- EmailAgentNetConfig.exe: The configuration panel application;
- EmailAgentNetRemote.dll: The assembly facilitating communication between the service and its configuration panel.
For the sake of simplicity, all three files should be put in the same folder. If the service and its control panel executables should be separated into two separate folders, the assembly EmailAgentNetRemote.dll must be copied into both folders.
To register the EmailAgent.NET service, open the .NET SDK Command Prompt and execute the command
Once this command is executed, the EmailAgent.NET service should appear in the Services list:
To make EmailAgent.NET start up automatically when the server starts, change the Startup type parameter under the Persits Software EmailAgent.NET properties from Manual to Automatic.
To unregister the service, execute the command
4.4 EmailAgent.NET Service Configuration
The EmailAgent.NET service is configured with the EmailAgentNetConfig.exe application. The configuration data is stored in the XML file emailagentnet.exe.config located in the same folder as the service executable itself. Editing this file directly is not recommended.
4.4.1 General Service Settings
The stoplight icon on the left side of the Configuration Panel displays the current service status: running, stopped, starting, stopping, and unregistered. To start and stop the service as a whole, use the Start Service and Stop Service buttons, respectively.
Initially, the service is configured to have a single queue. To add a new queue, use the Add button at the top of the configuration panel, to the right of the queue list. Use the Rename button to rename the currently selected queue. Use the Delete button to delete the current queue. Only a stopped queue (a queue containing no active threads) can be deleted.
4.4.2 Queue Parameters Tab
The Queue Parameters tab contains various queue parameters as well as displays the status of individual threads in the current queue via color-coded bulbs (right pane.) Each queue can be individually started and stopped via the Start Queue and Stop Queue buttons.
Message Queue Path
This parameter specifies the location of the message queue folder on the machine's hard drive and must point to an existing folder. EmailAgent.NET will automatically create 4 subfolders underneath the specified folder: \Sent, \Failed, \Incoming, and \Log. Successfully sent messages are copied from the main queue folder to the \Sent folder, and failed messages to the \Failed folder. Incoming message are saved in the \Incoming folder. The \Log folder contains the log files.
Simultaneous Threads
This parameter specifies the number of simultaneous SMTP sessions attempted by this queue. This number must be between 1 and 256. This parameter can be changed dynamically, while the queue is running. Note that an increase in the number of threads is reflected by the thread status pane immediately, while a decrease is reflected only if/when the queue is actively processing messages.
This parameter should be chosen carefully. A large number of threads may not necessarily lead to an increase in throughput as it may cause frequent socket timeouts or connection drops by your SMTP server.
Repeat sending after ... min.
When a non-fatal error occurs during an SMTP session, the message is left in the main queue for EmailAgent.NET to try and resend it. This parameter specifies (in minutes) how soon EmailAgent.NET should attempt to send this message again.
Poll message queue every ... sec.
This parameter specifies (in seconds) when to poll the message queue folder again if it is currently empty.
Cancel sending after ... hours
If a message generates a non-fatal error, EmailAgent.NET attempts to send it repeatedly for the specified number of hours. If unsuccessful, it copies the message to the \Failed folder. Once a message is in the \Failed folder, no further attempts are made to send it.
Socket Timeout
This parameter specifies the timeout value (in seconds) for all socket operations including SMTP and POP3 connections and communications. Increase this number if you get a lot of timeout errors in the log.
As of Version 5.3.0.3127, the timeout parameter can be used to specify the Receive and Send timeouts separately. By default, the parameter sets the Send timeout only, while the Receive timeout is 0 (which means "wait indefinitely.") If the value contains more than 4 digits, the last three (least significant) digits are treated as the Send timeout, while the three most significant digits are treated as the Receive timeout. For example, the value "12020" would mean a 12-second Receive timeout (012) and 20-second Send timeout (020). Setting the Receive timeout to a non-zero value is useful when the thread status indicator shows active threads that refuse to complete.
Do not move sent messages to the Sent folder
If this option is checked, successfully sent messages are deleted from the queue. Otherwise they are moved to the /Sent subfolder.
4.4.3 SMTP Tab
If the option Relay through SMTP host is chosen, EmailAgent.NET sends the messages to the specified Host via the specified Port.
If the TLS (Transport Layer Security) dropbox has TLS 1.0 or TLS 1.2 options selected, EmailAgent.NET attempts to connect to the Host securely via the TLS protocol. The Host must support TLS for this option to work. Many popular SMTP services such as smtp.gmail.com, smtp.office365.com, Amazon AWS, etc. require TLS. Quite often, SMTP servers requiring TLS operate on port 587 instead of port 25. If TLS 1.0 or TLS 1.2 options are selected and Port is set to 465, EmailAgent.NET will use the SSL protocol instead of TLS.
If the SMTP server requires authentication, check the Authentication box and use the parameters Username and Password to specify valid user credentials for this Host. Currently, EmailAgent.NET only supports the AUTH LOGIN authentication method.
Important: If the Host, Port and Username/Password values are set in your script via the corresponding AspEmail.NET properties, these values become part of the message queue file and take precedence over the EmailAgent.NET configuration. Do not set these properties in your script if you want the EmailAgent.NET configuration parameters to be used.
For backwards compatibility, the TLS=true flag cannot be part of a message queue file. To enable TLS in the message queue mode via your script, add one or two trailing spaces to the Host value. One space will enable TLS 1.0 and two spaces TLS 1.2 (AspEmail 5.5+ is required for TLS 1.2 support), as follows :
...
objMail.Queue = true;
objMail.Send();
If the option Relay through DNS is chosen, EmailAgent.NET attempts to obtain the recipient's SMTP server from his email address by looking up the MX records of the email's domain. DNS relay is slower but it frees the sender from needing his own SMTP relay server.
4.4.4 POP3 & Logging Tab
4.4.4.1 POP3
To take advantage of EmailAgent.NET's bounced message handling functionality, the following steps should be taken:
Step 1: Set up a "sink" mail account where all bounced messages would go. Let's call it bounced@mycompany.com. You may also use an existing mail account as your sink account.
Step 2: In your mail-sending script, set the property MailFrom to the "sink" address:
Step 3: On the POP3 & Logging tab of the EmailAgent.NET Configuration Panel, check the Enable POP3 checkbox and specify the POP3 Host, Username and Password for your sink account. Also make sure the checkbox Log POP3 retrievals is checked. The frequency of POP3 polling is controlled by the parameter Poll mailbox every ... min.
As of Version 5.4.0.3754, if the Port value is set to 995, EmailAgent.NET will attempt to connect to the POP3 server via the SSL/TLS secure protocol. Some POP3 servers, such as pop.gmail.com, require SSL/TLS and would not allow connection via the regular port 110. TLS 1.0 is used by default.
As of Version 5.6.0.5189, if the Port value is set to 99512, EmailAgent still connects to the POP3 server via Port 995 but via TLS 1.2 as opposed to TLS 1.0.
Step 4: During and after a batch send, EmailAgent will retrieve bounced messages from the "sink" account and place them into the \Incoming subfolder underneath the main message queue folder. At the same time, an entry will be added to the log, which may look as follows:
You can now retrieve and process all bounced mail addresses using the MailLogger object included with AspEmail.NET, as follows:
Logger.Open(@"c:\Queue\Log\EmailAgent.log");
foreach (LogEntry entry in Logger.Records)
{
if(entry.Status=="RECEIVED" && entry.Recipient.Length != 0)
{
// so something with entry.Recipient
}
}
Logger.Open("c:\Queue\Log\EmailAgent.log")
For Each entry As LogEntry in Logger.Records
If entry.Status="RECEIVED" And entry.Recipient.Length <> 0 Then
' Do something with entry.Recipient
End If
Next
You can replace the "Do something" line with some database script that removes the entry.Recipient value from your address database.
Note that entry.Recipient can sometimes be empty as it is not always possible to extract a recipient address from a bounced message, therefore you must test this property for an empty value.
4.4.4.2 Logging
EmailAgent.NET maintains log files in the /Log subfolder of the main queue folder. If the option Log error messages only is chosen, the service only logs failed sends. If the option Log all messages is chosen, all successful and failed sends are logged. If the checkbox Log POP3 retrievals is checked, POP3 retrievals are logged as well, which is necessary for the bounced message handling procedure described above.
Depending on the Create new log file listbox selection, EmailAgent.NET creates a new log file hourly, daily, weekly or monthly. If the Unlimited option is selected, the log file grows indefinitely. The button Open log opens the most recent log file.
The log files can be access programmatically via the MailLogger object included with AspEmail.NET. An example of such use is shown in the previous subsection. The method Open requires that the full path to a specific log file be specified. Another method, OpenMostRecent, requires that the log folder path be specified, and chooses the most recent log file automatically.
The Records property returns the collection of LogEntry objects representing individual log entries. The LogEntry object supports the following properties:
Text - returns the entire log entry text.
FileName - returns the filename of the message file responsible for this entry.
Time - returns timestamp information for this log entry.
Status - returns the strings "SUCCESS", "FAILURE", "ERROR", "SYSTEM" or "RECEIVED" for a successful send, failed send (non-fatal, message will be resent), failed send (fatal, message will not be resent), system error, or received email, respectively. Mail receiving is covered later in this chapter.
Recipient - returns the recipient address. This property is particularly useful for bounced mail handling purposes (covered above).
In rare cases, when EmailAgent.NET is unable to write to its internal log for whatever reason, it writes to the Windows log viewable with Event Viewer.
4.4.4.3 Debugging
To simplify troubleshooting, EmailAgent.NET can be configured to generate various debug messages viewable with the free Microsoft application DebugView (which can downloaded from various sites.) The Debug Flags parameter on the right side of the POP3 & Logging tab controls which groups of messages are to be generated. The value 0 disables all groups of debug messages, while the hex value FFFF enables them all.
Each debug message generated by EmailAgent.NET starts with the marker "EmailAgent.NET" followed by the flag value corresponding to this particular group of debug messages (in square brackets) followed by the queue name (in square brackets) followed by the actual message text.
The following groups of debug messages are currently defined:
| Group | Enabling flag (Hex) |
|---|---|
| SMTP sending thread start/end | 0001 |
| Message file polling thread start/end | 0002 |
| Obtaining message files from the queue folder | 0004 |
| POP3 thread start/end | 0008 |
| Waiter thread start/end | 0010 |
| Writing to Windows log | 0020 |
| Error writing to Windows log | 0040 |
| Error renaming a message file | 0080 |
| Acquiring a message file by an SMTP thread | 0100 |
| POP3 connection | 0200 |
| POP3 retrieval | 0400 |
| Error writing to the internal log | 0800 |
| Buffer overflow | 1000 |
| Setting TLS version | 2000 |
For example, to disable all messages marked [0004] (belonging to the group "Obtaining message files from the queue folder") and enable all others, use the Debug Flags parameter value FFFB (which is FFFF - 0004). Use the value 0 to disable all debug messages.
