|
I have a pet peeve about Managers (Project Managers, IT Department Managers,
and the like). It seems that some otherwise perfectly intelligent people,
when they become managers, suddenly go through this Kafkaesque transformation
and become - well, ogres (I can say this without hesitation,
because I was once a Manager, and it didn't last long, because I could
feel it starting to happen to me). Kind of Like Jeff Goldblum staring
at his crawling skin as he began to turn into -- The Fly! Their personalities
change; they seem to get these "hardened" views about certain
technologies and become ultra-dogmatic about whether or not their development
people are going to be allowed to use them to solve business problems.
(Thankfully, the boogers let me go back to being a developer and I remained
happy. My personality didn't change - it's still just perfect.)
Let me give you a couple of real - life examples. These both happen
to relate to MSMQ, and -- not coincidentally -- that's what this short
article is about, too. I was a Senior Developer at one outfit which had
a humongous e-commerce application with over 1200 SQL Server tables spread
out across at least five or six linked servers. One of the needs was
a reliable disconnected messaging scenario involving the clients dropping
notification messages about having uploaded a new product catalog, which
our humongous application was supposed to process and then update these
1200 or so tables - but first by sticking everything in a half dozen
"staging tables" for preprocessing. There really wasn't a reliable way
to deposit notification and status messages that could be picked up later.
Naturally, this is an MSMQ scenario, and when I proposed that we look
into it to the "Manager", you can guess what the response was. "Oh, no,
we've had REAL BAD EXPERIENCES with Message Queue. Absolutely not!".
Doh!
The other one was with another company that had a pretty big operation
that was almost totally centered around SQL Server. SQL Server did everything in
this enterprise - it made the notifications, it polled stuff with timed
jobs running out of SQL Agent, it sent the emails, everything with SQL
Server except maybe cleaning the men's room - and I bet if the manager
could have figured a way to make SQL Server do that, he would have added
it to the list. There was so much contention on their network what with
different jobs running at the same time as all the heavy data access,
that stuff would fail or timeout. I suggested several alternatives here,
including Windows Services, Message Queue, and even Multicast IP for
subscribing to notifications-- all of which were tossed out with comments
like "No.
I don't like WIndows Services. Absolutely not. " Jeesh!
Don't like Windows Services? Better not turn on your computer in the morning
then, cause it runs about twenty of them. . .
The bottom line is this: If the only tool you have to deal with programming
situations is a hammer, every problem starts to look like a nail. That's
a pretty heavy-handed, tunnel - vision approach to programming, and to
life, in general. When somebody says something like "We've had bad experiences
with ..." or "I don't like ...", what they are really saying is, "I'm
the boss, and since I'm too dumb (or unwilling) to figure out how to
use this technology reliably, we simply won't use it so that I don't
have to put my ego in my back pocket in the name of quality software
development". These guys could all use a good course in lateral
thinking.
One of the coolest features of MSMQ 3.0 is the ability to send messages
over HTTP - right through firewalls. This works great in workgroup mode,
and the Queue is simply exposed as an IIS Vroot. But before I get into
that, lets take a quick look at the feature matrix table (from MSDN online)
for Message Queue. This is broken down into New Features, Improved Features,
and Existing Features:
|
Delivering messages over HTTP transport |
SOAP Reliable Messaging Protocol (SRMP), an XML-based messaging protocol, has been developed for delivering high Quality of Service (QoS) messages. The direct, public, and private format names of administration and response queues can be included in messages sent over HTTP transport. Similarly, the names of administration and response queues in HTTP format can be included in messages sent over ordinary (non-HTTP) transport. |
|
Triggers |
Now a component in Message Queuing 3.0, rather than an add-on module that existed in MSMQ 1.0 and MSMQ 2.0. Triggers provide a mechanism that associates the arrival of each incoming message at a queue with a response that depends on the contents of the message and may invoke either a COM component or a stand-alone executable program. Business rules can be defined and invoked in response to such messages without any additional programming. |
|
Sending messages to multiple destinations |
Clients are able to send the same message to multiple recipient queues. Lists of destination queues can be specified explicitly by means of distribution group objects (distribution lists) in Active Directory® as well as in the form of multiple-element format names. Support for ensuring that messages sent to distribution lists and multiple-element format names will reach queues on downlevel computers is provided. In addition, message delivery to IP multicast destinations using the PGM protocol is supported. |
|
Message lookup |
A way to peek at or retrieve a specific message without using cursors to navigate through the queue until the message sought is located. This functionality is based on a 64-bit lookup identifier that is assigned to each message when it is placed in a destination queue. |
|
Improved Active Directory Integration |
Although Active Directory integration exists in MSMQ 2.0, Message Queuing 3.0 improves integration with, and extension of, the Active Directory service to store all configuration, security, and status information. Message Queuing clients now use the Lightweight Directory Access Protocol (LDAP) to access domain controllers and global catalog servers for Message Queuing-specific information in Active Directory directly without assistance from a Message Queuing server on a domain controller. |
|
Improved Microsoft Management Console (MMC) support |
Message Queuing is now administered more completely using snap-ins hosted in MMC . Administrative tasks formerly performed in Control Panel are now performed in snap-ins. |
|
Workgroup support |
Message Queuing can be installed in workgroup mode on computers belonging to a Windows Server 2003 workgroup, rather than to a domain. In addition, a computer on which Message Queuing is installed in a workgroup can later join a domain, and then separate from the domain. |
|
Improved Active/active cluster support |
In addition to MSMQ 3.0 full support of the active/active paradigm in a server cluster, which means that Message Queuing can run on all nodes in a server cluster simultaneously, triggers in MSMQ 3.0 are also integrated with active/active cluster support. |
|
Windows CE support |
Although MSMQ 2.0 was also available on handheld and palm-sized computers running the Windows CE 3.0 or later operating system, Windows CE also now supports the Message Queuing SRMP protocol for HTTP messaging, available in MSMQ 3.0. |
|
Enhanced Windows security integration |
MSMQ 3.0, like MSMQ 2.0, leverages security features of the Windows operating system through the use of access control, auditing, encryption, and authentication, and using security features such as the Kerberos security protocol. In addition, MSMQ 3.0 provides for the use of more secured remote reading, and the use of SSL authentication for HTTP/HTTPS messaging. |
|
Workgroup support |
MSMQ 3.0 can be installed and run in a Windows Server 2003 workgroup, rather than in a domain. In addition, Message Queuing that is installed in a workgroup can later join a domain, and then separate from the domain again. |
|
Transactional messaging |
Using transactional capabilities, you can couple several related actions in a single transaction, ensure messages are delivered in order, ensure messages are delivered only once, and confirm that messages were successfully retrieved from the destination queue. |
|
Message backup and restore |
Message storage files, log files, transaction log files, and registry settings can be backed up and restored in case of computer failure. |
|
Message prioritization |
Message prioritization allows urgent or important messages to to be sent before less important messages, so you can guarantee adequate response time for critical applications at the expense of less important applications. |
|
Guaranteed message delivery |
Messages can be stored on a disk-based queue and then later forwarded to provide guaranteed delivery. |
|
Connectionless messaging |
With store-and-forward message queuing, computers are not affected by network disturbances and do not have to establish sessions. Because a sessionless model is used at the application level, the source computer and the destination computer do not need to support the same network protocol. Unlike MSMQ 2.0, MSMQ 3.0 does not support the Internet Packet Exchange (IPX) protocol. |
|
Message routing |
Message Queuing provides message routing based on the physical topology of the network, session concentration needs, and transport connectivity. Session concentration facilitates the efficient usage of slow communication links. |
|
Dynamic queue creation |
You can create queues or change queue properties as you go without affecting messaging applications. |
|
Disparate system integration |
Message Queuing can be used across a wide variety of hardware platforms using connectivity products provided by other vendors. |
|
Mixed-mode operation |
MSMQ 3.0, unlike MSMQ 2.0 does not support a mixed-mode domain environments, which means that MSMQ 1.0 controller servers running on Windows NT 4.0 cannot exist with in a Windows Server 2003 domain with Windows Server 2003 domain controllers running MSMQ 3.0. |
To me, the two most interesting features are the very first two: delivering
messages over HTTP transport, and triggers. HTTP, for all the obvious reasons,
not to forget SOAP formatting. Triggers,
because we now have an easy way to create rules and associate the arrival
of each incoming message at a queue with a response that depends on the
contents of the message. We can either invoke a COM component or
a stand-alone .NET executable program. Our .NET executable for example,
can read the message in the Queue and do all kinds of neat stuff "behind
the scenes". A third and also very interesting new feature is
message delivery to IP multicast destinations using the PGM protocol. That
can get somewhat complex, so I'll leave it for a subsequent visit. So, let's
discover how to use the first two features, HTTP and Triggers.
HTTP Messages with MSMQ 3.0:
MSMQ 3.0 provides support for referencing URL-named queues, creating SOAP-compliant
(SRMP) messages that can be sent over HTTP, and authenticating HTTPS messages.
To send messages to a destination queue over HTTP, the target computer
must have the Message Queuing HTTP Support subcomponent and IIS installed.
Access to the queue can be restricted to specific users by changing the
permissions of the virtual directory in the IIS snap-in. If you have it installed, you will see an IIS application with the name (Surprise!) "MSMQ", and it will have nothing in it.
Destination queues for HTTP messages are opened using direct
format names that include the URL address of the target computer, the virtual directory
name, and the queue name separated by slashes. The default virtual directory
name is msmq, but Message Queuing can be configured by IIS to use a different
virtual directory. A direct format name containing HTTPS as the protocol
name invokes a secure HTTP transport through a Secure Sockets Layer (SSL)
connection.
The following are some examples of direct format names for sending HTTP
messages.
DIRECT=HTTP://Mike01/msmq/MyQueue (Within the enterprise)
DIRECT=HTTP://www.northwindtraders.com/msmq/MyQueue (Over the Internet)
DIRECT=HTTPS://www.northwindtraders.com/msmq/MyQueue (Over the Internet)
HTTP is automatically chosen as the transport for any Message Queuing message
sent using an HTTP format name. In this case, the message will be formatted
using the SOAP extensions for reliable messaging in the SOAP Reliable Messaging
Protocol (SRMP).
Note You cannot read messages in a queue on a remote computer if the queue
is opened using an HTTP direct format name; you can only use HTTP to send
messages.
A complete HTTP message includes a SOAP envelope and the SOAP attachments
associated with it. Here is a sample actual SOAP Message.
When sending an HTTP message, Message Queuing automatically transforms
the individual binary message properties to specific WS-Routing elements,
SRMP elements, and elements of the MSMQ namespace, and includes them in
the header portion of the SOAP envelope. On the receiver side, Message Queuing
maps these XML elements to the respective binary message properties before
storing the message. An HTTP message maintains the XML elements specifying
the message properties in the read-only PROPID_M_SOAP_ENVELOPE or MSMQMessage.SOAPEnvelope
property.
When authentication is requested, Message Queuing automatically signs HTTP
messages using an XML digital signature, and automatically includes the
correct signature - related elements in the SOAP message. All of this is
done automatically, without you having to write a single line of code!
Setting up a Queue for HTTP access with a Trigger
In our sample project, we will make some very simple assumptions that will
enable you to get MSMQ via HTTP access with a trigger up and running as
a "learning proof of concept". We will assume that subscribers
wish to have their name and email addresses entered into your Notifications
table for further processing, and that during the course of the normal operation
of some sort of client program located outside your network, the program
will make an HTTPRequest, sending an MSMQ subscribe message that will execute
your MSMQ Trigger upon receipt. On the server, your .NET executable will
read the Message out of the Queue, cast it to an instance of the "test"
class, get the Name and Email properties, and make a SQL insert into your
Notifications table for further processing.
Follow these steps to create the infrastructure for our sample project:
- Create
a new Private Queue called "notifications"
- This should be
a Transactional Queue.
- Create a new Trigger for Peeking,
enabled, serialized, called "Notify"
- Create a new Rule
called "Notify" with a Rule Condition
of "Message
Label contains Notification"
- For the Rule Action, specify
the path to the TriggerHappy project executable (on my machine,
this is "C:\CSHARPBIN\HTTPQueueSender\TriggerHappy\bin\Debug\HTTPSender.exe"
- In
Parameters, pass the MessageId as Variant, and the MessageBody
as String.
- In your Trigger's attached rules, ensure that the
Rule Notify is attached.
- Ensure that both the MessageQueue
and the MessageQueue Trigger services are running.
- Create
a new (local) Sql Server (or MSDE) database named "TEST".
- Execute
the Sql Script "NotificationsTable.sql" in Query Analyzer to
add the Notifications table to your database
Ok. We are now ready to explore the code, and how it works. First, let's
take a look at the HTTPSender Client UI so that you can get an idea of what
this is about:
We can specify the Queue name, and using the FormatName directive, the
full HTTP endpoint is visible. For the Info to send, we will specify the
user's name and email address as explained above. The SEND button will intitiate
the sending of the Message to the queue. Under the hood, here is the code
that actually performs this action:
private void button1_Click(object sender, System.EventArgs e)
{
try
{
System.Messaging.MessageQueue mq =
new System.Messaging.MessageQueue (label5.Text);
mq.DefaultPropertiesToSend.Recoverable=true;
mq.DefaultPropertiesToSend.Label="Notification";
Test myTest =
new Test(textBox2.Text,textBox3.Text);
System.Messaging.Message msg=new System.Messaging.Message();
msg.Formatter =new System.Messaging.BinaryMessageFormatter();
msg.Body=myTest;
mq.Send(msg,"Notification",System.Messaging.MessageQueueTransactionType.Single );
}
catch (System.Messaging.MessageQueueException mqx)
{
MessageBox.Show (mqx.Message) ;
}
} |
"Test", above, is simply an instance of a simple serializable
class that has two string properties, Name and Email.
This class is included in both the Sender and the Receiver (our .NET Executable
that is run by the Trigger) so that correct serialization and deserialization
can be performed on the Message Body.
At the receiving end, recall that our Trigger is going to execute an instance
of an "HTTPSender.exe" console app that will read the Message from the
Queue, and do the SQL insert. This is, of course, just a simple surrogate
for some business process that in production would obviously be much more
complex:
static void Main(string[] args)
{
// you can use the next line for debugging purposes
// System.Diagnostics.Debugger.Launch();
object MessageId="";
object MessageBody="";
if(args.Length>0)
{
MessageId=args[0];
// note we are passing the message body too, but we don't use it here
MessageBody=args[1];
}
MessageQueue q = new MessageQueue(@".\Private$\notifications");
if(MessageId!=null)
{
string msgId=((string)MessageId).Substring(((string)MessageId).IndexOf(@"\"));
}
System.Messaging.Message msg=null;
Test tst=null;
if(MessageId!=null)
{
// need to strip off the curly brackets to use ReceiveById!
string msgid=((string)MessageId).Replace("}","");
msgid=msgid.Replace("{","");
msg= q.ReceiveById(msgid);
Type[] expectedTypes= new Type[]{typeof(string)};
msg.Formatter =new System.Messaging.BinaryMessageFormatter();
System.Diagnostics.Debug.WriteLine(msg.Id.ToString());
tst=(Test)msg.Body;
}
string sSql="Insert into Notifications (NotifyDateTime,Name,Email)";
sSql+="Values('"+System.DateTime.Now+"','"+tst.Name +"','"+tst.Email +"')";
SqlConnection cn =new SqlConnection("server=(local);database=Test;user id=sa;password=;");
cn.Open();
SqlCommand cmd=new SqlCommand(sSql,cn);
cmd.CommandType=CommandType.Text;
cmd.ExecuteNonQuery();
cmd.Dispose();
cn.Close();
}
|
With that, we have completed the entire implementation of sending an MSMQ
message over the Internet via HTTP to a remote queue, and having a Trigger
on the remote queue spawn an instance of an Executable that handles some
required business logic in the enterprise.
There is much more to MSMQ 3.0 than described here. You have Acknowledgement Queues, where you can set the Administration Queue to the queue that receives acknowledgements of messages, and the AcknowledgementType defines the acknowledgements that we are interested in . There is also a Mapping XML file that you can set for HTTP to a secondary URL where acknowledgements can be picked up. Routing of messages to other queues, even other machines, can also be set up here. You determine which acknowledgement corresponds to a sent messages with the CorrelationId and MessageQueue.ReceiveByCorrelationId method. Messages can be involved in Transactions, and they can be Encrypted. We can also turn on Tracing to get reports on what is happening to messages in an enterprise environment.
Finally, we can use Queued Components derived from System.EnterpriseServices in COM+ to make it easier to call methods across a Message Queue. However, by using this simplified model, we also lose the ability to programmatically do fine-grained Message control as in the above example.
Conclusion: MSMQ 3.0 is an Enterprise - Level tool to
handle a wide range of both disconnected and connected Messaging scenarios.
Developers should carefully study the documentation. Contrary to what some
"Managers" may have to say about it, when properly configured
and tested, MSMQ is both reliable and fast, and can save many hours of
custom programming time.
Download the Solution that accompanies this article |