My Favorite NServiceBus Features - Transports
When anyone first considers building a message based system, one of the first things they look at is how to connect to a queuing technology to produce or consume messages. These days that is a relatively easy task to put together in a proof-of-concept project. RabbitMQ, Azure Service Bus and AWS SQS/SNS all make it easy to build a relatively simple single threaded solution.
Simple single threaded solutions are rarely the end goal for moving to a message based architecture though. If single threaded was acceptable, then you would have stayed with a synchronous and temporally coupled http API. The primary attributes that make the complexity of a message based system is the ability to elastically scale your solution and process your messages in parallel thread simultaneously.
Once you move on from the simple POC and start implementing threading, multiple instances, batching routing and more complex messaging patterns you will start to realize that there is a lot of complexity involved in getting it right. Keeping the business logic abstracted from the nuances of the queuing technology will be difficult and you will have a tendency to lean heavy on the features of the queuing technology.
Leaning heavy on the queuing technology may not seem like an issue initially, but if you need to switch queuing technology or the queuing technology fundamentally changes due to an upgrade, you are in trouble. Beyond the switching costs, you will end up with smart pipes and dumb endpoints which is the inverse of what you want. Business logic that creeps into your queueing technology will cause pain as it is difficult to automate tests, automate deployments and fixing issues will become increasingly difficult.
One of the foundational features of NServiceBus is the support of multiple queuing technologies known in NServiceBus parlance as a Transport. NServiceBus transports encapsulate the logic required to connect to their respective queue technologies to produce and consume messages.
Advantages of Transport Abstraction
Having an abstracted set of NServiceBus Transports available has a number of advantages.
Each queuing technology has it's own set of nuances around concurrency, transactions and scale that are encapsulated in the transport. The transport abstraction keeps your business logic separated from your business logic and from the other messaging patterns built into NServiceBus. In the code examples lower in this article, you can see how the business logic is separate from the transport.
With the encapsulation provided by the transport, your solution is not tightly coupled with a queue technology and encourages smart endpoints with dumb pipes. Having smart endpoints means you can better test your business logic in isolation and automated deployments become much easier and cleaner.
Switching queue technology, if necessary, is feasible with a minimum impact to your business logic. This includes the ability to quickly swap in a local development queue technology for development testing. For queuing technologies that do not have an option to run on your local machine, this is a must have option.
Even if you are convinced that you picked the best ever queuing technology and will never have a need to switch queuing technology, consider that over time the queuing technology will change. Most likely it will change for the better, but having options to move to a different queuing technology or make that upgrade that requires breaking changes will be important to you at that time.
Available Transports
There are a variety of NServiceBus transports that you can plug into your NServiceBus implementation. The transports can be categorized into three types that include Particular Supported Transports, Community Supported Transports and Custom Transports.
Particular Supported Transports
- Learning (file system)
- MSMQ
- Azure Service Bus
- Azure Service Bus (Legacy)
- Azure Storage Queues
- SQL Server
- RabbitMQ
- Amazon SQS
Community Supported Transports
There are also a number of community supported transports that are available for NServiceBus. Community transports are open sourced transports that are not supported by NServiceBus but are available for your implementation if you are using one of the less popular transport options.
Custom Transports
You also have the option to build your own transport abstraction by implementing the NServiceBus IDispatchMessages and IPushMessages interfaces. The NServiceBus documentation provides a good example of how to create a custom transport. This is a nice option if your organization is using a messaging technology that is not one of the mainstream options or you just want to understand how a transport works.
Transports In Action
To get a better understanding of how transports are configured in your NServiceBus endpoints and how the transport keeps things abstracted, I have included a few examples below.
Simple Transport Configuration Example
In this first example, we see where we add the transport to the endpoint with the UseTransport<>() method. In this case we are leveraging the Learning Transport.
endpointConfiguration.UseTransport<LearningTransport>();
The Learning Transport uses your local file system to simulate a queuing technology and is a good starting point to learn how to use NServiceBus or for local development since there is no setup or any special requirements to run the transport.
Configuring RabbitMQ Transport
In this example the RabbitMQ Transport is added to the endpoint. As you can see there are additional details like the connection string and the routing topology that need to be configured for RabbitMQ.
var transport = endpointConfiguration.UseTransport<RabbitMQTransport>();
transport.UseConventionalRoutingTopology();
transport.ConnectionString("host=localhost");
Regardless of the transport, the configuration of the transport is encapsulated in the endpoint configuration section of your application. In fact, it is relatively easy to add a conditional statement that loads the learning transport when you are running on your local machine and the RabbitMQ transport when deployed to a server.
Note: An endpoint is configured for one transport at a time. Attempting to configure an endpoint to run the LearningTransport and RabbitMQ at the same time is not going to work.
Message Handler
In this message handler, we see that there is no direct reference to the transport. This message handler essentially works the same if we are using the Learning Transport or the RabbitMQ Transport for example. The Handler will be invoked when a message of type MyAwesomeCommand is consumed from the queue and if we needed, we could send a message using the IMessageHandlerContext.
public class GenericAsyncHandler :
IHandleMessages<MyAwesomeCommand>
{
static ILog log = LogManager.GetLogger<GenericAsyncHandler>();
public Task Handle(MyAwesomeCommand message, IMessageHandlerContext context)
{
log.Info($"Received a message of type {message.GetType().Name}.");
return SomeLibrary.SomeAsyncMethod(message);
}
}
Sending message from a Controller
Similarly, if we have a ASP.NET Web API controller that needs to send a message, the IMessageSession can be injected into the controller and then used to send the message.
public SendMessageController(IMessageSession messageSession)
{
this.messageSession = messageSession;
}
...
[HttpGet]
public async Task<string> Get()
{
var message = new MyMessage();
await messageSession.Send(message)
.ConfigureAwait(false);
return "Message sent to endpoint";
}
Switching Transports
While switching transports in your NServiceBus implementation is possible, I do not want to give the impression that it is trivial to do so. If you have ever considered switching your ORM to target a different relational database you will understand why it is not trivial. When you switch the transport, most things will work the way you expect, but there will inevitably be assumptions based on your queuing technology that have crept into your implementation. Some of these assumptions will be around things like distributed transactions, routing and performance tuning.
For example, if you are using MSMQ as your queuing technology, then you have Distributed Transaction Coordinator (DTC) by default. This means your messages and database interactions are encapsulated in a distributed transaction. When you move from MSMQ to a transport like RabbitMQ that does not support DTC, suddenly the protections you may not have realized were there are gone. You will need to incorporate outbox or some other strategy to ensure transnational integrity.
While it is not trivial to switch transports, it is important to understand that the transport abstraction that NServiceBus provides makes it feasible to switch by providing a clean abstraction layer and tools for switching like routers.
If you had created your own messaging framework there is a high probability that much more of the queueing technology assumptions have been intermingled with your business logic. In addition, all of the hard work you put into building your framework from scratch will need to be duplicated from the ground up on your new transport.
Also keep in mind that there is documentation and assistance from consultants who have been through the process of switching transports available when switching NServiceBus transports.
Transport Migration
There are two strategies for moving from one transport to another. The first method is to do a big bang switch to the new transport which requires draining down your queues and multi-hour outage. If the outage is acceptable and/or you are in a hurry, this is probably the way to go. If you have more time to work with or an outage is not acceptable, then NServiceBus Router provides the ability to migrate endpoints to the new transport over time.
Router essentially allows you to run your system across multiple transports simultaneously. This is a complex subject that I will not get into depth in this article. I bring this subject up only to make you aware that it is an option for migrating to a new transport.
In Summary
NServiceBus transports are a fundamental feature in NServiceBus. They provide a layer of abstraction that keeps your business logic isolated from the queuing technology. This abstraction will enable you to create a solution that is much easier to develop, test and deploy. In addition, the NServiceBus transports provide you with options in the future when aspects of your queuing technology will inevitably change. This is why it is one of my favorite NServiceBus features.