Spring, RabbitMQ & Dead Letter Exchanges

RabbitMQ has become a staple for building job queues between the myriad of spring boot micro-serivces I've built at SRC:CLR. The Spring abstraction has allowed for quick and mostly painless development. What I hadn't found a need for was RabbitMQ's "Dead Letter Exchange" setup. Multiple times there had been discussions about using the dead letter pattern but I'd never gone that route. During one of my latest development sessions I decided to use the dead letter pattern and felt a quick example setting one up would be useful to share. I'll also cover some of the new Spring features that allowed me to write less code.

I started off with a blank spring boot project (which I now realize should have been a post prior to this one). I removed the extra items I didn't need and went about configuring a couple queues and setting some properties. I'll leave out some of the basic spring boot setup items but for the full code go here.

First a quick properties file:

# RABBIT
spring.rabbitmq.host=localhost
spring.rabbitmq.username=guest
spring.rabbitmq.password=guest
spring.rabbitmq.virtual-host=/

# Uncomment for full debugging
#logging.level.=DEBUG

The spring.rabbitmq. items are part of spring-boots auto-configuration setup and a full list can be found here. Those properties will load the RabbitAutoConfiguration spring-boot class which gives a set of default beans that you'd otherwise write yourself. More info on that class here.

Next we get into the RabbitMQ config:

@Autowired
private ConnectionFactory cachingConnectionFactory;

// Setting the annotation listeners to use the jackson2JsonMessageConverter
@Bean
public SimpleRabbitListenerContainerFactory rabbitListenerContainerFactory() {
  SimpleRabbitListenerContainerFactory factory = new SimpleRabbitListenerContainerFactory();
  factory.setConnectionFactory(cachingConnectionFactory);
  factory.setMessageConverter(jackson2JsonMessageConverter());
  return factory;
}

// Standardize on a single objectMapper for all message queue items
@Bean
public Jackson2JsonMessageConverter jackson2JsonMessageConverter() {
  return new Jackson2JsonMessageConverter();
}

@Bean
public Queue outgoingQueue() {
  Map<String, Object> args = new HashMap<String, Object>();
  // The default exchange
  args.put("x-dead-letter-exchange", "");
  // Route to the incoming queue when the TTL occurs
  args.put("x-dead-letter-routing-key", INCOMING_QUEUE);
  // TTL 5 seconds
  args.put("x-message-ttl", 5000);
  return new Queue(OUTGOING_QUEUE, false, false, false, args);
}

@Bean
public RabbitTemplate outgoingSender() {
  RabbitTemplate rabbitTemplate = new RabbitTemplate(cachingConnectionFactory);
  rabbitTemplate.setQueue(outgoingQueue().getName());
  rabbitTemplate.setRoutingKey(outgoingQueue().getName());
  rabbitTemplate.setMessageConverter(jackson2JsonMessageConverter());
  return rabbitTemplate;
}

@Bean
public Queue incomingQueue() {
  return new Queue(INCOMING_QUEUE);
}

The ConnectionFactory came from the auto-configuration and the SimpleRabbitListenerContainerFactory will be used for spring-rabbit's new annotations later. An important item is deciding on a message converter, I'm a fan of the Jackson2JsonMessageConverter so I added that. You'll also need to add jackson into your package manager to use this. (Note: As of writing this spring-boot only supports jackson 2.4.x and not 2.5)

The outgoingQueue is where we will set the important arguments to enable the dead-letter setup and the outgoingSender is a basic sender that can be autowired into services for quick sending to a specific queue.

• x-dead-letter-exchange - The exchange where the message will be republished. (In this case the same exchange)
• x-dead-letter-routing-key - The routing key used to route the message. Essentially the queue name.
• x-message-ttl - Defines how long the message will stay on the queue before its Time-To-Live expires and it's placed on the incoming queue.

The incomingQueue will receive the dead-lettered messages.

@Component
public class DeadLetterSendReceive {

  private static final Logger LOGGER = LoggerFactory.getLogger(DeadLetterSendReceive.class);

  @Autowired
  private RabbitTemplate outgoingSender;

  // Scheduled task to send an object every 5 seconds
  @Scheduled(fixedDelay = 5000)
  public void sender() {
    ExampleObject ex = new ExampleObject();
    LOGGER.info("Sending example object at " + ex.getDate());
    outgoingSender.convertAndSend(ex);
  }

  // Annotation to listen for an ExampleObject
  @RabbitListener(queues = MQConfig.INCOMING_QUEUE)
  public void handleMessage(ExampleObject exampleObject) {
    LOGGER.info("Received incoming object at " + exampleObject.getDate());
  }

}

Using a scheduled task I send an ExampleObject message every 5 seconds with the autowired outgoingSender. The handleMessage method will receive the messages as their TTL expires and they are placed on the incomingQueue.

The @RabbitListener annotation is new in spring-rabbit as of version 1.4. Previously I'd need to setup around 4 beans to receive messages, with this new annotation it's reduced to declaring the queue and setting a global factory to use the proper message convertor. Once those are set you're free to add the annotation in your project.

Running this code you will see log messages like the ones below (colored logs from spring-boot are awesome!):

INFO 35325 --- [pool-4-thread-1] v.c.deadletter.mq.DeadLetterSendReceive  : Sending example object at Fri Apr 24 08:35:41 PDT 2015
INFO 35325 --- [cTaskExecutor-1] v.c.deadletter.mq.DeadLetterSendReceive  : Received incoming object at Fri Apr 24 08:35:36 PDT 2015
INFO 35325 --- [pool-4-thread-1] v.c.deadletter.mq.DeadLetterSendReceive  : Sending example object at Fri Apr 24 08:35:46 PDT 2015
INFO 35325 --- [cTaskExecutor-1] v.c.deadletter.mq.DeadLetterSendReceive  : Received incoming object at Fri Apr 24 08:35:41 PDT 2015

Full code can be found here: https://github.com/cl4r1ty/spring-rabbitmq-dead-letter

You'll need RabbitMQ installed and running. The easiest way to do that is brew install rabbitmq then rabbitmq-server.

Once RabbitMQ is up from the root of the repo quickly run the example with the spring-boot maven plugin command mvn spring-boot:run.

Now you've got a quick setup for a dead-letter pattern using spring-rabbit which is much less code than RabbitMQ's Java implementation.