How to avoid receiving same message twice in RabbitMQ

In this post, we will see how to avoid receiving / processing duplicate message more than once in RabbitMQ.

Background:
RabbitMQ is a messaging queue software (also called message broker) that implemented the Advanced Message Queuing Protocol (AMQP). The producer submits message on queue and consumer processes it. But sometimes we use to see the issue with duplicate processing of same message in case where consumer decide to acknowledge message after processing it.

Problem Statement and Solution:
Suppose there is a message that is published into message broker. There can be two scenario in case of consumers for this message type:

• Single Consumer: 
• Consumer receives the message and start processing it and dies (or hung) in between: In this case if the acknowledgement has not been send back to RabbitMQ message broker, the message will remain in queue and when the consumer starts again, it will receive the same message. 
• Solution: In the above case, developer needs to know if the business logic is processed or not. So developer needs to implement status of the message in either database or flat file with the message id. If same message is received after consumer restarts, developer can take a call to process again or not
• Consumer receives message and processed it but still it receives same message again:
• Solution: If you are facing this issue, then the problem lies in the heart-beat  timeout between consumer and RabbitMQ. So here consumer has received the message and its thread is processing it but due to missed heart-beat (due to network latency or whatever reason) RabbitMQ assumes the consumer has died and it closes the connection with consumer. Meanwhile as the consumer thread is active, it is still processing the message and completed it successfully but it was unable to acknowledge it back to RabbitMQ. The solution lies in increasing the heart-beat timeout so that it doesn't misses the heart-beat.
• Multiple Consumer:
• Consumer receives the message and start processing it and dies (or hung) in between: In this case if the acknowledgement has not been send back to RabbitMQ message broker, the message will remain in queue and it has been send back to another consumer waiting for message
• Solution: In the above case, developer needs to know if the business logic is processed or not. So developer needs to implement status of the message in either database or flat file with the message id. If same message is received after consumer restarts, developer can take a call to process again or not
• Consumer receives message and processed it but other consumer receives same message again:
• Solution: If you are facing this issue, then the problem lies in the heart-beat  timeout between consumer and RabbitMQ. So here consumer has received the message and its thread is processing it but due to missed heart-beat (due to network latency or whatever reason) RabbitMQ assumes the consumer has died and it closes the connection with consumer. Meanwhile as the consumer thread is active, it is still processing the message and completed it successfully but it was unable to acknowledge it back to RabbitMQ. The solution lies in increasing the heart-beat timeout so that it doesn't misses the heart-beat. 

Thats all about problems related to consumer receiving duplicate messages and its possible solutions. If you have query / suggestion, please feel free to post in comment section. Thanks.