Rabbitmq cluster setup with HAproxy

Install RabbitMQ

Use the following steps to install RabbitmQ

http://www.cloudkb.net/rabbitmq-installation-centos/

Create the RabbiMQ Cluster

Clustering connects multiple nodes to form a single logical broker. Virtual hosts, exchanges, users and permissions are mirrored across all nodes in a cluster. A client connecting to any node can see all the queues in a cluster.

Clustering enables high availability of queues and increases the throughput.

A node can be a Disc node or RAM node. RAM node keeps the message state in memory with the exception of queue contents which can reside on a disk if the queue is persistent or too big to fit into memory.

RAM nodes perform better than Disc nodes because they don’t have to write to a disk as much as disk nodes. But, it is always recommended to have disk nodes for persistent queues.

We have used the following servers as Master and slave for Rabbitmq cluster setup with HAproxy.

rabbitmaster = 192.168.1.182
rabbitslave = 192.168.1.185
haproxy = 192.168.1.188
Stop RabbitMQ in Master and slave nodes and make sure service is stopped properly.

[[email protected] ~]# /etc/init.d/rabbitmq-server stop
Copy the file below to all nodes from the master. This cookie file needs to be the same across all nodes.

[[email protected] ~]# cat /var/lib/rabbitmq/.erlang.cookie
copy .erlang.cookie file to your nodes

Start RabbitMQ in master and all nodes.

[[email protected] ~]# /etc/init.d/rabbitmq-server start

Then run the following commands in all the nodes, except the master node:

[[email protected] ~]# rabbitmqctl stop_app

[[email protected] ~]# rabbitmqctl reset

[[email protected] ~]# rabbitmqctl start_app
Now, run the following commands in the master node:

[[email protected] ~]# rabbitmqctl stop_app

[[email protected] ~]# rabbitmqctl reset
Use the following command is executed to join the slaves to the cluster.

[[email protected] ~]# rabbitmqctl join_cluster [email protected]
Clustering node [email protected] with [email protected] …
…done.
Check the cluster status after execute it

[[email protected] ~]# rabbitmqctl cluster_status
Cluster status of node [email protected] …
[{nodes,[{disc,[[email protected],[email protected]]}]}] …done.
Setup HA Policy for your all cluster nodes

[[email protected] ~]# rabbitmqctl start_app
Starting node [email protected] …
…done.
[[email protected] ~]# rabbitmqctl set_policy ha-all “” ‘{“ha-mode”:”all”,”ha-sync-mode”:”automatic”}’
Setting policy “ha-all” for pattern [] to “{\”ha-mode\”:\”all\”,\”ha-sync-mode\”:\”automatic\”}” with priority “0” …
…done.

Install Management Plugin

The rabbitmq-management plugin provides an HTTP-based API for management and monitoring of your RabbitMQ server, along with a browser-based UI and a command line tool.
[[email protected] ~]# rabbitmq-plugins enable rabbitmq_management
The following plugins have been enabled:
mochiweb
webmachine
rabbitmq_web_dispatch
amqp_client
rabbitmq_management_agent
rabbitmq_management
Plugin configuration has changed. Restart RabbitMQ for changes to take effect.

 

Reset RabbitMQ admin password

#rabbitmqctl add_user demo demo
#rabbitmqctl set_user_tags demo administrator
#rabbitmqctl set_permissions -p / demo “.*” “.*” “.*”
Once it will be finished, connect web UI at: http://server-name:15672/

create test queues.

Go to your slave server and check that queue

[[email protected] ~]# rabbitmqctl list_queues
Listing queues …
test 0
…done.

HAproxy

HAProxy is a free, very fast and reliable solution offering high availability, load balancing, and proxying for TCP and HTTP-based applications. Install haproxy on your main server or any other server which you want.

Create a load balance in front of it and map the backend RabbitMQ instance.
#yum install haproxy

Edit /etc/haproxy/haproxy.cfg and create a simple TCP proxy for RabbitMQ.

#vi /etc/haproxy/haproxy.cfg

global
daemon

defaults
mode tcp
maxconn 10000
timeout connect 5s
timeout client 100s
timeout server 100s

listen rabbitmq 192.168.1.188:5670
mode tcp
balance roundrobin
server rabbitmaster 192.168.1.182:5672 check inter 5s rise 2 fall 3
server rabbitslave 192.168.1.185:5672 check inter 5s rise 2 fall 3

[[email protected] ~]# service haproxy restart
This setup will listen rabbitmq (192.168.1.182, 192.168.1.185 ) on port 5670 with ip address 192.168.1.188.

You have done RabbitMQ clustering with HAproxy configuration!!
If possible, you can setup on each of the RabbitMQ cluster nodes configuration and restart rabbitmq.

Edit /etc/rabbitmq/rabbitmq.config
[{rabbit,
[{cluster_nodes, {[‘[email protected]’, ‘[email protected]’], ram}}]}].

Errors :

If you receive these errors, start_app and then try to set ha policy.

[[email protected] ~]# rabbitmqctl set_policy ha-all “” ‘{“ha-mode”:”all”,”ha-sync-mode”:”automatic”}’
Setting policy “ha-all” for pattern [] to “{\”ha-mode\”:\”all\”,\”ha-sync-mode\”:\”automatic\”}” with priority “0” …
Error: {badarg,[{ets,lookup,[rabbit_registry,{runtime_parameter,policy}]},
{rabbit_registry,lookup_module,2},
{rabbit_runtime_parameters,lookup_component,1},
{rabbit_runtime_parameters,set_any0,5},
{rabbit_runtime_parameters,set_any,5},
{rpc,’-handle_call_call/6-fun-0-‘,5}]}

# rabbitmqctl start_app