签到成功

知道了

CNDBA社区CNDBA社区

RocketMQ5.1 DLedger Controller independent模式安装

2023-03-10 23:26 2591 3 原创 RocketMQ
作者： hbhe0316
本文链接：http://www.cndba.cn/hbhe0316/article/116481

在 DLedger 模式下，利用 Raft Commitlog 代替了原来的 Commitlog 了，使得 Commmitlog 具备了选举的能力，当 Master Broker 故障后，通过内部协商，从其他的 Slave Broker 中选出新的 Master，完成主备切换，同时 Raft 的算法也保证了 Commitlog 的一致性。但是存在一些缺点：http://www.cndba.cn/hbhe0316/article/116481

http://www.cndba.cn/hbhe0316/article/116481

1.想要具备选举切换的能力，单组 Broker 内的副本数必须 3 副本及以上(Raft协议决定)
2.副本 ACK 需要严格遵循 Raft 协议多数派的限制，3 副本需要 2 副本 ACK 后才能返回，5 副本需要 3 副本 ACK 后才能返回，副本越多可能耗时也可能越长。(这个也是最重要的一点)
3.DLedger 模式下，由于存储库使用了 OpenMessaging DLedger 存储，因此无法复用 RocketMQ 原生的存储和复制的能力（比如 transientStorePool 和零拷贝能力），且对维护造成了困难。

在RocketMQ5.0版本新增了DLedger Controller模式来解决上面对的痛点。

1.配置node01节点controller节点的配置文件http://www.cndba.cn/hbhe0316/article/116481

[root@node01 ~]# cd /opt/rocketmq/conf/controller/cluster-3n-independent/
[root@node01 cluster-3n-independent]# cat controller-n0.conf | grep -v "#"
controllerDLegerGroup = group1
controllerDLegerPeers = n0-127.0.0.1:9877;n1-127.0.0.1:9878;n2-127.0.0.1:9879
controllerDLegerSelfId = n0
controllerStorePath = /tmp/DledgerController
enableElectUncleanMaster = false
notifyBrokerRoleChanged = true


[root@node01 cluster-3n-independent]# cat controller-n1.conf | grep -v "#"
controllerDLegerGroup = group1
controllerDLegerPeers = n0-127.0.0.1:9877;n1-127.0.0.1:9878;n2-127.0.0.1:9879
controllerDLegerSelfId = n1
controllerStorePath = /tmp/DledgerController
enableElectUncleanMaster = false
notifyBrokerRoleChanged = true

[root@node01 cluster-3n-independent]# cat controller-n2.conf | grep -v "#"
controllerDLegerGroup = group1
controllerDLegerPeers = n0-127.0.0.1:9877;n1-127.0.0.1:9878;n2-127.0.0.1:9879
controllerDLegerSelfId = n2
controllerStorePath = /tmp/DledgerController
enableElectUncleanMaster = false
notifyBrokerRoleChanged = true

参数说明：
enableControllerInNamesrv：Nameserver 中是否开启 controller，默认 false。
controllerDLegerGroup：DLedger Raft Group 的名字，同一个 DLedger Raft Group 保持一致即可。
controllerDLegerPeers：DLedger Group 内各节点的端口信息，同一个 Group 内的各个节点配置必须要保证一致。
controllerDLegerSelfId：节点 id，必须属于 controllerDLegerPeers 中的一个；同 Group 内各个节点要唯一。
controllerStorePath：controller 日志存储位置。controller 是有状态的，controller 重启或宕机需要依靠日志来恢复数据，该目录非常重要，不可以轻易删除。
enableElectUncleanMaster：是否可以从 SyncStateSet 以外选举 Master，若为 true，可能会选取数据落后的副本作为 Master 而丢失消息，默认为 false。
notifyBrokerRoleChanged：当 Broker 副本组上角色发生变化时是否主动通知，默认为 true。http://www.cndba.cn/hbhe0316/article/116481

http://www.cndba.cn/hbhe0316/article/116481

2.启动node01节点controller进程

[root@node01 cluster-3n-namesrv-plugin]# cd /opt/rocketmq
[root@node01 rocketmq]# nohup bin/mqcontroller -c ./conf/controller/cluster-3n-independent/controller-n0.conf  &
[root@node01 rocketmq]# nohup bin/mqcontroller -c ./conf/controller/cluster-3n-independent/controller-n1.conf  &
[root@node01 rocketmq]# nohup bin/mqcontroller -c ./conf/controller/cluster-3n-independent/controller-n2.conf  &

3.启动node01节点nameserver

[root@node01 cluster-3n-namesrv-plugin]# cd /opt/rocketmq
[root@node01 rocketmq]# nohup sh bin/mqnamesrv &

4.配置node01节点brocker节点conf文件

[root@node01 rocketmq]# cd /opt/rocketmq/conf/controller/quick-start
[root@node01 quick-start]# cat broker-n0.conf | grep -v "#"
brokerClusterName = DefaultCluster
brokerName = broker-a
brokerId = -1
brokerRole = SLAVE
deleteWhen = 04
fileReservedTime = 48
enableControllerMode = true
controllerAddr = 127.0.0.1:9877;127.0.0.1:9878;127.0.0.1:9879 
namesrvAddr = 192.168.56.100:9876;192.168.56.101:9876
allAckInSyncStateSet=true
listenPort=30911
storePathRootDir=/tmp/rmqstore/node00
storePathCommitLog=/tmp/rmqstore/node00/commitlog

[root@node01 quick-start]# cat broker-n1.conf | grep -v "#"
brokerClusterName = DefaultCluster
brokerName = broker-a
brokerId = -1
brokerRole = SLAVE
deleteWhen = 04
fileReservedTime = 48
enableControllerMode = true
controllerAddr = 127.0.0.1:9877;127.0.0.1:9878;127.0.0.1:9879 
namesrvAddr = 192.168.56.100:9876;192.168.56.101:9876
allAckInSyncStateSet=true
listenPort=30921
storePathRootDir=/tmp/rmqstore/node01
storePathCommitLog=/tmp/rmqstore/node01/commitlog

[root@node01 quick-start]# cat broker-n2.conf | grep -v "#"
brokerClusterName = DefaultCluster
brokerName = broker-a
brokerId = -1
brokerRole = SLAVE
deleteWhen = 04
fileReservedTime = 48
enableControllerMode = true
controllerAddr = 127.0.0.1:9877;127.0.0.1:9878;127.0.0.1:9879 
namesrvAddr = 192.168.56.100:9876;192.168.56.101:9876
allAckInSyncStateSet=true
listenPort=30931
storePathRootDir=/tmp/rmqstore/node02
storePathCommitLog=/tmp/rmqstore/node02/commitlog

Broker搭配Controller DLedger模式可以实现高可用以及主备自动切换。 Broker的部署和之前的版本相同只是增加了一些与Controller DLedger相关配置：

http://www.cndba.cn/hbhe0316/article/116481

enableControllerMode：Broker controller 模式的总开关，只有该值为 true，自动主从切换模式才会打开。默认为 false。
controllerAddr：controller 的地址，多个 controller 中间用分号隔开。例如controllerAddr = 127.0.0.1:9877;127.0.0.1:9878;127.0.0.1:9879
syncBrokerMetadataPeriod：向 controller 同步 Broker 副本信息的时间间隔。默认 5000（5s）。
checkSyncStateSetPeriod：检查 SyncStateSet 的时间间隔，检查 SyncStateSet 可能会 shrink SyncState。默认5000（5s）。
syncControllerMetadataPeriod：同步 controller 元数据的时间间隔，主要是获取 active controller 的地址。默认10000（10s）。
haMaxTimeSlaveNotCatchup：表示 Slave 没有跟上 Master 的最大时间间隔，若在 SyncStateSet 中的 slave 超过该时间间隔会将其从 SyncStateSet 移除。默认为 15000（15s）。
storePathEpochFile：存储 epoch 文件的位置。epoch 文件非常重要，不可以随意删除。默认在 store 目录下。
allAckInSyncStateSet：若该值为 true，则一条消息需要复制到 SyncStateSet 中的每一个副本才会向客户端返回成功，可以保证消息不丢失。默认为 false。
syncFromLastFile：若 slave 是空盘启动，是否从最后一个文件进行复制。默认为 false。
asyncLearner：若该值为 true，则该副本不会进入 SyncStateSet，也就是不会被选举成 Master，而是一直作为一个 learner 副本进行异步复制。默认为false。
inSyncReplicas：需保持同步的副本组数量，默认为1，allAckInSyncStateSet=true 时该参数无效。
minInSyncReplicas：最小需保持同步的副本组数量，若 SyncStateSet 中副本个数小于 minInSyncReplicas 则 putMessage 直接返回 PutMessageStatus.IN_SYNC_REPLICAS_NOT_ENOUGH，默认为1。http://www.cndba.cn/hbhe0316/article/116481

5.启动node01节点的broker进程

[root@node01 rocketmq]# cd /opt/rocketmq
[root@node01 rocketmq]# nohup sh bin/mqbroker  -c ./conf/controller/quick-start/broker-n0.conf &
[root@node01 rocketmq]# nohup sh bin/mqbroker  -c ./conf/controller/quick-start/broker-n1.conf &
[root@node01 rocketmq]# nohup sh bin/mqbroker  -c ./conf/controller/quick-start/broker-n2.conf &

6.配置node02节点controller的配置文件

[root@node02 ~]# cd /opt/rocketmq/conf/controller/cluster-3n-independent/
[root@node02 cluster-3n-independent]# cat controller-n0.conf | grep -v "#"
controllerDLegerGroup = group1
controllerDLegerPeers = n0-127.0.0.1:9878;n1-127.0.0.1:9868;n2-127.0.0.1:9858
controllerDLegerSelfId = n0
controllerStorePath = /tmp/DledgerController
enableElectUncleanMaster = false
notifyBrokerRoleChanged = true

[root@node02 cluster-3n-independent]# cat controller-n1.conf | grep -v "#"
controllerDLegerGroup = group1
controllerDLegerPeers = n0-127.0.0.1:9878;n1-127.0.0.1:9868;n2-127.0.0.1:9858
controllerDLegerSelfId = n1
controllerStorePath = /tmp/DledgerController
enableElectUncleanMaster = false
notifyBrokerRoleChanged = true

[root@node02 cluster-3n-independent]# cat controller-n2.conf | grep -v "#"
controllerDLegerGroup = group1
controllerDLegerPeers = n0-127.0.0.1:9878;n1-127.0.0.1:9868;n2-127.0.0.1:9858
controllerDLegerSelfId = n2
controllerStorePath = /tmp/DledgerController
enableElectUncleanMaster = false
notifyBrokerRoleChanged = true

7.启动node02节点controller进程

http://www.cndba.cn/hbhe0316/article/116481
http://www.cndba.cn/hbhe0316/article/116481

[root@node02 cluster-3n-namesrv-plugin]# cd /opt/rocketmq
[root@node02 rocketmq]# nohup bin/mqcontroller -c ./conf/controller/cluster-3n-independent/controller-n0.conf  &
[root@node02 rocketmq]# nohup bin/mqcontroller -c ./conf/controller/cluster-3n-independent/controller-n1.conf  &
[root@node02 rocketmq]# nohup bin/mqcontroller -c ./conf/controller/cluster-3n-independent/controller-n2.conf  &

8.启动node02节点nameserver
启动nameserver

[root@node02 cluster-3n-namesrv-plugin]# cd /opt/rocketmq
[root@node02 rocketmq]# nohup sh bin/mqnamesrv &

9.配置node02节点brocker节点conf文件

[root@node02 rocketmq]# cd /opt/rocketmq/conf/controller/quick-start
[root@node02 quick-start]# cat broker-n0.conf | grep -v "#"
brokerClusterName = DefaultCluster
brokerName = broker-b
brokerId = -1
brokerRole = SLAVE
deleteWhen = 04
fileReservedTime = 48
enableControllerMode = true
controllerAddr = 127.0.0.1:9877;127.0.0.1:9878;127.0.0.1:9879
namesrvAddr = 192.168.56.100:9876;192.168.56.101:9876
allAckInSyncStateSet=true
listenPort=30911
storePathRootDir=/tmp/rmqstore/node00
storePathCommitLog=/tmp/rmqstore/node00/commitlog

[root@node02 quick-start]# cat broker-n1.conf | grep -v "#"
brokerClusterName = DefaultCluster
brokerName = broker-b
brokerId = -1
brokerRole = SLAVE
deleteWhen = 04
fileReservedTime = 48
enableControllerMode = true
controllerAddr = 127.0.0.1:9877;127.0.0.1:9878;127.0.0.1:9879
namesrvAddr = 192.168.56.100:9876;192.168.56.101:9876
allAckInSyncStateSet=true
listenPort=30921
storePathRootDir=/tmp/rmqstore/node01
storePathCommitLog=/tmp/rmqstore/node01/commitlog
[root@node02 quick-start]# 

[root@node02 quick-start]# cat broker-n2.conf | grep -v "#"
brokerClusterName = DefaultCluster
brokerName = broker-b
brokerId = -1
brokerRole = SLAVE
deleteWhen = 04
fileReservedTime = 48
enableControllerMode = true
controllerAddr = 127.0.0.1:9877;127.0.0.1:9878;127.0.0.1:9879
namesrvAddr = 192.168.56.100:9876;192.168.56.101:9876
allAckInSyncStateSet=true
listenPort=30931
storePathRootDir=/tmp/rmqstore/node02
storePathCommitLog=/tmp/rmqstore/node02/commitlog

10.启动node02节点的broker进程

[root@node02 rocketmq]# cd /opt/rocketmq
[root@node02 rocketmq]# nohup sh bin/mqbroker  -c ./conf/controller/quick-start/broker-n0.conf &
[root@node02 rocketmq]# nohup sh bin/mqbroker  -c ./conf/controller/quick-start/broker-n1.conf &
[root@node02 rocketmq]# nohup sh bin/mqbroker  -c ./conf/controller/quick-start/broker-n2.conf &

11.node01 节点启动脚本
启动脚本编写

[root@node01 quick-start]# cd /opt/
[root@node01 opt]# cat start.sh 
#!/bin/sh
cd /opt/rocketmq
nohup sh bin/mqnamesrv &
sh bin/controller/fast-try-independent-deployment.sh start
sleep 10
nohup sh bin/mqbroker  -c ./conf/controller/quick-start/broker-n0.conf &
sleep 10
nohup sh bin/mqbroker  -c ./conf/controller/quick-start/broker-n1.conf &
sleep 10
nohup sh bin/mqbroker  -c ./conf/controller/quick-start/broker-n2.conf &

12.node02 节点启动脚本

[root@node02 quick-start]# cd /opt/
[root@node02 opt]# cat start.sh 
#!/bin/sh
cd /opt/rocketmq
nohup sh bin/mqnamesrv &
sh bin/controller/fast-try-independent-deployment.sh start
sleep 10
nohup sh bin/mqbroker  -c ./conf/controller/quick-start/broker-n0.conf &
sleep 10
nohup sh bin/mqbroker  -c ./conf/controller/quick-start/broker-n1.conf &
sleep 10
nohup sh bin/mqbroker  -c ./conf/controller/quick-start/broker-n2.conf &

13.node01节点停止脚本http://www.cndba.cn/hbhe0316/article/116481

[root@node01 opt]#   cat stop.sh 
#!/bin/sh
ps -ef | grep java | grep -v grep | awk -F ' ' {'print $2'} | xargs kill -9
ps -ef | grep java | grep -v grep | awk -F ' ' {'print $2'} | xargs kill -9
ps -ef | grep java | grep -v grep | awk -F ' ' {'print $2'} | xargs kill -9

14.node02节点停止脚本

[root@node02 opt]#   cat stop.sh 
#!/bin/sh
ps -ef | grep java | grep -v grep | awk -F ' ' {'print $2'} | xargs kill -9
ps -ef | grep java | grep -v grep | awk -F ' ' {'print $2'} | xargs kill -9
ps -ef | grep java | grep -v grep | awk -F ' ' {'print $2'} | xargs kill -9

版权声明：本文为博主原创文章，未经博主允许不得转载。

MQ

用户评论
* 以下用户言论只代表其个人观点，不代表CNDBA社区的观点或立场
hbhe0316

hbhe0316

关注

1.只有承认无知，才能装下新的东西； 2.进步来自一点点滴滴的积累； 3.广博让你更优秀，而专业让你无法替代； 4.挫折和失败能够转换为一种财富。

  • 889
    原创
  • 1
    翻译
  • 13
    转载
  • 24
    评论
  • 访问：1141671次
  • 积分：1523
  • 等级：核心会员
  • 排名：第6名
精华文章

    达梦数据库运维实战

    Oracle 18c 必须掌握的新特性：管理与实战

    Oracle数据库问题解决方案和故障排除手册
    最新问题
    查看更多+
    热门文章
      热门用户
      推荐用户
        友情链接： CNDBA社区 | openGauss | openEuler | openAnolis | 广告联系: database@ustc.edu
        Copyright © 2016 All Rights Reserved. Powered by CNDBA · 皖ICP备2022006297号-1·

        QQ交流群

        注册联系QQ