openGauss cm_ctl: SSL verify certificate failed 错误 解决方法
作者:
dave
1故障现象
搭建一主两从环境,结果第二个从节点一直无法启动:
[dave@www.cndba.cn ~]$ gs_om -t status --detail
[ CMServer State ]
node node_ip instance state
-------------------------------------------------------------------------------
cm_ctl: cm_cs_ssl_send:1559 errCode:38,errMsg:ssl connection is closed, reason: 6.
cm_ctl: type is 0: shutdown SSL failed.
1 oracle 192.168.56.105 1 /data/openGauss/data/cmserver/cm_server Primary
2 oracle2 192.168.56.106 2 /data/openGauss/data/cmserver/cm_server Standby
3 oracle3 192.168.56.107 3 /data/openGauss/data/cmserver/cm_server Down
[ Cluster State ]
cluster_state : Degraded
redistributing : No
balanced : No
current_az : AZ_ALL
[ Datanode State ]
node node_ip instance state
---------------------------------------------------------------------------------
1 oracle 192.168.56.105 6001 /data/openGauss/install/data/dn P Standby Normal
2 oracle2 192.168.56.106 6002 /data/openGauss/install/data/dn S Primary Normal
3 oracle3 192.168.56.107 6003 /data/openGauss/install/data/dn S Down Unknown
[dave@www.cndba.cn ~]$
手工重启也不行:
[dave@www.cndba.cn3 ~]$ gs_om -t start
Starting cluster.
======================================================================
[GAUSS-51607] : Failed to start cluster. Error:
cm_ctl: checking cluster status.
cm_ctl: checking cluster status.
cm_ctl: checking finished in 974 ms.
cm_ctl: start cluster.
cm_ctl: start nodeid: 1
cm_ctl: start nodeid: 2
cm_ctl: start nodeid: 3
cm_ctl: SSL verify certificate failed: result code is 9, certificate is not yet validcm_ctl: create ssl connection failed.
cm_ctl: create ssl connection failed.
cm_ctl: socket is [4], 442 : create ssl failed: message contents do not agree with length in message type "S"
.
cm_ctl: SSL verify certificate failed: result code is 9, certificate is not yet validcm_ctl: create ssl connection failed.
cm_ctl: create ssl connection failed.
cm_ctl: socket is [4], 442 : create ssl failed: message contents do not agree with length in message type "S"
.
cm_ctl: SSL verify certificate failed: result code is 9, certificate is not yet validcm_ctl: create ssl connection failed.
cm_ctl: create ssl connection failed.
cm_ctl: socket is [4], 442 : create ssl failed: message contents do not agree with length in message type "S"
.
cm_ctl: SSL verify certificate failed: result code is 9, certificate is not yet validcm_ctl: create ssl connection failed.
cm_ctl: create ssl connection failed.
cm_ctl: socket is [4], 442 : create ssl failed: message contents do not agree with length in message type "S"
..
cm_ctl: SSL verify certificate failed: result code is 9, certificate is not yet validcm_ctl: create ssl connection failed.
cm_ctl: create ssl connection failed.
cm_ctl: socket is [4], 442 : create ssl failed: message contents do not agree with length in message type "S"
.
cm_ctl: SSL verify certificate failed: result code is 9, certificate is not yet validcm_ctl: create ssl connection failed.
cm_ctl: create ssl connection failed.
cm_ctl: cm_ctl can't establish an SSL connection, please check certificate file
The cluster may continue to start in the background.
If you want to see the cluster status, please try command gs_om -t status.
If you want to stop the cluster, please try command gs_om -t stop.
[GAUSS-51607] : Failed to start cluster. Error:
cm_ctl: checking cluster status.
cm_ctl: checking cluster status.
cm_ctl: checking finished in 974 ms.
cm_ctl: start cluster.
cm_ctl: start nodeid: 1
cm_ctl: start nodeid: 2
cm_ctl: start nodeid: 3
cm_ctl: SSL verify certificate failed: result code is 9, certificate is not yet validcm_ctl: create ssl connection failed.
cm_ctl: create ssl connection failed.
cm_ctl: socket is [4], 442 : create ssl failed: message contents do not agree with length in message type "S"
.
cm_ctl: SSL verify certificate failed: result code is 9, certificate is not yet validcm_ctl: create ssl connection failed.
cm_ctl: create ssl connection failed.
cm_ctl: socket is [4], 442 : create ssl failed: message contents do not agree with length in message type "S"
.
cm_ctl: SSL verify certificate failed: result code is 9, certificate is not yet validcm_ctl: create ssl connection failed.
cm_ctl: create ssl connection failed.
cm_ctl: socket is [4], 442 : create ssl failed: message contents do not agree with length in message type "S"
.
cm_ctl: SSL verify certificate failed: result code is 9, certificate is not yet validcm_ctl: create ssl connection failed.
cm_ctl: create ssl connection failed.
cm_ctl: socket is [4], 442 : create ssl failed: message contents do not agree with length in message type "S"
..
cm_ctl: SSL verify certificate failed: result code is 9, certificate is not yet validcm_ctl: create ssl connection failed.
cm_ctl: create ssl connection failed.
cm_ctl: socket is [4], 442 : create ssl failed: message contents do not agree with length in message type "S"
.
cm_ctl: SSL verify certificate failed: result code is 9, certificate is not yet validcm_ctl: create ssl connection failed.
cm_ctl: create ssl connection failed.
cm_ctl: cm_ctl can't establish an SSL connection, please check certificate file
2分析过程
这里主要错误提示是SSL 认证失败:
cm_ctl: SSL verify certificate failed: result code is 9, certificate is not yet validcm_ctl: create ssl connection failed.
但手工验证SSH 通信,是没有问题:
[dave@www.cndba.cn ~]$ ssh oracle date
Fri Apr 7 11:53:56 CST 2023
[dave@www.cndba.cn ~]$ ssh oracle2 date
Fri Apr 7 11:54:35 CST 2023
[dave@www.cndba.cn ~]$ ssh oracle3 date
Thu Apr 6 20:11:54 CST 2023
[dave@www.cndba.cn ~]$ ssh oracle3 date
Thu Apr 6 21:38:49 CST 2023
[dave@www.cndba.cn ~]$
因为是集群环境,就看了一下系统时间,结果故障节点真的不一样:
[dave@www.cndba.cn3 ~]$ date
Thu Apr 6 21:39:44 CST 2023
[dave@www.cndba.cn3 ~]$ ssh oracle date;
Fri Apr 7 13:22:23 CST 2023
3 解决方法
修改故障节点的时间,使所有节点保持一致,重启系统恢复正常:
[root@oracle3 cm_ctl]# date -s "2023-4-7 13:24:20";
Fri Apr 7 13:24:20 CST 2023
[root@oracle3 cm_ctl]# clock -w
[root@oracle3 cm_ctl]# date;ssh oracle date;
Fri Apr 7 13:24:28 CST 2023
root@oracle's password:
Fri Apr 7 13:24:24 CST 2023
[root@oracle3 cm_ctl]#
[dave@www.cndba.cn3 ~]$ gs_om -t start
Starting cluster.
======================================================================
Successfully started primary instance. Wait for standby instance.
======================================================================
.
Successfully started cluster.
======================================================================
cluster_state : Normal
redistributing : No
node_count : 3
Datanode State
primary : 1
standby : 2
secondary : 0
cascade_standby : 0
building : 0
abnormal : 0
down : 0
Successfully started cluster.
[dave@www.cndba.cn3 ~]$
[dave@www.cndba.cn ~]$ gs_om -t status --detail
[ CMServer State ]
node node_ip instance state
-------------------------------------------------------------------------------
1 oracle 192.168.56.105 1 /data/openGauss/data/cmserver/cm_server Primary
2 oracle2 192.168.56.106 2 /data/openGauss/data/cmserver/cm_server Standby
3 oracle3 192.168.56.107 3 /data/openGauss/data/cmserver/cm_server Standby
[ Cluster State ]
cluster_state : Normal
redistributing : No
balanced : No
current_az : AZ_ALL
[ Datanode State ]
node node_ip instance state
---------------------------------------------------------------------------------
1 oracle 192.168.56.105 6001 /data/openGauss/install/data/dn P Standby Normal
2 oracle2 192.168.56.106 6002 /data/openGauss/install/data/dn S Primary Normal
3 oracle3 192.168.56.107 6003 /data/openGauss/install/data/dn S Standby Normal
[dave@www.cndba.cn ~]$
版权声明:本文为博主原创文章,未经博主允许不得转载。