CNDBA社区
在前面我们看了Hadoop 集群的搭建，基本操作和动态增加节点，链接如下：
Linux 7.6 平台 Hadoop 3.1.1 集群搭建手册
https://www.cndba.cn/download/dave/6Hadoop 3.1.1 动态增加节点 操作示例
https://www.cndba.cn/dave/article/3261Hadoop HDFS 常用命令 汇总
https://www.cndba.cn/dave/article/3258
我们这里看下动态的从Hadoop 集群中删除节点。删除操作相对于增加节点要麻烦一些。具体操作步骤如下。
1 在NameNode节点修改配置文件
现在集群有5个datanode节点，我们删除datanode5。
[https://www.cndba.cn@hadoopslave5 ~]$ yarn node -list
2019-01-24 01:59:33,613 INFO client.RMProxy: Connecting to ResourceManager at hadoopmaster/192.168.20.80:8032
Total Nodes:5
Node-Id Node-State Node-Http-Address Number-of-Running-Containers
hadoopslave2:36452 RUNNING hadoopslave2:8042 0
hadoopslave3:39882 RUNNING hadoopslave3:8042 0
hadoopslave4:43669 RUNNING hadoopslave4:8042 0
hadoopslave1:44641 RUNNING hadoopslave1:8042 0
hadoopslave5:37795 RUNNING hadoopslave5:8042 0
2 修改excludes文件
在~/etc/hadoop目录下创建excludes 文件，配置需要删除的节点：
[https://www.cndba.cn@hadoopmaster hadoop]$ pwd
/home/cndba/hadoop/etc/hadoop
[https://www.cndba.cn@hadoopmaster hadoop]$ touch excludes
[https://www.cndba.cn@hadoopmaster hadoop]$ cat excludes
hadoopslave5
3 修改 ~/etc/hadoop/hdfs-site.xml 文件
在文件中添加如下内容：
<property>
<name>dfs.hosts.exclude</name>
<value>/home/cndba/hadoop/etc/hadoop/excludes</value>
</property>
4 修改 ~/etc/hadoop/mapred-site.xml文件
在文件中添加如下内容：
<property>
<name>mapred.hosts.exclude</name>
<value>/home/cndba/hadoop/etc/hadoop/excludes</value>
<final>true</final>
</property>
5 执行refreshNodes命令
完成以上修改后执行如下命令刷新namenode与yarn：
hdfs dfsadmin -refreshNodes
yarn rmadmin -refreshNodes
注意，这里执行refreshNodes可能没有反应，解决方法可以参考我的博客：
Hadoop 3.1.1 删除节点执行 refreshNodes 无反应解决方法
https://www.cndba.cn/dave/article/3262
[https://www.cndba.cn@hadoopmaster hadoop]$ hdfs dfsadmin -refreshNodes
Refresh nodes successful
[https://www.cndba.cn@hadoopmaster hadoop]$ yarn rmadmin -refreshNodes
WARNING: YARN_CONF_DIR has been replaced by HADOOP_CONF_DIR. Using value of YARN_CONF_DIR.
2019-01-24 02:28:13,773 INFO client.RMProxy: Connecting to ResourceManager at hadoopmaster/192.168.20.80:8033
[https://www.cndba.cn@hadoopmaster hadoop]$
Refresh命令会在后台进行Block块的移动，当DFS上文件较多时，等待时间会比较久。 在此期间可以通过hdfs dfsadmin -report 或 web 界面查看删除节点状态变由 Normal -> decomissioning -> Decommissioned。
Decommissioning datanodes (1):
Name: 192.168.20.85:9866 (hadoopslave5)
Hostname: hadoopslave5
Decommission Status : Decommission in progress
Configured Capacity: 89936470016 (83.76 GB)
DFS Used: 233910272 (223.07 MB)
Non DFS Used: 4794646528 (4.47 GB)
DFS Remaining: 84907913216 (79.08 GB)
DFS Used%: 0.26%
DFS Remaining%: 94.41%
Configured Cache Capacity: 0 (0 B)
Cache Used: 0 (0 B)
Cache Remaining: 0 (0 B)
Cache Used%: 100.00%
Cache Remaining%: 0.00%
Xceivers: 1
Last contact: Thu Jan 24 05:42:24 CST 2019
Last Block Report: Thu Jan 24 05:36:57 CST 2019
Num of Blocks: 608
Name: 192.168.20.85:9866 (hadoopslave5)
Hostname: hadoopslave5
Decommission Status : Decommissioned
Configured Capacity: 89936470016 (83.76 GB)
DFS Used: 233910272 (223.07 MB)
Non DFS Used: 4794576896 (4.47 GB)
DFS Remaining: 84907982848 (79.08 GB)
DFS Used%: 0.26%
DFS Remaining%: 94.41%
Configured Cache Capacity: 0 (0 B)
Cache Used: 0 (0 B)
Cache Remaining: 0 (0 B)
Cache Used%: 100.00%
Cache Remaining%: 0.00%
Xceivers: 1
Last contact: Thu Jan 24 05:44:57 CST 2019
Last Block Report: Thu Jan 24 05:36:57 CST 2019
Num of Blocks: 608
6 在删除节点停止datanode和nodemanager进程
在删除节点执行如下命令：
[https://www.cndba.cn@hadoopslave5 ~]$ yarn --daemon start nodemanager
[https://www.cndba.cn@hadoopslave5 ~]$ yarn --daemon stop nodemanager
[https://www.cndba.cn@hadoopslave5 ~]$ jps
28087 Jps
[https://www.cndba.cn@hadoopslave5 ~]$
7 在NameNode 节点执行自平衡操作
[https://www.cndba.cn@hadoopmaster hadoop]$ start-balancer.sh
[https://www.cndba.cn@hadoopmaster hadoop]$
8 修改 ~/etc/hadoop/hdfs-site.xml 文件
删除之前添加的内容：
<property>
<name>dfs.hosts.exclude</name>
<value>/home/cndba/hadoop/etc/hadoop/excludes</value>
</property>
9 修改workers文件
从workers文件中删除我们这里删除的节点，这样执行start-dfs时就不会启动该上的datanode了。
[https://www.cndba.cn@hadoopmaster hadoop]$ cat workers
hadoopslave1
hadoopslave2
hadoopslave3
hadoopslave4
[https://www.cndba.cn@hadoopmaster hadoop]$
10 查看hdfs报告
可以通过hdfs dfsadmin -report 命令查看，也可以直接通过web来查看，如下：
[https://www.cndba.cn@hadoopmaster ~]$ hdfs dfsadmin -report
Configured Capacity: 359745880064 (335.04 GB)
Present Capacity: 340375109632 (317.00 GB)
DFS Remaining: 330426462208 (307.73 GB)
DFS Used: 9948647424 (9.27 GB)
DFS Used%: 2.92%
Replicated Blocks:
Under replicated blocks: 2
Blocks with corrupt replicas: 0
Missing blocks: 0
Missing blocks (with replication factor 1): 0
Pending deletion blocks: 0
Erasure Coded Block Groups:
Low redundancy block groups: 0
Block groups with corrupt internal blocks: 0
Missing block groups: 0
Pending deletion blocks: 0
-------------------------------------------------
Live datanodes (4):
Name: 192.168.20.81:9866 (hadoopslave1)
Hostname: hadoopslave1
Decommission Status : Normal
Configured Capacity: 89936470016 (83.76 GB)
DFS Used: 3149541376 (2.93 GB)
Non DFS Used: 4855541760 (4.52 GB)
DFS Remaining: 81931386880 (76.30 GB)
DFS Used%: 3.50%
DFS Remaining%: 91.10%
Configured Cache Capacity: 0 (0 B)
Cache Used: 0 (0 B)
Cache Remaining: 0 (0 B)
Cache Used%: 100.00%
Cache Remaining%: 0.00%
Xceivers: 1
Last contact: Thu Jan 24 06:26:13 CST 2019
Last Block Report: Thu Jan 24 05:37:07 CST 2019
Num of Blocks: 39502
Name: 192.168.20.82:9866 (hadoopslave2)
Hostname: hadoopslave2
Decommission Status : Normal
Configured Capacity: 89936470016 (83.76 GB)
DFS Used: 3276431360 (3.05 GB)
Non DFS Used: 4854583296 (4.52 GB)
DFS Remaining: 81805455360 (76.19 GB)
DFS Used%: 3.64%
DFS Remaining%: 90.96%
Configured Cache Capacity: 0 (0 B)
Cache Used: 0 (0 B)
Cache Remaining: 0 (0 B)
Cache Used%: 100.00%
Cache Remaining%: 0.00%
Xceivers: 1
Last contact: Thu Jan 24 06:26:12 CST 2019
Last Block Report: Thu Jan 24 05:50:00 CST 2019
Num of Blocks: 39539
Name: 192.168.20.83:9866 (hadoopslave3)
Hostname: hadoopslave3
Decommission Status : Normal
Configured Capacity: 89936470016 (83.76 GB)
DFS Used: 3319832576 (3.09 GB)
Non DFS Used: 4866662400 (4.53 GB)
DFS Remaining: 81749975040 (76.14 GB)
DFS Used%: 3.69%
DFS Remaining%: 90.90%
Configured Cache Capacity: 0 (0 B)
Cache Used: 0 (0 B)
Cache Remaining: 0 (0 B)
Cache Used%: 100.00%
Cache Remaining%: 0.00%
Xceivers: 1
Last contact: Thu Jan 24 06:26:14 CST 2019
Last Block Report: Thu Jan 24 05:37:06 CST 2019
Num of Blocks: 39725
Name: 192.168.20.84:9866 (hadoopslave4)
Hostname: hadoopslave4
Decommission Status : Normal
Configured Capacity: 89936470016 (83.76 GB)
DFS Used: 202842112 (193.45 MB)
Non DFS Used: 4793982976 (4.46 GB)
DFS Remaining: 84939644928 (79.11 GB)
DFS Used%: 0.23%
DFS Remaining%: 94.44%
Configured Cache Capacity: 0 (0 B)
Cache Used: 0 (0 B)
Cache Remaining: 0 (0 B)
Cache Used%: 100.00%
Cache Remaining%: 0.00%
Xceivers: 1
Last contact: Thu Jan 24 06:26:12 CST 2019
Last Block Report: Thu Jan 24 05:36:57 CST 2019
Num of Blocks: 946
Dead datanodes (1):
Name: 192.168.20.85:9866 (hadoopslave5)
Hostname: hadoopslave5
Decommission Status : Decommissioned
Configured Capacity: 89936470016 (83.76 GB)
DFS Used: 233914368 (223.08 MB)
Non DFS Used: 4794642432 (4.47 GB)
DFS Remaining: 84907913216 (79.08 GB)
DFS Used%: 0.26%
DFS Remaining%: 94.41%
Configured Cache Capacity: 0 (0 B)
Cache Used: 0 (0 B)
Cache Remaining: 0 (0 B)
Cache Used%: 100.00%
Cache Remaining%: 0.00%
Xceivers: 1
Last contact: Thu Jan 24 06:13:36 CST 2019
Last Block Report: Thu Jan 24 05:36:57 CST 2019
Num of Blocks: 608
至此HDFS删除节点操作完成。
版权声明：本文为博主原创文章，未经博主允许不得转载。
