签到成功

知道了

CNDBA社区CNDBA社区

MongoDB 对现有集合进行分片 操作方法

2022-09-14 19:43 1805 0 原创 MongoDB
作者: dave

在之前的博客我们介绍了3节点分片集群的搭建,

http://www.cndba.cn/dave/article/108667

MongoDB 4.4 分片集群(3 shard) 搭建手册
https://www.cndba.cn/cndba/dave/article/107970

分片集群在使用之前,需要特别注意,必须先启用sharding,在对集合指定分片键,这个在之前的博客有测试案例:

mongos> sh.enableSharding("cndba")
mongos> sh.shardCollection("cndba.user", { _id : "hashed" } )

如果没有启动sharding或者没有对集合指定分片键,那么新建的库会随机在一个shard上生成,对接的集合也会在该库上创建。

我们这里先创建一张测试表:

mongos> sh.status()
--- Sharding Status ---
  sharding version: {
        "_id" : 1,
        "minCompatibleVersion" : 5,
        "currentVersion" : 6,
        "clusterId" : ObjectId("62ee1cade37908b0d66ba541")
  }
  shards:
        {  "_id" : "shard1",  "host" : "shard1/192.168.56.101:10001,192.168.56.102:10001,192.168.56.103:10001",  "state" : 1 }
        {  "_id" : "shard2",  "host" : "shard2/192.168.56.101:10002,192.168.56.102:10002,192.168.56.103:10002",  "state" : 1 }
……

mongos> use cndba
switched to db cndba
mongos> for(var i=1;i<=100000;i++){db.user.save({_id:i,"name":"cndba"})};

测试虚拟机性能有限,提期结束了会话:

mongos> db.user.count()
88034

成功插入了8.8w的数据,新建的库是没有启用sharding功能,现在将单表转成分片。

mongos> db.user.stats()
{
        "sharded" : false,
        "primary" : "shard2",
        "capped" : false,
        "wiredTiger" : {
                "metadata" : {
                        "formatVersion" : 1
                },
……

mongos>  sh.enableSharding("cndba");
{
        "ok" : 1,
        "operationTime" : Timestamp(1659801295, 7),
        "$clusterTime" : {
                "clusterTime" : Timestamp(1659801295, 7),
                "signature" : {
                        "hash" : BinData(0,"7ARnLcQXjPVEhp05A43NJdz7/iE="),
                        "keyId" : NumberLong("7128666789528993815")
                }
        }
}
mongos>

指定分片键:http://www.cndba.cn/dave/article/108667

http://www.cndba.cn/dave/article/108667

mongos> sh.shardCollection("cndba.user", { _id : "hashed" } )
{
        "ok" : 0,
        "errmsg" : "Please create an index that starts with the proposed shard key before sharding the collection",
        "code" : 72,
        "codeName" : "InvalidOptions",
        "operationTime" : Timestamp(1659801328, 4),
        "$clusterTime" : {
                "clusterTime" : Timestamp(1659801328, 4),
                "signature" : {
                        "hash" : BinData(0,"nZi/7vtzhXQAoDnxXQi37EF4pig="),
                        "keyId" : NumberLong("7128666789528993815")
                }
        }
}
mongos>

这里提示我们创建一个索引:http://www.cndba.cn/dave/article/108667

mongos> db.user.ensureIndex({_id:"hashed"})
{
        "raw" : {
                "shard2/192.168.56.101:10002,192.168.56.102:10002,192.168.56.103:10002" : {
                        "createdCollectionAutomatically" : false,
                        "numIndexesBefore" : 1,
                        "numIndexesAfter" : 2,
                        "commitQuorum" : "votingMembers",
                        "ok" : 1
                }
        },
        "ok" : 1,
        "operationTime" : Timestamp(1659801422, 5),
        "$clusterTime" : {
                "clusterTime" : Timestamp(1659801422, 5),
                "signature" : {
                        "hash" : BinData(0,"76e0gXnINdDM1eX0t+/NynedInA="),
                        "keyId" : NumberLong("7128666789528993815")
                }
        }
}

再次对集合分片:http://www.cndba.cn/dave/article/108667

mongos> sh.shardCollection("cndba.user", { _id : "hashed" } )
{
        "collectionsharded" : "cndba.user",
        "collectionUUID" : UUID("18cf0073-fa77-42ad-971b-71125ee968f6"),
        "ok" : 1,
        "operationTime" : Timestamp(1659801434, 35),
        "$clusterTime" : {
                "clusterTime" : Timestamp(1659801434, 35),
                "signature" : {
                        "hash" : BinData(0,"8g5rYYSUFkCED7X0dPqXGkYLsJ0="),
                        "keyId" : NumberLong("7128666789528993815")
                }
        }
}
mongos>

查看分片状态:

http://www.cndba.cn/dave/article/108667

mongos> db.user.stats()
{
        "sharded" : true,
        "capped" : false,
        "wiredTiger" : {
                "metadata" : {
                        "formatVersion" : 1
                },
……
    "shards" : {
                "shard2" : {
                        "ns" : "cndba.user",
                        "size" : 2993156,
                        "count" : 88034,
                        "avgObjSize" : 34,
                        "storageSize" : 1077248,
                        "freeStorageSize" : 352256,
                        "capped" : false,
……

此时数据还没有分片。http://www.cndba.cn/dave/article/108667

http://www.cndba.cn/dave/article/108667

Mongodb 的数据均衡是启用的:http://www.cndba.cn/dave/article/108667

mongos> sh.getBalancerState()
true

但是并没有运行:http://www.cndba.cn/dave/article/108667

mongos> sh.status()
--- Sharding Status ---
  sharding version: {
        "_id" : 1,
        "minCompatibleVersion" : 5,
        "currentVersion" : 6,
        "clusterId" : ObjectId("62ee1cade37908b0d66ba541")
  }
  shards:
        {  "_id" : "shard1",  "host" : "shard1/192.168.56.101:10001,192.168.56.102:10001,192.168.56.103:10001",  "state" : 1 }
        {  "_id" : "shard2",  "host" : "shard2/192.168.56.101:10002,192.168.56.102:10002,192.168.56.103:10002",  "state" : 1 }
  active mongoses:
        "4.4.16" : 1
  autosplit:
        Currently enabled: yes
  balancer:
        Currently enabled:  yes
        Currently running:  no
        Failed balancer rounds in last 5 attempts:  0
        Migration Results for the last 24 hours:
                No recent migrations
  databases:
        {  "_id" : "cndba",  "primary" : "shard2",  "partitioned" : true,  "version" : {  "uuid" : UUID("a153446f-4c48-455b-aff9-d12d6c18b58d"),  "lastMod" : 1 } }
                cndba.user
                        shard key: { "_id" : "hashed" }
                        unique: false
                        balancing: true
                        chunks:
                                shard2  1
                        { "_id" : { "$minKey" : 1 } } -->> { "_id" : { "$maxKey" : 1 } } on : shard2 Timestamp(1, 0)
        {  "_id" : "config",  "primary" : "config",  "partitioned" : true }
                config.system.sessions
                        shard key: { "_id" : 1 }
                        unique: false
                        balancing: true
                        chunks:
                                shard1  512
                                shard2  512
                        too many chunks to print, use verbose if you want to force print
mongos>

mongos> sh.isBalancerRunning()
false

这里主要是因为我们测试的数据量太小,还没达到迁移的要求。 生产环境会自动启用迁移。 另外均衡器在执行块迁移操作时候会占用实例中节点的资源,对业务多少会有些影响。为了避免块迁移给业务带来影响我们可以设置均衡器的活动窗口,让其在指定的时间段内工作。

版权声明:本文为博主原创文章,未经博主允许不得转载。

用户评论
* 以下用户言论只代表其个人观点,不代表CNDBA社区的观点或立场
dave

dave

关注

人的一生应该是这样度过的:当他回首往事的时候,他不会因为虚度年华而悔恨,也不会因为碌碌无为而羞耻;这样,在临死的时候,他就能够说:“我的整个生命和全部精力,都已经献给世界上最壮丽的事业....."

  • 2261
    原创
  • 3
    翻译
  • 578
    转载
  • 191
    评论
  • 访问:7965478次
  • 积分:4346
  • 等级:核心会员
  • 排名:第1名
精华文章
    最新问题
    查看更多+
    热门文章
      热门用户
      推荐用户
        Copyright © 2016 All Rights Reserved. Powered by CNDBA · 皖ICP备2022006297号-1·

        QQ交流群

        注册联系QQ