1 Jq 工具说明

---

在之前的博客中我们了解了MongoDB 4.4 之后日志格式发生了变化，如下：

MongoDB 4.4 日志格式 / 组件 / 等级设置 说明
https://www.cndba.cn/dave/article/108020

http://www.cndba.cn/cndba/dave/article/108021

新格式采用JSON 来记录，这种格式对DBA来说，并不是很友好。 不过可以通过jq工具来格式化输出日志内容。

Jq是开源的JSON 解释工具，可以直接从github上下载：

https://stedolan.github.io/jq/download/

下载后的文件名是：jq-linux64这种，直接重命名即可。

将文件复制到bin 目录下，这样可以省去配置环境变量：

[dave@www.cndba.cn_3 logs]# cp jq /usr/bin/
[dave@www.cndba.cn_3 logs]# chmod a+x /usr/bin/jq

然后使用jq工具来格式化输出log 日志。

输出所有日志：http://www.cndba.cn/cndba/dave/article/108021

[dave@www.cndba.cn_3 logs]# more shard1.log |jqhttp://www.cndba.cn/cndba/dave/article/108021

查看最后2条日志：

[dave@www.cndba.cn_3 logs]# cat shard1.log |tail -2|jq
{
  "t": {
    "$date": "2022-05-23T14:05:51.994+08:00"
  },
  "s": "I",
  "c": "NETWORK",
  "id": 51800,
  "ctx": "conn110",
  "msg": "client metadata",
  "attr": {
    "remote": "172.31.185.165:58432",
    "client": "conn110",
    "doc": {
      "driver": {
        "name": "NetworkInterfaceTL",
        "version": "5.0.8"
      },
      "os": {
        "type": "Linux",
        "name": "CentOS Linux release 7.8.2003 (Core)",
        "architecture": "x86_64",
        "version": "Kernel 3.10.0-1127.el7.x86_64"
      }
    }
  }
}
{
  "t": {
    "$date": "2022-05-23T14:05:51.996+08:00"
  },
  "s": "I",
  "c": "ACCESS",
  "id": 20250,
  "ctx": "conn110",
  "msg": "Authentication succeeded",
  "attr": {
    "mechanism": "SCRAM-SHA-256",
    "speculative": true,
    "principalName": "__system",
    "authenticationDatabase": "local",
    "remote": "172.31.185.165:58432",
    "extraInfo": {}
  }
}
[dave@www.cndba.cn_3 logs]#

2 jq示例

---

官网有更多的示例:

https://www.mongodb.com/docs/v5.0/reference/log-messages/#std-label-log-message-parsinghttp://www.cndba.cn/cndba/dave/article/108021

Counting Unique Messages

jq -r “.msg” /var/log/mongodb/mongod.log | sort | uniq -c | sort -rn | head -10

Monitoring Connections

jq -r ‘.attr.remote’ /var/log/mongodb/mongod.log | grep -v ‘null’ | sort | uniq -c | sort -r

jq -r ‘.attr.remote’ /var/log/mongodb/mongod.log | grep -v ‘null’ | awk -F’:’ ‘{print $1}’ | sort | uniq -c | sort -rhttp://www.cndba.cn/cndba/dave/article/108021

Analyzing Driver Connectionshttp://www.cndba.cn/cndba/dave/article/108021

jq -cr ‘.attr.doc.driver’ /var/log/mongodb/mongod.log | grep -v null | sort | uniq -c | sort -rn

http://www.cndba.cn/cndba/dave/article/108021

Analyzing Client Typeshttp://www.cndba.cn/cndba/dave/article/108021

jq -r ‘.attr.doc.os.type’ /var/log/mongodb/mongod.log | grep -v null | sort | uniq -c | sort -rn

Analyzing Slow Queries

jq ‘. | select(.attr.durationMillis>=2000)’ /var/log/mongodb/mongod.log

http://www.cndba.cn/cndba/dave/article/108021

Filtering by Component

jq ‘. | select(.c==”REPL”)’ /var/log/mongodb/mongod.log

jq ‘. | select(.c!=”REPL”)’ /var/log/mongodb/mongod.log

jq ‘. | select( .c as $c | [“REPL”, “STORAGE”] | index($c) )’ /var/log/mongodb/mongod.log

Filtering by Known Log ID

{"t":{"$date":"2020-06-01T13:06:59.027-0500"},"s":"I", "c":"NETWORK", "id":22943,"ctx":"listener","msg":"connection accepted from {session_remote} #{session_id} ({connectionCount}{word} now open)","attr":{"session_remote":"127.0.0.1:61298","session_id":164,"connectionCount":11,"word":" connections"}}
{"t":{"$date":"2020-06-01T13:07:03.490-0500"},"s":"I", "c":"NETWORK", "id":22944,"ctx":"conn157","msg":"end connection {remote} ({connectionCount}{word} now open)","attr":{"remote":"127.0.0.1:61298","connectionCount":10,"word":" connections"}}

jq '. | select( .id as $id | [22943, 22944] | index($id) )' /var/log/mongodb/mongod.log

Filtering by Date Range

jq ‘. | select(.t[“$date”] >= “2020-04-15T00:00:00.000” and .t[“$date”] <= “2020-04-15T23:59:59.999”)’ /var/log/mongodb/mongod.log

jq ‘. | select(.t[“$date”] >= “2020-05-01T00:00:00.000” and .t[“$date”] <= “2020-05-31T23:59:59.999” and .attr.remote)’ /var/log/mongodb/mongod.log

版权声明：本文为博主原创文章，未经博主允许不得转载。