签到成功

知道了

CNDBA社区CNDBA社区

MongoDB 日志 管理

2022-04-29 14:47 1835 0 原创 MongoDB
作者: dave

在之前的博客我们了解到了MongoDB的2个存储引擎:

MongoDB 存储引擎 说明
https://www.cndba.cn/dave/article/107958

在这两种存储引擎下都需要进行日志记录。 注意这里是引擎自己的WAL 日志,记录事务操作的,而非软件的运行log日志。

1 WiredTiger存储引擎的WAL日志


WiredTiger使用检查点提供磁盘上数据的一致视图,并允许MongoDB从最后一个检查点恢复。但是,如果MongoDB在检查点之间意外退出,则需要日志记录来恢复在最后一个检查点之后发生的信息。

从MongoDB 4.0开始,对于使用WiredTiger存储引擎的副本集成员,不能指定—nojournal选项或storage.journal.enabled:false。

使用日记记录恢复过程:http://www.cndba.cn/dave/article/107961

  1. 在数据文件中查找以找到最后一个检查点的标识符。
  2. 在日志文件中搜索与最后一个检查点的标识符匹配的记录。
  3. 从上一个检查点开始,应用日志文件中记录的操作。

1.1 日志记录过程

在3.2 之后的版本中启用日志功能后,WiredTiger为每个客户端发起的写操作创建一个日记记录。日志记录包含内部写操作触发的所有的写入操作。比如,对集合中文档的更新可能会导致对索引的修改;WiredTiger会创建单个日志记录,用来包含更新操作及其关联的索引修改。http://www.cndba.cn/dave/article/107961

WiredTiger引擎使用内存缓冲来存储日记记录( journal records)。 日记记录在内存中占用的buffer 最多128kB。

WiredTiger引擎在以下任一情况下都会同步buffer中的journal records 到磁盘:

  1. For replica set members (primary and secondary members),
    If there are operations waiting for oplog entries. Operations that can wait for oplog entries include:
    forward scanning queries against the oplog
    read operations performed as part of causally consistent sessions
    Additionally for secondary members, after every batch application of the oplog entries.

  2. If a write operation includes or implies a write concern of j: true.

  3. At every 100 milliseconds (See storage.journal.commitIntervalMs).

    http://www.cndba.cn/dave/article/107961

  4. When WiredTiger creates a new journal file. Because MongoDB uses a journal file size limit of 100 MB, WiredTiger creates a new journal file approximately every 100 MB of data.

注意事项:

http://www.cndba.cn/dave/article/107961
http://www.cndba.cn/dave/article/107961

In between write operations, while the journal records remain in the WiredTiger buffers, updates can be lost following a hard shutdown of mongod.

1.2 日志文件(Journal Files)

For the journal files, MongoDB creates a subdirectory named journal under the dbPath directory. WiredTiger journal files have names with the following format WiredTigerLog. where is a zero-padded number starting from 0000000001.

[root@cndba data]# tree .
.
├── collection-0--3142372022616416207.wt
├── collection-0-6981002967078284256.wt
……
│   └── metrics.interim
├── index-11--3142372022616416207.wt
……
├── journal
│   ├── WiredTigerLog.0000000006
│   ├── WiredTigerPreplog.0000000001
│   └── WiredTigerPreplog.0000000002
├── _mdb_catalog.wt
├── mongod.lock
├── sizeStorer.wt
├── storage.bson
├── WiredTiger
├── WiredTigerHS.wt
├── WiredTiger.lock
├── WiredTiger.turtle
└── WiredTiger.wt

2 directories, 38 files
[root@cndba data]#

1.2.1 Journal Records

Journal files contain a record per each client initiated write operation

  • The journal record includes any internal write operations caused by the initial write. For example, an update to a document in a collection may result in modifications to the indexes; WiredTiger creates a single journal record that includes both the update operation and its associated index modifications.
  • Each record has a unique identifier.
  • The minimum journal record size for WiredTiger is 128 bytes.

1.2.2 Compression

By default, MongoDB configures WiredTiger to use snappy compression for its journaling data. To specify a different compression algorithm or no compression, use the storage.wiredTiger.engineConfig.journalCompressor setting.

NOTE
If a log record less than or equal to 128 bytes (the mininum log record size for WiredTiger), WiredTiger does not compress that record.http://www.cndba.cn/dave/article/107961

1.2.3 Journal File Size Limit

WiredTiger journal files for MongoDB have a maximum size limit of approximately 100 MB.

  • Once the file exceeds that limit, WiredTiger creates a new journal file.
  • WiredTiger automatically removes old journal files to maintain only the files needed to recover from last checkpoint.

这里讲的很清晰,WiredTiger 会自动删除最后一次checkpoint 之后恢复不需要的journal files。 所以我们这里的需要只有0000000006。

├── journal
│   ├── WiredTigerLog.0000000006
│   ├── WiredTigerPreplog.0000000001
│   └── WiredTigerPreplog.0000000002

1.2.4 Pre-Allocation

WiredTiger pre-allocates journal files.

2 In-Memory存储引擎的Journaling


Starting in MongoDB Enterprise version 3.2.6, the In-Memory Storage Engine is part of general availability (GA). Because its data is kept in memory, there is no separate journal. Write operations with a write concern of j: true are immediately acknowledged.http://www.cndba.cn/dave/article/107961http://www.cndba.cn/dave/article/107961

If any voting member of a replica set uses the in-memory storage engine, you must set writeConcernMajorityJournalDefault to false.http://www.cndba.cn/dave/article/107961

PS: http://www.cndba.cn/dave/article/107961

Mongodb 中文社区的中文文档基本是拿软件直接翻译的,很多意思都不对,所以又直接看官方手册了。 为节省时间,就不翻译了。存档记录一下。

版权声明:本文为博主原创文章,未经博主允许不得转载。

用户评论
* 以下用户言论只代表其个人观点,不代表CNDBA社区的观点或立场
dave

dave

关注

人的一生应该是这样度过的:当他回首往事的时候,他不会因为虚度年华而悔恨,也不会因为碌碌无为而羞耻;这样,在临死的时候,他就能够说:“我的整个生命和全部精力,都已经献给世界上最壮丽的事业....."

  • 2297
    原创
  • 3
    翻译
  • 644
    转载
  • 198
    评论
  • 访问:9617463次
  • 积分:4542
  • 等级:核心会员
  • 排名:第1名
精华文章
    最新问题
    查看更多+
    热门文章
      热门用户
      推荐用户
        Copyright © 2016 All Rights Reserved. Powered by CNDBA · 皖ICP备2022006297号-1·

        QQ交流群

        注册联系QQ