签到成功

知道了

CNDBA社区CNDBA社区

MongoDB 日志 管理

2022-04-29 14:47 1996 0 原创 MongoDB
作者: dave

在之前的博客我们了解到了MongoDB的2个存储引擎:

MongoDB 存储引擎 说明
https://www.cndba.cn/dave/article/107958

在这两种存储引擎下都需要进行日志记录。 注意这里是引擎自己的WAL 日志,记录事务操作的,而非软件的运行log日志。

1 WiredTiger存储引擎的WAL日志


WiredTiger使用检查点提供磁盘上数据的一致视图,并允许MongoDB从最后一个检查点恢复。但是,如果MongoDB在检查点之间意外退出,则需要日志记录来恢复在最后一个检查点之后发生的信息。

从MongoDB 4.0开始,对于使用WiredTiger存储引擎的副本集成员,不能指定—nojournal选项或storage.journal.enabled:false。

使用日记记录恢复过程:

  1. 在数据文件中查找以找到最后一个检查点的标识符。
  2. 在日志文件中搜索与最后一个检查点的标识符匹配的记录。
  3. 从上一个检查点开始,应用日志文件中记录的操作。

1.1 日志记录过程

在3.2 之后的版本中启用日志功能后,WiredTiger为每个客户端发起的写操作创建一个日记记录。日志记录包含内部写操作触发的所有的写入操作。比如,对集合中文档的更新可能会导致对索引的修改;WiredTiger会创建单个日志记录,用来包含更新操作及其关联的索引修改。

WiredTiger引擎使用内存缓冲来存储日记记录( journal records)。 日记记录在内存中占用的buffer 最多128kB。http://www.cndba.cn/cndba/dave/article/107961

WiredTiger引擎在以下任一情况下都会同步buffer中的journal records 到磁盘:

http://www.cndba.cn/cndba/dave/article/107961

  1. For replica set members (primary and secondary members),
    If there are operations waiting for oplog entries. Operations that can wait for oplog entries include:
    forward scanning queries against the oplog
    read operations performed as part of causally consistent sessions
    Additionally for secondary members, after every batch application of the oplog entries.

  2. If a write operation includes or implies a write concern of j: true.

  3. At every 100 milliseconds (See storage.journal.commitIntervalMs).http://www.cndba.cn/cndba/dave/article/107961

  4. When WiredTiger creates a new journal file. Because MongoDB uses a journal file size limit of 100 MB, WiredTiger creates a new journal file approximately every 100 MB of data.

注意事项:http://www.cndba.cn/cndba/dave/article/107961

In between write operations, while the journal records remain in the WiredTiger buffers, updates can be lost following a hard shutdown of mongod.

1.2 日志文件(Journal Files)

For the journal files, MongoDB creates a subdirectory named journal under the dbPath directory. WiredTiger journal files have names with the following format WiredTigerLog. where is a zero-padded number starting from 0000000001.

[root@cndba data]# tree .
.
├── collection-0--3142372022616416207.wt
├── collection-0-6981002967078284256.wt
……
│   └── metrics.interim
├── index-11--3142372022616416207.wt
……
├── journal
│   ├── WiredTigerLog.0000000006
│   ├── WiredTigerPreplog.0000000001
│   └── WiredTigerPreplog.0000000002
├── _mdb_catalog.wt
├── mongod.lock
├── sizeStorer.wt
├── storage.bson
├── WiredTiger
├── WiredTigerHS.wt
├── WiredTiger.lock
├── WiredTiger.turtle
└── WiredTiger.wt

2 directories, 38 files
[root@cndba data]#

1.2.1 Journal Records

Journal files contain a record per each client initiated write operationhttp://www.cndba.cn/cndba/dave/article/107961

http://www.cndba.cn/cndba/dave/article/107961

  • The journal record includes any internal write operations caused by the initial write. For example, an update to a document in a collection may result in modifications to the indexes; WiredTiger creates a single journal record that includes both the update operation and its associated index modifications.
  • Each record has a unique identifier.
  • The minimum journal record size for WiredTiger is 128 bytes.

1.2.2 Compression

By default, MongoDB configures WiredTiger to use snappy compression for its journaling data. To specify a different compression algorithm or no compression, use the storage.wiredTiger.engineConfig.journalCompressor setting.

NOTE
If a log record less than or equal to 128 bytes (the mininum log record size for WiredTiger), WiredTiger does not compress that record.

1.2.3 Journal File Size Limit

WiredTiger journal files for MongoDB have a maximum size limit of approximately 100 MB.

  • Once the file exceeds that limit, WiredTiger creates a new journal file.
  • WiredTiger automatically removes old journal files to maintain only the files needed to recover from last checkpoint.

这里讲的很清晰,WiredTiger 会自动删除最后一次checkpoint 之后恢复不需要的journal files。 所以我们这里的需要只有0000000006。http://www.cndba.cn/cndba/dave/article/107961

http://www.cndba.cn/cndba/dave/article/107961

├── journal
│   ├── WiredTigerLog.0000000006
│   ├── WiredTigerPreplog.0000000001
│   └── WiredTigerPreplog.0000000002

1.2.4 Pre-Allocation

WiredTiger pre-allocates journal files.

2 In-Memory存储引擎的Journaling


Starting in MongoDB Enterprise version 3.2.6, the In-Memory Storage Engine is part of general availability (GA). Because its data is kept in memory, there is no separate journal. Write operations with a write concern of j: true are immediately acknowledged.

If any voting member of a replica set uses the in-memory storage engine, you must set writeConcernMajorityJournalDefault to false.http://www.cndba.cn/cndba/dave/article/107961

PS: http://www.cndba.cn/cndba/dave/article/107961

Mongodb 中文社区的中文文档基本是拿软件直接翻译的,很多意思都不对,所以又直接看官方手册了。 为节省时间,就不翻译了。存档记录一下。

版权声明:本文为博主原创文章,未经博主允许不得转载。

用户评论
* 以下用户言论只代表其个人观点,不代表CNDBA社区的观点或立场
dave

dave

关注

人的一生应该是这样度过的:当他回首往事的时候,他不会因为虚度年华而悔恨,也不会因为碌碌无为而羞耻;这样,在临死的时候,他就能够说:“我的整个生命和全部精力,都已经献给世界上最壮丽的事业....."

  • 2297
    原创
  • 3
    翻译
  • 680
    转载
  • 200
    评论
  • 访问:10143773次
  • 积分:4578
  • 等级:核心会员
  • 排名:第1名
精华文章
    最新问题
    查看更多+
    热门文章
      热门用户
      推荐用户
        Copyright © 2016 All Rights Reserved. Powered by CNDBA · 皖ICP备2022006297号-1·

        QQ交流群

        注册联系QQ