在之前的博客我们了解到了MongoDB的2个存储引擎:
MongoDB 存储引擎 说明
https://www.cndba.cn/dave/article/107958
在这两种存储引擎下都需要进行日志记录。 注意这里是引擎自己的WAL 日志,记录事务操作的,而非软件的运行log日志。
1 WiredTiger存储引擎的WAL日志
WiredTiger使用检查点提供磁盘上数据的一致视图,并允许MongoDB从最后一个检查点恢复。但是,如果MongoDB在检查点之间意外退出,则需要日志记录来恢复在最后一个检查点之后发生的信息。
从MongoDB 4.0开始,对于使用WiredTiger存储引擎的副本集成员,不能指定—nojournal选项或storage.journal.enabled:false。
使用日记记录恢复过程:
- 在数据文件中查找以找到最后一个检查点的标识符。
- 在日志文件中搜索与最后一个检查点的标识符匹配的记录。
- 从上一个检查点开始,应用日志文件中记录的操作。
1.1 日志记录过程
在3.2 之后的版本中启用日志功能后,WiredTiger为每个客户端发起的写操作创建一个日记记录。日志记录包含内部写操作触发的所有的写入操作。比如,对集合中文档的更新可能会导致对索引的修改;WiredTiger会创建单个日志记录,用来包含更新操作及其关联的索引修改。
WiredTiger引擎使用内存缓冲来存储日记记录( journal records)。 日记记录在内存中占用的buffer 最多128kB。
WiredTiger引擎在以下任一情况下都会同步buffer中的journal records 到磁盘:
For replica set members (primary and secondary members),
If there are operations waiting for oplog entries. Operations that can wait for oplog entries include:
forward scanning queries against the oplog
read operations performed as part of causally consistent sessions
Additionally for secondary members, after every batch application of the oplog entries.If a write operation includes or implies a write concern of j: true.
At every 100 milliseconds (See storage.journal.commitIntervalMs).
- When WiredTiger creates a new journal file. Because MongoDB uses a journal file size limit of 100 MB, WiredTiger creates a new journal file approximately every 100 MB of data.
注意事项:
In between write operations, while the journal records remain in the WiredTiger buffers, updates can be lost following a hard shutdown of mongod.
1.2 日志文件(Journal Files)
For the journal files, MongoDB creates a subdirectory named journal under the dbPath directory. WiredTiger journal files have names with the following format WiredTigerLog.
[root@cndba data]# tree .
.
├── collection-0--3142372022616416207.wt
├── collection-0-6981002967078284256.wt
……
│ └── metrics.interim
├── index-11--3142372022616416207.wt
……
├── journal
│ ├── WiredTigerLog.0000000006
│ ├── WiredTigerPreplog.0000000001
│ └── WiredTigerPreplog.0000000002
├── _mdb_catalog.wt
├── mongod.lock
├── sizeStorer.wt
├── storage.bson
├── WiredTiger
├── WiredTigerHS.wt
├── WiredTiger.lock
├── WiredTiger.turtle
└── WiredTiger.wt
2 directories, 38 files
[root@cndba data]#
1.2.1 Journal Records
Journal files contain a record per each client initiated write operation
- The journal record includes any internal write operations caused by the initial write. For example, an update to a document in a collection may result in modifications to the indexes; WiredTiger creates a single journal record that includes both the update operation and its associated index modifications.
- Each record has a unique identifier.
- The minimum journal record size for WiredTiger is 128 bytes.
1.2.2 Compression
By default, MongoDB configures WiredTiger to use snappy compression for its journaling data. To specify a different compression algorithm or no compression, use the storage.wiredTiger.engineConfig.journalCompressor setting.
NOTE
If a log record less than or equal to 128 bytes (the mininum log record size for WiredTiger), WiredTiger does not compress that record.
1.2.3 Journal File Size Limit
WiredTiger journal files for MongoDB have a maximum size limit of approximately 100 MB.
- Once the file exceeds that limit, WiredTiger creates a new journal file.
- WiredTiger automatically removes old journal files to maintain only the files needed to recover from last checkpoint.
这里讲的很清晰,WiredTiger 会自动删除最后一次checkpoint 之后恢复不需要的journal files。 所以我们这里的需要只有0000000006。
├── journal
│ ├── WiredTigerLog.0000000006
│ ├── WiredTigerPreplog.0000000001
│ └── WiredTigerPreplog.0000000002
1.2.4 Pre-Allocation
WiredTiger pre-allocates journal files.
2 In-Memory存储引擎的Journaling
Starting in MongoDB Enterprise version 3.2.6, the In-Memory Storage Engine is part of general availability (GA). Because its data is kept in memory, there is no separate journal. Write operations with a write concern of j: true are immediately acknowledged.
If any voting member of a replica set uses the in-memory storage engine, you must set writeConcernMajorityJournalDefault to false.
PS:
Mongodb 中文社区的中文文档基本是拿软件直接翻译的,很多意思都不对,所以又直接看官方手册了。 为节省时间,就不翻译了。存档记录一下。
版权声明:本文为博主原创文章,未经博主允许不得转载。
- 上一篇:MongoDB 存储引擎 说明
- 下一篇:MongoDB 的 启动 和 关闭



