MongoDB ServerのPoint In Time Recoveryの操作方法を説明します。Point In Time Recoveryとはある時点のデータに戻すリストア手法です。MongoDB用語で説明すれば、mongodumpによるバックアップ取得後にoplogを適用して任意の時刻の状態に復旧させる操作です。IPA試験で出題されるようなベンダー非依存の表現を用いれば、バックアップ取得後に更新後ログを適用して復旧させる操作です。
前提
公式ドキュメント
参考になる公式ドキュメントを以下に示します。
動作確認済環境
- Rocky Linux 8.6
- MongoDB Server 6.0.2
構成図
以下の構成で動作確認をします。
172.16.1.0/24
+-----------------+-----------------+
| | |
ens192 | .10 ens192 | .20 ens192 | .30
+------+------+ +------+------+ +------+------+
| linux010 | | linux020 | | linux030 |
| (Rocky8.6) | | (Rocky8.6) | | (Rocky8.6) |
| (Primary) | | (Secondary) | | (Secondary) |
+-------------+ +-------------+ +-------------+
事前設定
3台のMongoDB Serverでレプリケーションが組めるように、/etc/mongod.confを記述します。以下に/etc/mongod.confの設定の要所を記します。
#vi /etc/mongod.conf # network interfaces net: port: 27017 bindIp: 0.0.0.0 #replication: replication: replSetName: "rs0" oplogSizeMB: 1024
以下のような設定を投入し、レプリケーションを開始します。
rs.initiate(
{
_id: "rs0",
members: [
{ _id : 0, host : "172.16.1.10:27017", priority : 4 },
{ _id : 1, host : "172.16.1.20:27017", priority : 2 },
{ _id : 2, host : "172.16.1.30:27017", priority : 1 }
]
}
)
バックアップ操作とテストデータ投入
操作1. 5件のデータ投入とdump取得
テスト用のデータ5件を登録します。
mongosh --quiet test << EOF
db.inventory.insertMany( [
{ item: "canvas", qty: 100, size: { h: 28, w: 35.5, uom: "cm" }, status: "A" },
{ item: "journal", qty: 25, size: { h: 14, w: 21, uom: "cm" }, status: "A" },
{ item: "mat", qty: 85, size: { h: 27.9, w: 35.5, uom: "cm" }, status: "A" },
{ item: "mousepad", qty: 25, size: { h: 19, w: 22.85, uom: "cm" }, status: "P" },
{ item: "notebook", qty: 50, size: { h: 8.5, w: 11, uom: "in" }, status: "P" },
] );
EOF
この時点のバックアップ(dump)を取得します。操作例は以下の通りです。
mongodump \ --host=127.0.0.1 \ --port=27017 \ --out=dump
操作ログは以下の通りです。この時の操作時刻をメモに控えておきましょう。以下出力の場合は21:20です。
[root@linux010 ~]# date 2022年 10月 28日 金曜日 21:20:30 JST [root@linux010 ~]# mongodump \ > --host=127.0.0.1 \ > --port=27017 \ > --out=dump 2022-10-28T21:20:30.831+0900 writing admin.system.version to dump/admin/system.version.bson 2022-10-28T21:20:30.833+0900 done dumping admin.system.version (1 document) 2022-10-28T21:20:30.834+0900 writing test.inventory to dump/test/inventory.bson 2022-10-28T21:20:30.834+0900 writing config.tenantMigrationRecipients to dump/config/tenantMigrationRecipients.bson 2022-10-28T21:20:30.836+0900 writing config.tenantMigrationDonors to dump/config/tenantMigrationDonors.bson 2022-10-28T21:20:30.837+0900 writing config.external_validation_keys to dump/config/external_validation_keys.bson 2022-10-28T21:20:30.840+0900 done dumping config.tenantMigrationRecipients (0 documents) 2022-10-28T21:20:30.840+0900 done dumping test.inventory (5 documents) 2022-10-28T21:20:30.842+0900 done dumping config.external_validation_keys (0 documents) 2022-10-28T21:20:30.842+0900 writing config.system.preimages to dump/config/system.preimages.bson 2022-10-28T21:20:30.843+0900 done dumping config.tenantMigrationDonors (0 documents) 2022-10-28T21:20:30.843+0900 done dumping config.system.preimages (0 documents) [root@linux010 ~]#
念の為、dumpを取得できている事を確認します。
[root@linux010 ~]# ls -l dump/ 合計 0 drwxr-xr-x 2 root root 69 10月 28 21:20 admin drwxr-xr-x 2 root root 318 10月 28 21:20 config drwxr-xr-x 2 root root 59 10月 28 21:20 test [root@linux010 ~]#
操作2. 3件のデータ投入
3件のデータを投入します。
mongosh --quiet test << EOF
db.inventory.insertMany( [
{ item: "paper", qty: 100, size: { h: 8.5, w: 11, uom: "in" }, status: "D" },
{ item: "planner", qty: 75, size: { h: 22.85, w: 30, uom: "cm" }, status: "D" },
{ item: "postcard", qty: 45, size: { h: 10, w: 15.25, uom: "cm" }, status: "A" },
] );
EOF
操作ログは以下の通りです。この時の操作時刻をメモに控えておきましょう。以下出力の場合は21:40です。
[root@linux010 ~]# date
2022年 10月 28日 金曜日 21:40:03 JST
[root@linux010 ~]# mongosh --quiet test << EOF
> db.inventory.insertMany( [
> { item: "paper", qty: 100, size: { h: 8.5, w: 11, uom: "in" }, status: "D" },
> { item: "planner", qty: 75, size: { h: 22.85, w: 30, uom: "cm" }, status: "D" },
> { item: "postcard", qty: 45, size: { h: 10, w: 15.25, uom: "cm" }, status: "A" },
> ] );
> EOF
rs0 [direct: primary] test> db.inventory.insertMany( [
... { item: "paper", qty: 100, size: { h: 8.5, w: 11, uom: "in" }, status: "D" },
... { item: "planner", qty: 75, size: { h: 22.85, w: 30, uom: "cm" }, status: "D" },
... { item: "postcard", qty: 45, size: { h: 10, w: 15.25, uom: "cm" }, status: "A" },
... ] );
{
acknowledged: true,
insertedIds: {
'0': ObjectId("635bcda7b9197bae6f89d758"),
'1': ObjectId("635bcda7b9197bae6f89d759"),
'2': ObjectId("635bcda7b9197bae6f89d75a")
}
}
rs0 [direct: primary] test> [root@linux010 ~]#
操作3. 2件のデータ投入とoplog取得
2件のデータを投入します。
mongosh --quiet test << EOF
db.inventory.insertMany( [
{ item: "sketchbook", qty: 80, size: { h: 14, w: 21, uom: "cm" }, status: "A" },
{ item: "sketch pad", qty: 95, size: { h: 22.85, w: 30.5, uom: "cm" }, status: "A" }
] );
EOF
操作ログは以下の通りです。この時の操作時刻をメモに控えておきましょう。以下出力の場合は22:10です。
[root@linux010 ~]# date
2022年 10月 28日 金曜日 22:10:30 JST
[root@linux010 ~]# mongosh --quiet test << EOF
> db.inventory.insertMany( [
> { item: "sketchbook", qty: 80, size: { h: 14, w: 21, uom: "cm" }, status: "A" },
> { item: "sketch pad", qty: 95, size: { h: 22.85, w: 30.5, uom: "cm" }, status: "A" }
> ] );
> EOF
rs0 [direct: primary] test> db.inventory.insertMany( [
... { item: "sketchbook", qty: 80, size: { h: 14, w: 21, uom: "cm" }, status: "A" },
... { item: "sketch pad", qty: 95, size: { h: 22.85, w: 30.5, uom: "cm" }, status: "A" }
... ] );
{
acknowledged: true,
insertedIds: {
'0': ObjectId("635bd4ccb7ee5d99e8af0cb8"),
'1': ObjectId("635bd4ccb7ee5d99e8af0cb9")
}
}
rs0 [direct: primary] test> [root@linux010 ~]#
10件のデータが登録された時点のoplogを取得します。操作例は以下の通りです。
mongodump \ --host=127.0.0.1 \ --port=27017 \ --db=local \ --collection=oplog.rs \ --out=oplog
確かにoplogを取得できている事を確認します。
[root@linux010 ~]# ls -l oplog/local/
合計 44
-rw-r--r-- 1 root root 39638 10月 28 22:10 oplog.rs.bson
-rw-r--r-- 1 root root 185 10月 28 22:10 oplog.rs.metadata.json
[root@linux010 ~]# bsondump oplog/local/oplog.rs.bson | tail -n 3
2022-10-28T22:11:37.735+0900 337 objects found
{"lsid":{"id":{"$binary":{"base64":"349nCBjHSxSzcqHad9Y9vA==","subType":"04"}},"uid":{"$binary":{"base64":"47DEQpj8HBSa+/TImW+5JCeuQeRkm5NMpJWZG3hSuFU=","subType":"00"}}},"txnNumber":{"$numberLong":"1"},"op":"i","ns":"test.inventory","ui":{"$binary":{"base64":"nFqbQjS0RmOmXKaJZ17Ipw==","subType":"04"}},"o":{"_id":{"$oid":"635bd4ccb7ee5d99e8af0cb8"},"item":"sketchbook","qty":{"$numberInt":"80"},"size":{"h":{"$numberInt":"14"},"w":{"$numberInt":"21"},"uom":"cm"},"status":"A"},"o2":{"_id":{"$oid":"635bd4ccb7ee5d99e8af0cb8"}},"stmtId":{"$numberInt":"0"},"ts":{"$timestamp":{"t":1666962636,"i":1}},"t":{"$numberLong":"1"},"v":{"$numberLong":"2"},"wall":{"$date":{"$numberLong":"1666962636673"}},"prevOpTime":{"ts":{"$timestamp":{"t":0,"i":0}},"t":{"$numberLong":"-1"}}}
{"lsid":{"id":{"$binary":{"base64":"349nCBjHSxSzcqHad9Y9vA==","subType":"04"}},"uid":{"$binary":{"base64":"47DEQpj8HBSa+/TImW+5JCeuQeRkm5NMpJWZG3hSuFU=","subType":"00"}}},"txnNumber":{"$numberLong":"1"},"op":"i","ns":"test.inventory","ui":{"$binary":{"base64":"nFqbQjS0RmOmXKaJZ17Ipw==","subType":"04"}},"o":{"_id":{"$oid":"635bd4ccb7ee5d99e8af0cb9"},"item":"sketch pad","qty":{"$numberInt":"95"},"size":{"h":{"$numberDouble":"22.85"},"w":{"$numberDouble":"30.5"},"uom":"cm"},"status":"A"},"o2":{"_id":{"$oid":"635bd4ccb7ee5d99e8af0cb9"}},"stmtId":{"$numberInt":"1"},"ts":{"$timestamp":{"t":1666962636,"i":2}},"t":{"$numberLong":"1"},"v":{"$numberLong":"2"},"wall":{"$date":{"$numberLong":"1666962636673"}},"prevOpTime":{"ts":{"$timestamp":{"t":1666962636,"i":1}},"t":{"$numberLong":"1"}}}
{"op":"n","ns":"","o":{"msg":"periodic noop"},"ts":{"$timestamp":{"t":1666962651,"i":1}},"t":{"$numberLong":"1"},"v":{"$numberLong":"2"},"wall":{"$date":{"$numberLong":"1666962651680"}}}
[root@linux010 ~]#
リストア操作
それでは操作2の時点に戻すpoint-in-time recoveryに挑戦してみましょう。
まずはリストア前のinventoryコレクションの件数を確認します。これが10件から8件に減れば想定通りのリストアがなされています。
[root@linux010 ~]# mongosh --quiet --eval 'db.inventory.find().count()' 10 [root@linux010 ~]#
22:00時点に戻せば操作2時点になります。mongorestoreコマンドはpoint-in-time recoveryで戻す時刻をepoc timeで指定する必要がありますので、22:00のepoc timeを調べます。操作例は以下の通りです。
[root@linux010 ~]# date -d "2022/10/28 22:00" +%s 1666958400 [root@linux010 ~]#
想定外のデータを消すために、mongorestoreコマンドにdropオプションを付与します。また、point-in-time recoveryをするには、
mongorestore \ --host=127.0.0.1 \ --port=27017 \ --drop \ --dir=dump \ --oplogReplay \ --oplogLimit=1666962000 \ --oplogFile=oplog/local/oplog.rs.bson
操作ログは以下の通りです。多くのログが出力されますが、末尾にrestored successfullyと表示される事は最低限確認しておきましょう。
[root@linux010 ~]# mongorestore \
> --host=127.0.0.1 \
> --port=27017 \
> --drop \
> --dir=dump \
> --oplogReplay \
> --oplogLimit=1666962000 \
> --oplogFile=oplog/local/oplog.rs.bson
2022-10-28T22:15:37.282+0900 preparing collections to restore from
2022-10-28T22:15:37.283+0900 reading metadata for test.inventory from dump/test/inventory.metadata.json
2022-10-28T22:15:37.283+0900 reading metadata for config.external_validation_keys from dump/config/external_validation_keys.metadata.json
2022-10-28T22:15:37.283+0900 reading metadata for config.system.preimages from dump/config/system.preimages.metadata.json
2022-10-28T22:15:37.283+0900 reading metadata for config.tenantMigrationDonors from dump/config/tenantMigrationDonors.metadata.json
2022-10-28T22:15:37.283+0900 reading metadata for config.tenantMigrationRecipients from dump/config/tenantMigrationRecipients.metadata.json
2022-10-28T22:15:37.284+0900 dropping collection test.inventory before restoring
2022-10-28T22:15:37.288+0900 dropping collection config.external_validation_keys before restoring
2022-10-28T22:15:37.288+0900 cannot drop system collection config.system.preimages, skipping
2022-10-28T22:15:37.288+0900 restoring config.system.preimages from dump/config/system.preimages.bson
2022-10-28T22:15:37.290+0900 dropping collection config.tenantMigrationDonors before restoring
2022-10-28T22:15:37.303+0900 finished restoring config.system.preimages (0 documents, 0 failures)
2022-10-28T22:15:37.303+0900 dropping collection config.tenantMigrationRecipients before restoring
2022-10-28T22:15:37.322+0900 restoring test.inventory from dump/test/inventory.bson
2022-10-28T22:15:37.342+0900 restoring config.tenantMigrationDonors from dump/config/tenantMigrationDonors.bson
2022-10-28T22:15:37.352+0900 restoring config.external_validation_keys from dump/config/external_validation_keys.bson
2022-10-28T22:15:37.360+0900 finished restoring config.tenantMigrationDonors (0 documents, 0 failures)
2022-10-28T22:15:37.360+0900 finished restoring test.inventory (5 documents, 0 failures)
2022-10-28T22:15:37.364+0900 finished restoring config.external_validation_keys (0 documents, 0 failures)
2022-10-28T22:15:37.377+0900 restoring config.tenantMigrationRecipients from dump/config/tenantMigrationRecipients.bson
2022-10-28T22:15:37.389+0900 finished restoring config.tenantMigrationRecipients (0 documents, 0 failures)
2022-10-28T22:15:37.389+0900 replaying oplog
2022-10-28T22:15:37.424+0900 skipping applying the config.transactions namespace in applyOps
2022-10-28T22:15:37.425+0900 applied 25 oplog entries
2022-10-28T22:15:37.426+0900 no indexes to restore for collection test.inventory
2022-10-28T22:15:37.426+0900 restoring indexes for collection config.tenantMigrationDonors from metadata
2022-10-28T22:15:37.426+0900 index: &idx.IndexDocument{Options:primitive.M{"expireAfterSeconds":0, "name":"TenantMigrationDonorTTLIndex", "v":2}, Key:primitive.D{primitive.E{Key:"expireAt", Value:1}}, PartialFilterExpression:primitive.D(nil)}
2022-10-28T22:15:37.426+0900 restoring indexes for collection config.tenantMigrationRecipients from metadata
2022-10-28T22:15:37.426+0900 index: &idx.IndexDocument{Options:primitive.M{"expireAfterSeconds":0, "name":"TenantMigrationRecipientTTLIndex", "v":2}, Key:primitive.D{primitive.E{Key:"expireAt", Value:1}}, PartialFilterExpression:primitive.D(nil)}
2022-10-28T22:15:37.426+0900 restoring indexes for collection config.transactions from metadata
2022-10-28T22:15:37.426+0900 index: &idx.IndexDocument{Options:primitive.M{"name":"parent_lsid", "v":2}, Key:primitive.D{primitive.E{Key:"parentLsid", Value:1}, primitive.E{Key:"_id.txnNumber", Value:1}, primitive.E{Key:"_id", Value:1}}, PartialFilterExpression:primitive.D{primitive.E{Key:"parentLsid", Value:primitive.D{primitive.E{Key:"$exists", Value:true}}}}}
2022-10-28T22:15:37.426+0900 no indexes to restore for collection config.image_collection
2022-10-28T22:15:37.426+0900 no indexes to restore for collection config.system.indexBuilds
2022-10-28T22:15:37.426+0900 restoring indexes for collection config.system.sessions from metadata
2022-10-28T22:15:37.426+0900 index: &idx.IndexDocument{Options:primitive.M{"expireAfterSeconds":1800, "name":"lsidTTLIndex", "v":2}, Key:primitive.D{primitive.E{Key:"lastUse", Value:1}}, PartialFilterExpression:primitive.D(nil)}
2022-10-28T22:15:37.429+0900 restoring indexes for collection config.external_validation_keys from metadata
2022-10-28T22:15:37.429+0900 index: &idx.IndexDocument{Options:primitive.M{"expireAfterSeconds":0, "name":"ExternalKeysTTLIndex", "v":2}, Key:primitive.D{primitive.E{Key:"ttlExpiresAt", Value:1}}, PartialFilterExpression:primitive.D(nil)}
2022-10-28T22:15:37.430+0900 Failed: config.transactions: error creating indexes for config.transactions: createIndex error: (IllegalOperation) not allowed to create index on config.transactions
2022-10-28T22:15:37.430+0900 5 document(s) restored successfully. 0 document(s) failed to restore.
[root@linux010 ~]#
inventoryコレクションの件数が10件から8件に減った事を確認します。
[root@linux010 ~]# mongosh --quiet --eval 'db.inventory.find().count()' 8 [root@linux010 ~]#
