MongoDB PITR(Point In Time Recovery)

スポンサーリンク

MongoDB ServerのPoint In Time Recoveryの操作方法を説明します。Point In Time Recoveryとはある時点のデータに戻すリストア手法です。MongoDB用語で説明すれば、mongodumpによるバックアップ取得後にoplogを適用して任意の時刻の状態に復旧させる操作です。IPA試験で出題されるようなベンダー非依存の表現を用いれば、バックアップ取得後に更新後ログを適用して復旧させる操作です。

前提

公式ドキュメント

参考になる公式ドキュメントを以下に示します。

動作確認済環境

  • Rocky Linux 8.6
  • MongoDB Server 6.0.2

構成図

以下の構成で動作確認をします。

                   172.16.1.0/24
        +-----------------+-----------------+
        |                 |                 |
 ens192 | .10      ens192 | .20      ens192 | .30
 +------+------+   +------+------+   +------+------+ 
 |  linux010   |   |  linux020   |   |  linux030   | 
 | (Rocky8.6)  |   | (Rocky8.6)  |   | (Rocky8.6)  | 
 | (Primary)   |   | (Secondary) |   | (Secondary) | 
 +-------------+   +-------------+   +-------------+

事前設定

3台のMongoDB Serverでレプリケーションが組めるように、/etc/mongod.confを記述します。以下に/etc/mongod.confの設定の要所を記します。

#vi /etc/mongod.conf 

# network interfaces
net:
  port: 27017
  bindIp: 0.0.0.0

#replication:
replication:
  replSetName: "rs0"
  oplogSizeMB: 1024

以下のような設定を投入し、レプリケーションを開始します。

rs.initiate(
  {
    _id: "rs0",
    members: [
      { _id : 0, host : "172.16.1.10:27017", priority : 4 },
      { _id : 1, host : "172.16.1.20:27017", priority : 2 },
      { _id : 2, host : "172.16.1.30:27017", priority : 1 }
    ]
  }
)

バックアップ操作とテストデータ投入

操作1. 5件のデータ投入とdump取得

テスト用のデータ5件を登録します。

mongosh --quiet test << EOF
db.inventory.insertMany( [
   { item: "canvas", qty: 100, size: { h: 28, w: 35.5, uom: "cm" }, status: "A" },
   { item: "journal", qty: 25, size: { h: 14, w: 21, uom: "cm" }, status: "A" },
   { item: "mat", qty: 85, size: { h: 27.9, w: 35.5, uom: "cm" }, status: "A" },
   { item: "mousepad", qty: 25, size: { h: 19, w: 22.85, uom: "cm" }, status: "P" },
   { item: "notebook", qty: 50, size: { h: 8.5, w: 11, uom: "in" }, status: "P" },
] );
EOF

この時点のバックアップ(dump)を取得します。操作例は以下の通りです。

mongodump \
  --host=127.0.0.1 \
  --port=27017 \
  --out=dump

操作ログは以下の通りです。この時の操作時刻をメモに控えておきましょう。以下出力の場合は21:20です。

[root@linux010 ~]# date
2022年 10月 28日 金曜日 21:20:30 JST
[root@linux010 ~]# mongodump \
>   --host=127.0.0.1 \
>   --port=27017 \
>   --out=dump
2022-10-28T21:20:30.831+0900  writing admin.system.version to dump/admin/system.version.bson
2022-10-28T21:20:30.833+0900  done dumping admin.system.version (1 document)
2022-10-28T21:20:30.834+0900  writing test.inventory to dump/test/inventory.bson
2022-10-28T21:20:30.834+0900  writing config.tenantMigrationRecipients to dump/config/tenantMigrationRecipients.bson
2022-10-28T21:20:30.836+0900  writing config.tenantMigrationDonors to dump/config/tenantMigrationDonors.bson
2022-10-28T21:20:30.837+0900  writing config.external_validation_keys to dump/config/external_validation_keys.bson
2022-10-28T21:20:30.840+0900  done dumping config.tenantMigrationRecipients (0 documents)
2022-10-28T21:20:30.840+0900  done dumping test.inventory (5 documents)
2022-10-28T21:20:30.842+0900  done dumping config.external_validation_keys (0 documents)
2022-10-28T21:20:30.842+0900  writing config.system.preimages to dump/config/system.preimages.bson
2022-10-28T21:20:30.843+0900  done dumping config.tenantMigrationDonors (0 documents)
2022-10-28T21:20:30.843+0900  done dumping config.system.preimages (0 documents)
[root@linux010 ~]# 

念の為、dumpを取得できている事を確認します。

[root@linux010 ~]# ls -l dump/
合計 0
drwxr-xr-x 2 root root  69 10月 28 21:20 admin
drwxr-xr-x 2 root root 318 10月 28 21:20 config
drwxr-xr-x 2 root root  59 10月 28 21:20 test
[root@linux010 ~]# 

操作2. 3件のデータ投入

3件のデータを投入します。

mongosh --quiet test << EOF
db.inventory.insertMany( [
   { item: "paper", qty: 100, size: { h: 8.5, w: 11, uom: "in" }, status: "D" },
   { item: "planner", qty: 75, size: { h: 22.85, w: 30, uom: "cm" }, status: "D" },
   { item: "postcard", qty: 45, size: { h: 10, w: 15.25, uom: "cm" }, status: "A" },
] );
EOF

操作ログは以下の通りです。この時の操作時刻をメモに控えておきましょう。以下出力の場合は21:40です。

[root@linux010 ~]# date
2022年 10月 28日 金曜日 21:40:03 JST
[root@linux010 ~]# mongosh --quiet test << EOF
> db.inventory.insertMany( [
>    { item: "paper", qty: 100, size: { h: 8.5, w: 11, uom: "in" }, status: "D" },
>    { item: "planner", qty: 75, size: { h: 22.85, w: 30, uom: "cm" }, status: "D" },
>    { item: "postcard", qty: 45, size: { h: 10, w: 15.25, uom: "cm" }, status: "A" },
> ] );
> EOF
rs0 [direct: primary] test> db.inventory.insertMany( [
...    { item: "paper", qty: 100, size: { h: 8.5, w: 11, uom: "in" }, status: "D" },
...    { item: "planner", qty: 75, size: { h: 22.85, w: 30, uom: "cm" }, status: "D" },
...    { item: "postcard", qty: 45, size: { h: 10, w: 15.25, uom: "cm" }, status: "A" },
... ] );
{
  acknowledged: true,
  insertedIds: {
    '0': ObjectId("635bcda7b9197bae6f89d758"),
    '1': ObjectId("635bcda7b9197bae6f89d759"),
    '2': ObjectId("635bcda7b9197bae6f89d75a")
  }
}
rs0 [direct: primary] test> [root@linux010 ~]# 

操作3. 2件のデータ投入とoplog取得

2件のデータを投入します。

mongosh --quiet test << EOF
db.inventory.insertMany( [
   { item: "sketchbook", qty: 80, size: { h: 14, w: 21, uom: "cm" }, status: "A" },
   { item: "sketch pad", qty: 95, size: { h: 22.85, w: 30.5, uom: "cm" }, status: "A" }
] );
EOF

操作ログは以下の通りです。この時の操作時刻をメモに控えておきましょう。以下出力の場合は22:10です。

[root@linux010 ~]# date
2022年 10月 28日 金曜日 22:10:30 JST
[root@linux010 ~]# mongosh --quiet test << EOF
> db.inventory.insertMany( [
>    { item: "sketchbook", qty: 80, size: { h: 14, w: 21, uom: "cm" }, status: "A" },
>    { item: "sketch pad", qty: 95, size: { h: 22.85, w: 30.5, uom: "cm" }, status: "A" }
> ] );
> EOF
rs0 [direct: primary] test> db.inventory.insertMany( [
...    { item: "sketchbook", qty: 80, size: { h: 14, w: 21, uom: "cm" }, status: "A" },
...    { item: "sketch pad", qty: 95, size: { h: 22.85, w: 30.5, uom: "cm" }, status: "A" }
... ] );
{
  acknowledged: true,
  insertedIds: {
    '0': ObjectId("635bd4ccb7ee5d99e8af0cb8"),
    '1': ObjectId("635bd4ccb7ee5d99e8af0cb9")
  }
}
rs0 [direct: primary] test> [root@linux010 ~]# 

10件のデータが登録された時点のoplogを取得します。操作例は以下の通りです。

mongodump \
  --host=127.0.0.1 \
  --port=27017 \
  --db=local \
  --collection=oplog.rs \
  --out=oplog

確かにoplogを取得できている事を確認します。

[root@linux010 ~]# ls -l oplog/local/
合計 44
-rw-r--r-- 1 root root 39638 10月 28 22:10 oplog.rs.bson
-rw-r--r-- 1 root root   185 10月 28 22:10 oplog.rs.metadata.json
[root@linux010 ~]# bsondump oplog/local/oplog.rs.bson | tail -n 3
2022-10-28T22:11:37.735+0900  337 objects found
{"lsid":{"id":{"$binary":{"base64":"349nCBjHSxSzcqHad9Y9vA==","subType":"04"}},"uid":{"$binary":{"base64":"47DEQpj8HBSa+/TImW+5JCeuQeRkm5NMpJWZG3hSuFU=","subType":"00"}}},"txnNumber":{"$numberLong":"1"},"op":"i","ns":"test.inventory","ui":{"$binary":{"base64":"nFqbQjS0RmOmXKaJZ17Ipw==","subType":"04"}},"o":{"_id":{"$oid":"635bd4ccb7ee5d99e8af0cb8"},"item":"sketchbook","qty":{"$numberInt":"80"},"size":{"h":{"$numberInt":"14"},"w":{"$numberInt":"21"},"uom":"cm"},"status":"A"},"o2":{"_id":{"$oid":"635bd4ccb7ee5d99e8af0cb8"}},"stmtId":{"$numberInt":"0"},"ts":{"$timestamp":{"t":1666962636,"i":1}},"t":{"$numberLong":"1"},"v":{"$numberLong":"2"},"wall":{"$date":{"$numberLong":"1666962636673"}},"prevOpTime":{"ts":{"$timestamp":{"t":0,"i":0}},"t":{"$numberLong":"-1"}}}
{"lsid":{"id":{"$binary":{"base64":"349nCBjHSxSzcqHad9Y9vA==","subType":"04"}},"uid":{"$binary":{"base64":"47DEQpj8HBSa+/TImW+5JCeuQeRkm5NMpJWZG3hSuFU=","subType":"00"}}},"txnNumber":{"$numberLong":"1"},"op":"i","ns":"test.inventory","ui":{"$binary":{"base64":"nFqbQjS0RmOmXKaJZ17Ipw==","subType":"04"}},"o":{"_id":{"$oid":"635bd4ccb7ee5d99e8af0cb9"},"item":"sketch pad","qty":{"$numberInt":"95"},"size":{"h":{"$numberDouble":"22.85"},"w":{"$numberDouble":"30.5"},"uom":"cm"},"status":"A"},"o2":{"_id":{"$oid":"635bd4ccb7ee5d99e8af0cb9"}},"stmtId":{"$numberInt":"1"},"ts":{"$timestamp":{"t":1666962636,"i":2}},"t":{"$numberLong":"1"},"v":{"$numberLong":"2"},"wall":{"$date":{"$numberLong":"1666962636673"}},"prevOpTime":{"ts":{"$timestamp":{"t":1666962636,"i":1}},"t":{"$numberLong":"1"}}}
{"op":"n","ns":"","o":{"msg":"periodic noop"},"ts":{"$timestamp":{"t":1666962651,"i":1}},"t":{"$numberLong":"1"},"v":{"$numberLong":"2"},"wall":{"$date":{"$numberLong":"1666962651680"}}}
[root@linux010 ~]# 

リストア操作

それでは操作2の時点に戻すpoint-in-time recoveryに挑戦してみましょう。

まずはリストア前のinventoryコレクションの件数を確認します。これが10件から8件に減れば想定通りのリストアがなされています。

[root@linux010 ~]# mongosh --quiet --eval 'db.inventory.find().count()'
10
[root@linux010 ~]# 

22:00時点に戻せば操作2時点になります。mongorestoreコマンドはpoint-in-time recoveryで戻す時刻をepoc timeで指定する必要がありますので、22:00のepoc timeを調べます。操作例は以下の通りです。

[root@linux010 ~]# date -d "2022/10/28 22:00" +%s
1666958400
[root@linux010 ~]# 

想定外のデータを消すために、mongorestoreコマンドにdropオプションを付与します。また、point-in-time recoveryをするには、

mongorestore \
  --host=127.0.0.1 \
  --port=27017 \
  --drop \
  --dir=dump \
  --oplogReplay \
  --oplogLimit=1666962000 \
  --oplogFile=oplog/local/oplog.rs.bson

操作ログは以下の通りです。多くのログが出力されますが、末尾にrestored successfullyと表示される事は最低限確認しておきましょう。

[root@linux010 ~]# mongorestore \
>   --host=127.0.0.1 \
>   --port=27017 \
>   --drop \
>   --dir=dump \
>   --oplogReplay \
>   --oplogLimit=1666962000 \
>   --oplogFile=oplog/local/oplog.rs.bson
2022-10-28T22:15:37.282+0900  preparing collections to restore from
2022-10-28T22:15:37.283+0900  reading metadata for test.inventory from dump/test/inventory.metadata.json
2022-10-28T22:15:37.283+0900  reading metadata for config.external_validation_keys from dump/config/external_validation_keys.metadata.json
2022-10-28T22:15:37.283+0900  reading metadata for config.system.preimages from dump/config/system.preimages.metadata.json
2022-10-28T22:15:37.283+0900  reading metadata for config.tenantMigrationDonors from dump/config/tenantMigrationDonors.metadata.json
2022-10-28T22:15:37.283+0900  reading metadata for config.tenantMigrationRecipients from dump/config/tenantMigrationRecipients.metadata.json
2022-10-28T22:15:37.284+0900  dropping collection test.inventory before restoring
2022-10-28T22:15:37.288+0900  dropping collection config.external_validation_keys before restoring
2022-10-28T22:15:37.288+0900  cannot drop system collection config.system.preimages, skipping
2022-10-28T22:15:37.288+0900  restoring config.system.preimages from dump/config/system.preimages.bson
2022-10-28T22:15:37.290+0900  dropping collection config.tenantMigrationDonors before restoring
2022-10-28T22:15:37.303+0900  finished restoring config.system.preimages (0 documents, 0 failures)
2022-10-28T22:15:37.303+0900  dropping collection config.tenantMigrationRecipients before restoring
2022-10-28T22:15:37.322+0900  restoring test.inventory from dump/test/inventory.bson
2022-10-28T22:15:37.342+0900  restoring config.tenantMigrationDonors from dump/config/tenantMigrationDonors.bson
2022-10-28T22:15:37.352+0900  restoring config.external_validation_keys from dump/config/external_validation_keys.bson
2022-10-28T22:15:37.360+0900  finished restoring config.tenantMigrationDonors (0 documents, 0 failures)
2022-10-28T22:15:37.360+0900  finished restoring test.inventory (5 documents, 0 failures)
2022-10-28T22:15:37.364+0900  finished restoring config.external_validation_keys (0 documents, 0 failures)
2022-10-28T22:15:37.377+0900  restoring config.tenantMigrationRecipients from dump/config/tenantMigrationRecipients.bson
2022-10-28T22:15:37.389+0900  finished restoring config.tenantMigrationRecipients (0 documents, 0 failures)
2022-10-28T22:15:37.389+0900  replaying oplog
2022-10-28T22:15:37.424+0900  skipping applying the config.transactions namespace in applyOps
2022-10-28T22:15:37.425+0900  applied 25 oplog entries
2022-10-28T22:15:37.426+0900  no indexes to restore for collection test.inventory
2022-10-28T22:15:37.426+0900  restoring indexes for collection config.tenantMigrationDonors from metadata
2022-10-28T22:15:37.426+0900  index: &idx.IndexDocument{Options:primitive.M{"expireAfterSeconds":0, "name":"TenantMigrationDonorTTLIndex", "v":2}, Key:primitive.D{primitive.E{Key:"expireAt", Value:1}}, PartialFilterExpression:primitive.D(nil)}
2022-10-28T22:15:37.426+0900  restoring indexes for collection config.tenantMigrationRecipients from metadata
2022-10-28T22:15:37.426+0900  index: &idx.IndexDocument{Options:primitive.M{"expireAfterSeconds":0, "name":"TenantMigrationRecipientTTLIndex", "v":2}, Key:primitive.D{primitive.E{Key:"expireAt", Value:1}}, PartialFilterExpression:primitive.D(nil)}
2022-10-28T22:15:37.426+0900  restoring indexes for collection config.transactions from metadata
2022-10-28T22:15:37.426+0900  index: &idx.IndexDocument{Options:primitive.M{"name":"parent_lsid", "v":2}, Key:primitive.D{primitive.E{Key:"parentLsid", Value:1}, primitive.E{Key:"_id.txnNumber", Value:1}, primitive.E{Key:"_id", Value:1}}, PartialFilterExpression:primitive.D{primitive.E{Key:"parentLsid", Value:primitive.D{primitive.E{Key:"$exists", Value:true}}}}}
2022-10-28T22:15:37.426+0900  no indexes to restore for collection config.image_collection
2022-10-28T22:15:37.426+0900  no indexes to restore for collection config.system.indexBuilds
2022-10-28T22:15:37.426+0900  restoring indexes for collection config.system.sessions from metadata
2022-10-28T22:15:37.426+0900  index: &idx.IndexDocument{Options:primitive.M{"expireAfterSeconds":1800, "name":"lsidTTLIndex", "v":2}, Key:primitive.D{primitive.E{Key:"lastUse", Value:1}}, PartialFilterExpression:primitive.D(nil)}
2022-10-28T22:15:37.429+0900  restoring indexes for collection config.external_validation_keys from metadata
2022-10-28T22:15:37.429+0900  index: &idx.IndexDocument{Options:primitive.M{"expireAfterSeconds":0, "name":"ExternalKeysTTLIndex", "v":2}, Key:primitive.D{primitive.E{Key:"ttlExpiresAt", Value:1}}, PartialFilterExpression:primitive.D(nil)}
2022-10-28T22:15:37.430+0900  Failed: config.transactions: error creating indexes for config.transactions: createIndex error: (IllegalOperation) not allowed to create index on config.transactions
2022-10-28T22:15:37.430+0900  5 document(s) restored successfully. 0 document(s) failed to restore.
[root@linux010 ~]# 

inventoryコレクションの件数が10件から8件に減った事を確認します。

[root@linux010 ~]# mongosh --quiet --eval 'db.inventory.find().count()'
8
[root@linux010 ~]# 
タイトルとURLをコピーしました