溫馨提示×

溫馨提示×

您好，登錄后才能下訂單哦！

密碼登錄×
登錄注冊×
其他方式登錄
點擊 登錄注冊 即表示同意《億速云用戶服務條款》

mongo副本集mongos啟動報錯

發(fā)布時間：2020-06-13 15:07:05 來源：網(wǎng)絡 閱讀：1883 作者：hanshilei1032 欄目：MongoDB數(shù)據(jù)庫

背景

項目采用mongo副本集的形式存儲數(shù)據(jù)，經(jīng)常出現(xiàn)無故斷電導致某個副本mongos啟動不起來的問題。

環(huán)境介紹

mongodb副本集：
mongo01：192.168.36.218
mongo02：192.168.36.219
mongo03：192.168.36.220

報錯信息

mongo03副本上mongos啟動不起來，執(zhí)行命令報錯：

[root@localhost ~]# mongos --configdb 192.168.36.218:20000,192.168.36.219:20000,192.168.36.220:20000 --port 30000 --chunkSize 500 --logpath /home/mongo/logs/mongos.log --logappend --fork
about to fork child process, waiting until server is ready for connections.
forked process: 79748
ERROR: child process failed, exited with error number 5

查看mongos.log，錯誤信息如下：

2018-06-25T09:13:47.607+0800 I CONTROL  [main] ***** SERVER RESTARTED *****
2018-06-25T09:13:47.612+0800 I CONTROL  [main] ** WARNING: You are running this process as the root user, which is not recommended.
2018-06-25T09:13:47.613+0800 I CONTROL  [main] 
2018-06-25T09:13:47.613+0800 I SHARDING [mongosMain] MongoS version 3.2.1 starting: pid=80904 port=30000 64-bit host=MongoDB03 (--help for usage)
2018-06-25T09:13:47.613+0800 I CONTROL  [mongosMain] db version v3.2.1
2018-06-25T09:13:47.613+0800 I CONTROL  [mongosMain] git version: a14d55980c2cdc565d4704a7e3ad37e4e535c1b2
2018-06-25T09:13:47.613+0800 I CONTROL  [mongosMain] allocator: tcmalloc
2018-06-25T09:13:47.613+0800 I CONTROL  [mongosMain] modules: none
2018-06-25T09:13:47.613+0800 I CONTROL  [mongosMain] build environment:
2018-06-25T09:13:47.613+0800 I CONTROL  [mongosMain]     distarch: x86_64
2018-06-25T09:13:47.613+0800 I CONTROL  [mongosMain]     target_arch: x86_64
2018-06-25T09:13:47.613+0800 I CONTROL  [mongosMain] options: { net: { port: 30000 }, processManagement: { fork: true }, sharding: { chunkSize: 500, configDB: "192.168.36.218:20000,192.168.36.219:20000,192.168.36.220:20000" }, systemLog: { destination: "file", logAppend: true, path: "/home/mongo/logs/mongos.log" } }
2018-06-25T09:13:47.613+0800 I SHARDING [mongosMain] Updating config server connection string to: 192.168.36.218:20000,192.168.36.219:20000,192.168.36.220:20000
2018-06-25T09:13:47.625+0800 W SHARDING [mongosMain] config servers 192.168.36.218:20000 and 192.168.36.220:20000 differ
2018-06-25T09:13:47.627+0800 W SHARDING [mongosMain] config servers 192.168.36.218:20000 and 192.168.36.220:20000 differ
2018-06-25T09:13:47.628+0800 W SHARDING [mongosMain] config servers 192.168.36.218:20000 and 192.168.36.220:20000 differ
2018-06-25T09:13:47.630+0800 W SHARDING [mongosMain] config servers 192.168.36.218:20000 and 192.168.36.220:20000 differ
2018-06-25T09:13:47.630+0800 E SHARDING [mongosMain] Error initializing sharding system: ConfigServersInconsistent hash from 192.168.36.218:20000: { chunks: "d41d8cd98f00b204e9800998ecf8427e", databases: "95954cb16c029767f4ad050712a28f49", shards: "68f4b37fec8c2ac97cc985aa01f37717", version: "b25e55c19a8c75c87b4f950dcf5eb088" } vs hash from 192.168.36.220:20000: {}

如上我們發(fā)現(xiàn)：

config servers 192.168.36.218:20000 and 192.168.36.220:20000 differ

配置服務器192.168.36.218:20000和損壞的配置服務器192.168.36.220:20000不一致，也就是說mongo01:20000和mongo03:20000上面的配置不一樣

修復

那我們該如何修復呢？其實我們可以將mongo01:20000上的config庫導入到mongo03:20000的config庫中，以解決上面的問題。

執(zhí)行過程

第一步：備份mongo01:20000的config庫

[root@localhost ~]#  mongodump --host 192.168.36.218:20000 -d config -o /home/config
2018-06-25T10:06:06.225+0800    writing config.actionlog to 
2018-06-25T10:06:06.226+0800    writing config.locks to 
2018-06-25T10:06:06.226+0800    writing config.mongos to 
2018-06-25T10:06:06.226+0800    writing config.lockpings to 
2018-06-25T10:06:06.227+0800    done dumping config.locks (3 documents)
2018-06-25T10:06:06.228+0800    done dumping config.lockpings (2 documents)
2018-06-25T10:06:06.229+0800    done dumping config.mongos (3 documents)
2018-06-25T10:06:06.229+0800    writing config.shards to 
2018-06-25T10:06:06.229+0800    writing config.settings to 
2018-06-25T10:06:06.229+0800    writing config.version to 
2018-06-25T10:06:06.230+0800    done dumping config.shards (1 document)
2018-06-25T10:06:06.230+0800    writing config.databases to 
2018-06-25T10:06:06.230+0800    done dumping config.settings (1 document)
2018-06-25T10:06:06.230+0800    done dumping config.version (1 document)
2018-06-25T10:06:06.230+0800    writing config.changelog to 
2018-06-25T10:06:06.230+0800    writing config.chunks to 
2018-06-25T10:06:06.231+0800    done dumping config.databases (1 document)
2018-06-25T10:06:06.231+0800    writing config.tags to 
2018-06-25T10:06:06.232+0800    done dumping config.chunks (0 documents)
2018-06-25T10:06:06.232+0800    done dumping config.changelog (1 document)
2018-06-25T10:06:06.232+0800    done dumping config.tags (0 documents)
2018-06-25T10:06:06.355+0800    done dumping config.actionlog (8160 documents)
[root@localhost ~]#

第二步：導入備份的config到mongo03:20000中

[root@localhost ~]#  mongorestore --host 192.168.36.220:20000 -d config /home/config/config
2018-06-25T10:08:09.136+0800    building a list of collections to restore from /home/config/config dir
2018-06-25T10:08:09.162+0800    reading metadata for config.actionlog from /home/config/config/actionlog.metadata.json
2018-06-25T10:08:09.163+0800    reading metadata for config.locks from /home/config/config/locks.metadata.json
2018-06-25T10:08:09.163+0800    reading metadata for config.changelog from /home/config/config/changelog.metadata.json
2018-06-25T10:08:09.164+0800    reading metadata for config.mongos from /home/config/config/mongos.metadata.json
2018-06-25T10:08:09.164+0800    restoring config.locks from /home/config/config/locks.bson
2018-06-25T10:08:09.165+0800    restoring config.mongos from /home/config/config/mongos.bson
2018-06-25T10:08:09.175+0800    error: multiple errors in bulk operation:
  - E11000 duplicate key error collection: config.mongos index: _id_ dup key: { : "MongoDB01:30000" }
  - E11000 duplicate key error collection: config.mongos index: _id_ dup key: { : "MongoDB02:30000" }

2018-06-25T10:08:09.175+0800    restoring indexes for collection config.mongos from metadata
2018-06-25T10:08:09.204+0800    finished restoring config.mongos (3 documents)
2018-06-25T10:08:09.204+0800    restoring indexes for collection config.locks from metadata
2018-06-25T10:08:09.204+0800    reading metadata for config.lockpings from /home/config/config/lockpings.metadata.json
2018-06-25T10:08:09.205+0800    restoring config.lockpings from /home/config/config/lockpings.bson
2018-06-25T10:08:09.215+0800    restoring config.actionlog from /home/config/config/actionlog.bson
2018-06-25T10:08:09.269+0800    restoring config.changelog from /home/config/config/changelog.bson
2018-06-25T10:08:09.284+0800    restoring indexes for collection config.changelog from metadata
2018-06-25T10:08:09.284+0800    finished restoring config.locks (3 documents)
2018-06-25T10:08:09.284+0800    reading metadata for config.shards from /home/config/config/shards.metadata.json
2018-06-25T10:08:09.284+0800    restoring config.shards from /home/config/config/shards.bson
2018-06-25T10:08:09.286+0800    finished restoring config.changelog (1 document)
2018-06-25T10:08:09.286+0800    reading metadata for config.version from /home/config/config/version.metadata.json
2018-06-25T10:08:09.286+0800    restoring config.version from /home/config/config/version.bson
2018-06-25T10:08:09.287+0800    error: multiple errors in bulk operation:
  - E11000 duplicate key error collection: config.lockpings index: _id_ dup key: { : "MongoDB01:30000:1523346496:1804289383" }
  - E11000 duplicate key error collection: config.lockpings index: _id_ dup key: { : "MongoDB02:30000:1529566939:1804289383" }

2018-06-25T10:08:09.287+0800    restoring indexes for collection config.lockpings from metadata
2018-06-25T10:08:09.297+0800    restoring indexes for collection config.shards from metadata
2018-06-25T10:08:09.318+0800    finished restoring config.lockpings (2 documents)
2018-06-25T10:08:09.318+0800    reading metadata for config.databases from /home/config/config/databases.metadata.json
2018-06-25T10:08:09.318+0800    restoring config.databases from /home/config/config/databases.bson
2018-06-25T10:08:09.330+0800    restoring indexes for collection config.version from metadata
2018-06-25T10:08:09.370+0800    finished restoring config.shards (1 document)
2018-06-25T10:08:09.370+0800    reading metadata for config.settings from /home/config/config/settings.metadata.json
2018-06-25T10:08:09.370+0800    restoring config.settings from /home/config/config/settings.bson
2018-06-25T10:08:09.372+0800    finished restoring config.version (1 document)
2018-06-25T10:08:09.372+0800    reading metadata for config.tags from /home/config/config/tags.metadata.json
2018-06-25T10:08:09.372+0800    restoring config.tags from /home/config/config/tags.bson
2018-06-25T10:08:09.374+0800    restoring indexes for collection config.tags from metadata
2018-06-25T10:08:09.382+0800    restoring indexes for collection config.databases from metadata
2018-06-25T10:08:09.400+0800    finished restoring config.tags (0 documents)
2018-06-25T10:08:09.400+0800    reading metadata for config.chunks from /home/config/config/chunks.metadata.json
2018-06-25T10:08:09.400+0800    restoring config.chunks from /home/config/config/chunks.bson
2018-06-25T10:08:09.409+0800    restoring indexes for collection config.settings from metadata
2018-06-25T10:08:09.409+0800    finished restoring config.databases (1 document)
2018-06-25T10:08:09.411+0800    finished restoring config.settings (1 document)
2018-06-25T10:08:09.555+0800    restoring indexes for collection config.chunks from metadata
2018-06-25T10:08:09.581+0800    finished restoring config.chunks (0 documents)
2018-06-25T10:08:09.754+0800    restoring indexes for collection config.actionlog from metadata
2018-06-25T10:08:09.754+0800    finished restoring config.actionlog (8160 documents)
2018-06-25T10:08:09.754+0800    done
[root@localhost ~]#

注：我是在生產(chǎn)環(huán)境中執(zhí)行的，當時把mongo01:20000和mongo02:20000上面的config庫全部備份了一下，然后按順序全部導入到了mongo03:20000中(備份和導入命令同上)。按理說應該只導入mongo01:20000就可以了，由于是生產(chǎn)環(huán)境，我就不加以測試了，等到下次再出現(xiàn)這種問題再試試吧。同時也歡迎大家測試一下，然后回復到評論區(qū)，幫助大家解決一下問題。

嘗試啟動mongo03上的mongos

[root@localhost ~]#  mongos --configdb 192.168.36.218:20000,192.168.36.219:20000,192.168.36.220:20000 --port 30000 --chunkSize 500 --logpath /home/mongo/logs/mongos.log --logappend --fork
about to fork child process, waiting until server is ready for connections.
forked process: 4286
child process started successfully, parent exiting
[root@localhost ~]# ps -ef | grep mongos
root     44760     1  0 Jun21 ?        00:25:14 mongos --configdb 192.168.36.218:20000,192.168.36.219:20000,192.168.36.220:20000 --port 30000 --chunkSize 500 --logpath /home/mongo/logs/mongos.log --logappend --fork
root     66128 66090  0 11:40 pts/0    00:00:00 grep mongos
[root@localhost ~]#

如上，我們可以看到mongos成功啟動并在后臺保持運行。

向AI問一下細節(jié)

免責聲明：本站發(fā)布的內(nèi)容（圖片、視頻和文字）以原創(chuàng)、轉載和分享為主，文章觀點不代表本網(wǎng)站立場，如果涉及侵權請聯(lián)系站長郵箱：is@yisu.com進行舉報，并提供相關證據(jù)，一經(jīng)查實，將立刻刪除涉嫌侵權內(nèi)容。

AI
助
手