您好,登錄后才能下訂單哦!
測(cè)試環(huán)境,由于測(cè)試需求,重新format namenode后,導(dǎo)致datanode無法正常啟動(dòng)。
1. 查看datanode日志,可以發(fā)現(xiàn)錯(cuò)誤“Initialization failed for Block pool <registering> (Datanode Uuid unassigned)”
2018-01-27 20:09:49,052 FATAL org.apache.hadoop.hdfs.server.datanode.DataNode: Initialization failed for Block pool <registering> (Datanode Uuid unassigned) service to c6704/192.168.67.104:9000. Exiting. java.io.IOException: All specified directories are failed to load. at org.apache.hadoop.hdfs.server.datanode.DataStorage.recoverTransitionRead(DataStorage.java:478) at org.apache.hadoop.hdfs.server.datanode.DataNode.initStorage(DataNode.java:1361) at org.apache.hadoop.hdfs.server.datanode.DataNode.initBlockPool(DataNode.java:1326) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.verifyAndSetNamespaceInfo(BPOfferService.java:317) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.connectToNNAndHandshake(BPServiceActor.java:223) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:801) at java.lang.Thread.run(Thread.java:745) 2018-01-27 20:09:49,056 FATAL org.apache.hadoop.hdfs.server.datanode.DataNode: Initialization failed for Block pool <registering> (Datanode Uuid unassigned) service to c6705/192.168.67.105:9000. Exiting. java.io.IOException: All specified directories are failed to load. at org.apache.hadoop.hdfs.server.datanode.DataStorage.recoverTransitionRead(DataStorage.java:478) at org.apache.hadoop.hdfs.server.datanode.DataNode.initStorage(DataNode.java:1361) at org.apache.hadoop.hdfs.server.datanode.DataNode.initBlockPool(DataNode.java:1326) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.verifyAndSetNamespaceInfo(BPOfferService.java:317) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.connectToNNAndHandshake(BPServiceActor.java:223) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:801) at java.lang.Thread.run(Thread.java:745) 2018-01-27 20:09:49,069 WARN org.apache.hadoop.hdfs.server.datanode.DataNode: Ending block pool service for: Block pool <registering> (Datanode Uuid unassigned) service to c6705/192.168.67.105:9000 2018-01-27 20:09:49,070 WARN org.apache.hadoop.hdfs.server.datanode.DataNode: Ending block pool service for: Block pool <registering> (Datanode Uuid unassigned) service to c6704/192.168.67.104:9000 2018-01-27 20:09:49,192 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Removed Block pool <registering> (Datanode Uuid unassigned) 2018-01-27 20:09:51,193 WARN org.apache.hadoop.hdfs.server.datanode.DataNode: Exiting Datanode 2018-01-27 20:09:51,204 INFO org.apache.hadoop.util.ExitUtil: Exiting with status 0 2018-01-27 20:09:51,208 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: SHUTDOWN_MSG: /************************************************************ SHUTDOWN_MSG: Shutting down DataNode at c6706.python279.org/192.168.67.106 ************************************************************/
2. 經(jīng)過百度,根據(jù)日志描述,原因是datanode的clusterID 和 namenode的clusterID 不匹配。
打開hdfs-site.xml中關(guān)于datanode和namenode對(duì)應(yīng)的目錄,分別打開其中的current/VERSION文件,進(jìn)行對(duì)比。
3. namenode的VERSION內(nèi)容如下:
[hdfs@c6704 $ cat /data/hadoop/hdfs/name/current/VERSION #Sat Jan 27 00:46:30 UTC 2018 namespaceID=1148548909 clusterID=CID-aedb2e82-77f2-4056-b676-dca88083215d cTime=0 storageType=NAME_NODE blockpoolID=BP-1099214307-192.168.67.104-1517013990445 layoutVersion=-63
4. datanode的VERSION文件內(nèi)容如下:
[hdfs@c6706 ~]$ cat /data/hadoop/hdfs/data/current/VERSION #Sat Jan 27 00:20:21 UTC 2018 storageID=DS-8f0fdd04-e967-43cd-bd41-93b826b675b8 clusterID=CID-b27ecfd8-64ba-4e43-bd82-4ef6f2edd60c cTime=0 datanodeUuid=264b1b43-82c0-411c-859f-32761edc7465 storageType=DATA_NODE layoutVersion=-56 5. namenode和datano
de的版本是不同的,決定備份datanode,并清空VERSION,然后啟動(dòng)datanode,問題依舊。檢查VERSION,內(nèi)容是空的。
[hdfs@c6706 current]$ cp VERSION VERSION.bk [hdfs@c6706 current]$ echo > VERSION [hdfs@c6706 current]$ cat VERSION
6. 刪除VERSION,再次啟動(dòng)datanode,VERSION內(nèi)容已經(jīng)同步。
$ cat VERSION #Sun Jan 28 01:29:46 UTC 2018 storageID=DS-1c1f5e05-df2c-40de-b39b-d6d54e3c4894 clusterID=CID-aedb2e82-77f2-4056-b676-dca88083215d ##<<<<<同步了 cTime=0 datanodeUuid=948d5780-053e-4752-9476-fb1d1debda72 storageType=DATA_NODE layoutVersion=-56
7. 通過頁面也可以查詢到datanode了。
8. 問題原因
執(zhí)行hdfs namenode -format后,current目錄會(huì)刪除并重新生成,其中VERSION文件中的clusterID也會(huì)隨之變化,而datanode的VERSION文件中的clusterID保持不變,造成兩個(gè)clusterID不一致。
所以為了避免這種情況,可以再執(zhí)行的namenode格式化之后,刪除datanode的current文件夾,或者修改datanode的VERSION文件中出clusterID與namenode的VERSION文件中的clusterID一樣,然后重新啟動(dòng)datanode。
參考:
http://blog.csdn.net/liuxinghao/article/details/40121843
免責(zé)聲明:本站發(fā)布的內(nèi)容(圖片、視頻和文字)以原創(chuàng)、轉(zhuǎn)載和分享為主,文章觀點(diǎn)不代表本網(wǎng)站立場(chǎng),如果涉及侵權(quán)請(qǐng)聯(lián)系站長(zhǎng)郵箱:is@yisu.com進(jìn)行舉報(bào),并提供相關(guān)證據(jù),一經(jīng)查實(shí),將立刻刪除涉嫌侵權(quán)內(nèi)容。