1 启动异常现象
1）通过hbase shell执行命令，发现处于“Master is initializing”状态。
2）Hbase集群初始化异常，namespace table 和meta table长期处于transition状态。
2016-11-24 13:05:45,738 INFO org.apache.hadoop.hbase.zookeeper.MetaTableLocator: Deleting hbase:meta region location in ZooKeeper
2016-11-24 13:05:45,740 INFO org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper: Node /chenqun/meta-region-server already deleted, retry=false
2016-11-24 13:05:45,764 WARN org.apache.hadoop.hbase.zookeeper.ZKTableStateManager: Moving table hbase:meta state from null to ENABLED
2016-11-24 13:05:45,775 INFO org.apache.hadoop.hbase.master.AssignmentManager: Assigning hbase:meta,,1.1588230740 to hadoop-dn-003,60020,1479963874824
2016-11-24 13:05:45,775 INFO org.apache.hadoop.hbase.master.RegionStates: Transition {1588230740 state=OFFLINE, ts=1479963945754, server=null} to {1588230740 state=PENDING_OPEN, ts=1479963945775, server=hadoop-dn-003,60020,1479963874824}
2016-11-24 13:05:45,856 INFO org.apache.hadoop.hbase.master.ServerManager: AssignmentManager hasn't finished failover cleanup; waiting
2016-11-24 13:05:45,870 INFO org.apache.hadoop.hbase.master.RegionStates: Transition {1588230740 state=PENDING_OPEN, ts=1479963945775, server=hadoop-dn-003,60020,1479963874824} to {1588230740 state=OPENING, ts=1479963945870, server=hadoop-dn-003,60020,1479963874824}
2016-11-24 13:05:46,182 INFO org.apache.hadoop.hbase.master.RegionStates: Transition {1588230740 state=OPENING, ts=1479963945870, server=hadoop-dn-003,60020,1479963874824} to {1588230740 state=OPEN, ts=1479963946182, server=hadoop-dn-003,60020,1479963874824}
2016-11-24 13:05:46,184 INFO org.apache.hadoop.hbase.coordination.ZkOpenRegionCoordination: Handling OPENED of 1588230740 from hadoop-nn-002,60000,1479963871806; deleting unassigned node
2016-11-24 13:05:46,189 INFO org.apache.hadoop.hbase.master.HMaster: hbase:meta assigned=1, rit=false, location=hadoop-dn-003,60020,1479963874824
2016-11-24 13:05:46,329 INFO org.apache.hadoop.hbase.MetaMigrationConvertingToPB: hbase:meta doesn't have any entries to update.
2016-11-24 13:05:46,329 INFO org.apache.hadoop.hbase.MetaMigrationConvertingToPB: META already up-to date with PB serialization
2016-11-24 13:05:46,343 INFO org.apache.hadoop.hbase.master.AssignmentManager: Clean cluster startup. Assigning user regions
2016-11-24 13:05:46,345 INFO org.apache.hadoop.hbase.master.AssignmentManager: Joined the cluster in 16ms, failover=false
2016-11-24 13:05:46,359 INFO org.apache.hadoop.hbase.master.TableNamespaceManager: Namespace table not found. Creating...
3）hbase hbck检测发现集群处于inconsistent 状态。
2 inconsistencies detected.
Status: INCONSISTENT
2 启动失败原因分析
1) 主机名/DNS没有解析成功，导致节点间通信异常
2）把主机名/DNS和IP对应关系加到/etc/hosts文件中，重启hmaster和regionserver即可解决。