大数据学习：Hbase集群安装配置入门

作为Hadoop大数据生态的重要组件，Hbase的学习是非常重要的一块，Hbase作为Hadoop生态原生支持的数据库，基于列式存储，能够承载规模庞大的数据存储需求任务。今天的大数据学习分享，我们就主要来讲讲Hbase集群安装配置入门。


一、搭建集群环境

1、解压文件

tar -zxvf hbase-1.3.1-bin.tar.gz

2、配置环境变量

vim /etc/profile
export HBASE_HOME=/opt/hbase-1.3.1
export PATH=$PATH:$HBASE_HOME/bin
source /etc/profile

3、配置：hbase-env

vim /opt/hbase-1.3.1/conf/hbase-env.sh
export JAVA_HOME=/opt/jdk1.8
export HBASE_MANAGES_ZK=false

4、配置：hbase-site

vim /opt/hbase-1.3.1/conf/hbase-site.xml

<configuration>
    <!--HDFS存储-->
    <property>
        <name>hbase.rootdir</name>
        <value>hdfs://hop01:9000/HBase</value>
    </property>
    <!--开启集群-->
    <property>
        <name>hbase.cluster.distributed</name>
        <value>true</value>
    </property>
    <!-- 端口 -->
    <property>
        <name>hbase.master.port</name>
        <value>16000</value>
    </property>
    <!--ZK集群-->
    <property>  
        <name>hbase.zookeeper.quorum</name>
         <value>hop01,hop02,hop03</value>
    </property>
    <!--ZK数据-->
    <property>  
        <name>hbase.zookeeper.property.dataDir</name>
         <value>/data/zookeeper/data/</value>
    </property>
</configuration>

5、配置：regionservers

vim /opt/hbase-1.3.1/conf/regionservers

hop01
hop02
hop03

6、配置：软连接

软连接hadoop配置文件到HBase

ln -s /opt/hadoop2.7/etc/hadoop/core-site.xml /opt/hbase-1.3.1/conf/core-site.xml
ln -s /opt/hadoop2.7/etc/hadoop/hdfs-site.xml /opt/hbase-1.3.1/conf/hdfs-site.xml

7、同步集群服务环境

也可以手动配置集群，或者使用同步命令。

xsync hbase/

8、启动集群

在hop01节点启动即可。

/opt/hbase-1.3.1/bin/start-hbase.sh

启动日志：

hop03: starting regionserver, logging to /opt/hbase-1.3.1/bin/../logs/hbase-root-regionserver-hop03.out
hop02: starting regionserver, logging to /opt/hbase-1.3.1/bin/../logs/hbase-root-regionserver-hop02.out
hop01: starting regionserver, logging to /opt/hbase-1.3.1/bin/../logs/hbase-root-regionserver-hop01.out

9、查看状态

jps

HMaster：主节点
HRegionServer：分区节点

10、停止集群

在hop01节点停止即可。

/opt/hbase-1.3.1/bin/stop-hbase.sh

11、查看界面

http://hop01:16010

二、Hbase基础Shell命令

1、切入客户端

/opt/hbase-1.3.1/bin/hbase shell

2、查看表

hbase(main):002:0> list

3、创建表

hbase(main):003:0> create 'user','info'
0 row(s) in 2.7910 seconds
=> Hbase::Table - user

4、查看表结构

hbase(main):010:0> describe 'user'

5、添加数据

put 'user','id01','info:name','tom'
put 'user','id01','info:age','18'
put 'user','id01','info:sex','male'
put 'user','id02','info:name','jack'
put 'user','id02','info:age','20'
put 'user','id02','info:sex','female'

6、查看表数据

hbase(main):010:0> scan 'user'
ROW     COLUMN+CELL                                                                            
id01    column=info:age, timestamp=1594448524308, value=18                                     
id01    column=info:name, timestamp=1594448513534, value=tom                                   
id01    column=info:sex, timestamp=1594448530817, value=male                                   
id02    column=info:age, timestamp=1594448542631, value=20                                     
id02    column=info:name, timestamp=1594448536520, value=jack                                  
id02    column=info:sex, timestamp=1594448548005, value=female

这些表结构和数据会在集群之间自动同步。

7、查询指定列

hbase(main):012:0> get 'user','id01'
COLUMN      CELL                                                                                   
info:age    timestamp=1594448524308, value=18                                                      
info:name   timestamp=1594448513534, value=tom                                                      
info:sex    timestamp=1594448530817, value=male

8、统计行数

hbase(main):013:0> count 'user'

9、删除行数据

hbase(main):014:0> deleteall 'user','id02'

10、清空表数据

hbase(main):016:0> truncate 'user'

11、删除表

hbase(main):018:0> disable 'user'
hbase(main):019:0> drop 'user'

三、JDBC基础查询

1、核心依赖

<dependency>
    <groupId>org.apache.hbase</groupId>
    <artifactId>hbase-client</artifactId>
    <version>1.3.1</version>
</dependency>

2、基础配置

这里连接zookeeper集群地址即可。

zookeeper:
  address: 集群地址Url，逗号分隔
编写HBase配置和常用工具方法。

@Component
public class HBaseConfig {

    private static String address;
    private static final Object lock=new Object();
    public static Configuration configuration = null;
    public static ExecutorService executor = null;
    public static Connection connection = null;

    /**
     * 获取连接
     */
    public static Connection getConnection(){
        if(null == connection){
            synchronized (lock) {
                if(null == connection){
                    configuration = new Configuration();
                    configuration.set("hbase.zookeeper.quorum", address);
                    try {
                        executor = Executors.newFixedThreadPool(10);
                        connection = ConnectionFactory.createConnection(configuration, executor);
                    } catch (Exception e) {
                        e.printStackTrace();
                    }
                }
            }
        }
        return connection;
    }

    /**
     * 获取 HBaseAdmin
     */
    public static HBaseAdmin getHBaseAdmin(){
        HBaseAdmin admin = null;
        try{
            admin = (HBaseAdmin)getConnection().getAdmin();
        }catch(Exception e){
            e.printStackTrace();
        }
        return admin;
    }
    /**
     * 获取 Table
     */
    public static Table getTable(TableName tableName) {
        Table table = null ;
        try{
            table = getConnection().getTable(tableName);
        }catch(Exception e){
            e.printStackTrace();
        }
        return table ;
    }
    /**
     * 关闭资源
     */
    public static void close(HBaseAdmin admin,Table table){
        try {
            if(admin!=null) {
                admin.close();
            }
            if(table!=null) {
                table.close();
            }
        } catch (IOException e) {
            e.printStackTrace();
        }
    }

    @Value("${zookeeper.address}")
    public void setAddress (String address) {
        HBaseConfig.address = address;
    }
}

3、查询案例

查询数据参考上述全表扫描结果：

@RestController
public class HBaseController {

    /**
     * 扫描全表
     */
    @GetMapping("/scanTable")
    public String scanTable () throws Exception {
        Table table = HBaseConfig.getTable(TableName.valueOf("user"));
        try {
            ResultScanner resultScanner = table.getScanner(new Scan());
            for (Result result : resultScanner) {
                printResult(result);
            }
        } finally {
            HBaseConfig.close(null, table);
        }
        return "success";
    }

    /**
     * 根据RowKey扫描
     */
    @GetMapping("/scanRowKey")
    public void scanRowKey() throws Exception {
        String rowKey = "id02";
        Table table = HBaseConfig.getTable(TableName.valueOf("user"));
        try {
            Result result = table.get(new Get(rowKey.getBytes()));
            printResult(result);
        } finally {
            HBaseConfig.close(null, table);
        }
    }

    /**
     * 输出 Result
     */
    private void printResult (Result result){
        NavigableMap<byte[], NavigableMap<byte[], NavigableMap<Long, byte[]>>> map = result.getMap();
        Set<Map.Entry<byte[], NavigableMap<byte[], NavigableMap<Long, byte[]>>>> set = map.entrySet();
        for (Map.Entry<byte[], NavigableMap<byte[], NavigableMap<Long, byte[]>>> entry : set) {
            Set<Map.Entry<byte[], NavigableMap<Long, byte[]>>> entrySet = entry.getValue().entrySet();
            for (Map.Entry<byte[], NavigableMap<Long, byte[]>> entry2 : entrySet) {
                System.out.print(new String(result.getRow()));
                System.out.print("\t");
                System.out.print(new String(entry.getKey()));
                System.out.print(":");
                System.out.print(new String(entry2.getKey()));
                System.out.print(" value = ");
                System.out.println(new String(entry2.getValue().firstEntry().getValue()));
            }
        }
    }
}

关于大数据学习，Hbase集群安装配置入门，以上就为大家做了简单的介绍了。在现有的大数据技术生态体系当中，Hbase是必学的组件，而前期的安装配置，是入门的关键，要多多练习。成都加米谷大数据，专业大数据培训机构，大数据开发、数据分析与挖掘，零基础班本月正在招生中，课程大纲及试学视频可联系客服获取！