使用MySQL作为Hive的Metastore

黎明lm

使用MySQL作为Hive的Metastore


前提是成功安装了HIVE和MYSQL

在hive-site.xml中添加如下内容，指定METASTORE的地址以及连接方式
<property>     <name>javax.jdo.option.ConnectionURL</name>     <value>jdbc:mysql://10.20.151.10:3306/hive?characterEncoding=UTF-8</value>     <description>JDBC connect string for a JDBC metastore</description>  </property>  <property>     <name>javax.jdo.option.ConnectionDriverName</name>     <value>com.mysql.jdbc.Driver</value>     <description>Driver class name for a JDBC metastore</description>  </property>  <property>     <name>javax.jdo.option.ConnectionUserName</name>     <value>hive_user</value>     <description>username to use against metastore database</description>  </property>  <property>     <name>javax.jdo.option.ConnectionPassword</name>     <value>123</value>     <description>password to use against metastore database</description>  </property>

然后登陆到HIVE客户端，创建一个表试试

[gpadmin1@hadoop5 hive-0.6.0]$ bin/hiveHive history file=/tmp/gpadmin1/hive_job_log_gpadmin1_201106081130_1156785421.txthive> show tables;FAILED: Error in metadata: javax.jdo.JDOFatalDataStoreException: Unknown database 'hive'NestedThrowables:com.mysql.jdbc.exceptions.jdbc4.MySQLSyntaxErrorException: Unknown database 'hive'FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask


报错了，提示很明显，识别不到名称为hive的database，难道要自己创建？试试


[Intranet root@www.linuxidc.com /var/lib/mysql]#mysql -u root -pEnter password: Welcome to the MySQL monitor.  Commands end with ; or \g.Your MySQL connection id is 41Server version: 5.5.12 MySQL Community Server (GPL)Copyright (c) 2000, 2010, Oracle and/or its affiliates. All rights reserved.Oracle is a registered trademark of Oracle Corporation and/or itsaffiliates. Other names may be trademarks of their respectiveowners.Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.mysql> show databases;+--------------------+| Database           |+--------------------+| information_schema || mysql              || performance_schema || test               |+--------------------+4 rows in set (0.00 sec)mysql> create database hive;Query OK, 1 row affected (0.00 sec)mysql> show databases;+--------------------+| Database           |+--------------------+| information_schema || hive               || mysql              || performance_schema || test               |+--------------------+5 rows in set (0.00 sec)

再登陆到HIVE里看看

[gpadmin1@hadoop5 hive-0.6.0]$ bin/hiveHive history file=/tmp/gpadmin1/hive_job_log_gpadmin1_201106081130_544334815.txthive> show table;                       FAILED: Parse Error: line 0:-1 mismatched input '<EOF>' expecting EXTENDED in show statementhive> show tables;OKTime taken: 5.173 secondshive>         CREATE TABLE u_tmp1 (id1 INT,    >            id2 int    >            )    >          ROW FORMAT DELIMITED    >          FIELDS TERMINATED BY ',';OKTime taken: 0.266 secondshive> show tables;                         OKu_tmp1Time taken: 0.197 secondshive>
注：

修改数据库的binlog
Java代码  
set global binlog_format='MIXED';  

READ-COMMITTED需要把bin-log以mixed方式来记录
否则进入hive，会如下错误
FAILED: Error in metadata: javax.jdo.JDOException: Couldnt obtain a new sequence (unique id) : Binary logging not possible. Message: Transaction level 'READ-COMMITTED' in InnoDB is not safe for binlog mode 'STATEMENT'