hbase与sqoop的集成

hbase与sqoop的集成

1、sqoop抽取mysql表到hbase中 export HBASE_HOME=/opt/cdh-5.3.6/hbase-0.98.6-cdh5.3.6export HADOOP_HOME=/opt/cdh-5.3.6/hadoop-2.5.0-cdh5.3.6HADOOP_CLASSPATH=`${HBASE_HOME}/bin/hbase mapredcp`:${HBASE_HOME}/conf bin/sqoop import \--connect jdbc:mysql://localhost:3306/chavin \--username root -password mysql \--table emp \--columns "empno,ename" \--hbase-table imp_emp \--column-family info \--hbase-row-key empno -m 1 export HBASE_HOME=/opt/cdh-5.3.6/hbase-0.98.6-cdh5.3.6export HADOOP_HOME=/opt/cdh-5.3.6/hadoop-2.5.0-cdh5.3.6HADOOP_CLASSPATH=`${HBASE_HOME}/bin/hbase mapredcp`:${HBASE_HOME}/conf bin/sqoop import \--connect jdbc:mysql://localhost:3306/chavin \--username root -password mysql \--table emp \--columns "empno,ename" \--hbase-table imp_emp02 \--column-family info \--hbase-row-key ename -m 1 export HBASE_HOME=/opt/cdh-5.3.6/hbase-0.98.6-cdh5.3.6export HADOOP_HOME=/opt/cdh-5.3.6/hadoop-2.5.0-cdh5.3.6HADOOP_CLASSPATH=`${HBASE_HOME}/bin/hbase mapredcp`:${HBASE_HOME}/conf bin/sqoop import \--connect jdbc:mysql://localhost:3306/chavin \--username root -password mysql \--table emp \--columns "empno,ename" \--hbase-table imp_emp04 --hbase-create-table \--column-family info \--hbase-row-key empno \--hbase-bulkload \--compress -m 1 2、sqoop抽取mysql表到hive中export HIVE_HOME=/opt/cdh-5.3.6/hive-0.13.1-cdh5.3.6export HADOOP_HOME=/opt/cdh-5.3.6/hadoop-2.5.0-cdh5.3.6HADOOP_CLASSPATH=${HIVE_HOME}/conf:${HADOOP_HOME}/conf bin/sqoop import \--connect jdbc:mysql://localhost:3306/chavin \--username root -password mysql \--table emp \--columns "empno,ename" \--hive-import --hive-database chavin \--create-hive-table --hive-table h_emp01 \--hive-partition-key dt \--hive-partition-value '2017-10-09' \--compress -m 1 3、sqoop实现rdbms与hdfs数据交互 bin/sqoop import \--connect jdbc:mysql://db01:3306/chavin \--username root \--password mysql \--table emp \--target-dir /user/hadoop/sqoop/import/emp \--num-mappers 1 \--as-parquetfile bin/sqoop export \--connect jdbc:mysql://db01:3306/chavin \--username root \--password mysql \--table emp01 \--export-dir /user/hadoop/sqoop/import/emp

版权声明：本文内容由网络用户投稿，版权归原作者所有，本站不拥有其著作权，亦不承担相应法律责任。如果您发现本站中有涉嫌抄袭或描述失实的内容，请联系我们jiasou666@gmail.com 处理，核实后本网站将在24小时内删除侵权内容。