sqoop详细教程.docxVIP

Apache Sqoop是用来实现结构型数据（如关系数据库）和Hadoop之间进行数据迁移的工具。它充分利用了MapReduce的并行特点以批处理的方式加快数据的传输，同时也借助MapReduce实现了容错。项目地址：?/ \t /articles/_blank/目前为止，已经演化出了2个版本：sqoop1和sqoop2。sqoop1的最新版本是1.4.5，sqoop2的最新版本是1.99.3；1.99.3和1.4.5是不兼容的，并且功能尚未开发完成，还不适合在生产环境部署。sqoop支持的数据库：Databaseversion--direct?support?connect string matchesHSQLDB1.8.0+Nohttp://jdbchsqldb*/ \t /articles/_blankjdbc:hsqldb:*//MySQL5.0+Yesjdbc:mysql://Oracle10.2.0+Nohttp://jdbcoracle*/ \t /articles/_blankjdbc:oracle:*//PostgreSQL8.3+Yes (import only)jdbc:postgresql:/guojian@localtest:~/work$ sudo apt-get install sqoopguojian@localtest:~/work$ sqoop helpusage: sqoop COMMAND [ARGS]Available commands:codegen Generate code to interact with database recordscreate-hive-table Import a table definition into Hiveeval Evaluate a SQL statement and display the resultsexport Export an HDFS directory to a database tablehelpList available commandsimport Import a table from a database to HDFSimport-all-tables Import tables from a database to HDFSjob Work with saved jobslist-databasesList available databases on a serverlist-tablesList available tables in a databasemerge Merge results of incremental importsmetastore Run a standalone Sqoop metastoreversion Display version informationSee sqoop help COMMAND for information on a specific command.import是将关系数据库迁移到HDFS上guojian@localtest:~/work$ sqoop import --connectjdbc:mysql://76/hivemeta2db?--username root -password passwd --table sdsguojian@localtest:~/work$ hadoop fs -ls /user/guojian/sdsFound 5 items-rw-r--r-- 3 guojian cug_test 0 2014-09-11 16:04 /user/guojian/sds/_SUCCESS-rw-r--r-- 3 guojian cug_test 483 2014-09-11 16:03 /user/guojian/sds/part-m-00000.snappy-rw-r--r-- 3 guojian cug_test 504 2014-09-11 16:04 /user/guojian/sds/part-m-00001.snappy-rw-r--r-- 3 guojian cug_test 1001 2014-09-11 16:03 /user/guojian/sds/part-m-00002.snappy-rw-r--r-- 3 guojian cug_test 952 2014-09-11 16:03 /us