hadoop2.x编译安装

hadoop2.x编译安装

Build instructions for Hadoop (Hadoop 编译安装,参考hadoop源码包中BUILDING.txt文档) ----------------------------------------------------------------------------------Requirements（编译需求，需要满足以下条件才能编译成功）: * Unix System* JDK 1.6+* Maven 3.0 or later* Findbugs 1.3.9 (if running findbugs)* ProtocolBuffer 2.5.0* CMake 2.6 or newer (if compiling native code)* Zlib devel (if compiling native code)* openssl devel ( if compiling native hadoop-pipes )* Internet connection for first build (to fetch all Maven and Hadoop dependencies) 安装以上依赖条件： yum groupinstall “Development tools”yum -y install cmake zlib openssl gcc zlib-devel openssl-devel make tar -zxvf protobuf-2.5.0.tar.gzmake && make installprotoc --version 安装maven（apache-maven-3.0.5-bin.zip）解压、配置环境变量-略[root@chavin ~]# mvn -versionApache Maven 3.0.5 (r01de14724cdef164cd33c7c8c2fe155faf9602da; 2013-02-19 21:51:28+0800)Maven home: /usr/local/maven305Java version: 1.7.0_67, vendor: Oracle CorporationJava home: /usr/java/jdk1.7.0_67/jreDefault locale: en_US, platform encoding: UTF-8OS name: "linux", version: "2.6.32-431.el6.x86_64", arch: "amd64", family: "unix"配置软件库：[root@chavin ~]# ll /home/hadoop/.m2/total 16drwxrwxr-x. 56 hadoop hadoop  4096 Mar  3 20:13 repository-rw-r--r--.  1 hadoop hadoop 10224 Mar  3 20:13 settings.xml其中repository通过tar包repository.tar.gz解压得到。 编译前安装findbugs 下载：findbugs-3.0.0.tar.gz 解压:tar  -zxvf findbugs-3.0.0.tar.gz 设置环境变量：export FINDBUGS_HOME = /usr/local/findbugsexport PATH=$PATH:$FINDBUGS_HOME/bin 执行命令：source /etc/profile或者 . /etc/profile 验证：findbugs -version ----------------------------------------------------------------------------------编译选项：Build options:   * Use -Pnative to compile/bundle native code  * Use -Pdocs to generate & bundle the documentation in the distribution (using -Pdist)  * Use -Psrc to create a project source TAR.GZ  * Use -Dtar to create a TAR with the distribution (using -Pdist) Snappy build options:    Snappy is a compression library that can be utilized by the native code.   It is currently an optional component, meaning that Hadoop can be built with   or without this dependency.   * Use -Drequire.snappy to fail the build if libsnappy.so is not found.    If this option is not specified and the snappy library is missing,    we silently build a version of libhadoop.so that cannot make use of snappy.    This option is recommended if you plan on making use of snappy and want    to get more repeatable builds.   * Use -Dsnappy.prefix to specify a nonstandard location for the libsnappy    header files and library files. You do not need this option if you have    installed snappy using a package manager.  * Use -Dsnappy.lib to specify a nonstandard location for the libsnappy library    files.  Similarly to snappy.prefix, you do not need this option if you have    installed snappy using a package manager.  * Use -Dbundle.snappy to copy the contents of the snappy.lib directory into    the final tar file. This option requires that -Dsnappy.lib is also given,    and it ignores the -Dsnappy.prefix option. OpenSSL build options:    OpenSSL includes a crypto library that can be utilized by the native code.   It is currently an optional component, meaning that Hadoop can be built with   or without this dependency.   * Use -Drequire.openssl to fail the build if libcrypto.so is not found.    If this option is not specified and the openssl library is missing,    we silently build a version of libhadoop.so that cannot make use of    openssl. This option is recommended if you plan on making use of openssl     and want to get more repeatable builds.  * Use -Dopenssl.prefix to specify a nonstandard location for the libcrypto    header files and library files. You do not need this option if you have    installed openssl using a package manager.  * Use -Dopenssl.lib to specify a nonstandard location for the libcrypto library    files. Similarly to openssl.prefix, you do not need this option if you have    installed openssl using a package manager.  * Use -Dbundle.openssl to copy the contents of the openssl.lib directory into    the final tar file. This option requires that -Dopenssl.lib is also given,    and it ignores the -Dopenssl.prefix option.    Tests options:   * Use -DskipTests to skip tests when running the following Maven goals:    'package',  'install', 'deploy' or 'verify'  * -Dtest=,,....  * -Dtest.exclude=  * -Dtest.exclude.pattern=**/.java,**/.java-------------------------------------------------------------------------------------编译方法：Building distributions: Create binary distribution without native code and without documentation:   $ mvn package -Pdist -DskipTests -Dtar Create binary distribution with native code and with documentation:   $ mvn package -Pdist,native,docs -DskipTests -Dtar Create source distribution:   $ mvn package -Psrc -DskipTests Create source and binary distributions with native code and documentation:   $ mvn package -Pdist,native,docs,src -DskipTests -Dtar Create a local staging version of the website (in /tmp/hadoop-site)   $ mvn clean site; mvn site:stage -DstagingDirectory=/tmp/hadoop-site ---------------------------------------------------------------------------------- Handling out of memory errors in builds（编译过程中可能遇到的问题） ---------------------------------------------------------------------------------- If the build process fails with an out of memory error, you should be able to fixit by increasing the memory used by maven -which can be done via the environmentvariable MAVEN_OPTS. Here is an example setting to allocate between 256 and 512 MB of heap space toMaven export MAVEN_OPTS="-Xms256m -Xmx512m" ---------------------------------------------------------------------------------- 编译命令：mvn package -Pdist,native -DskipTests -Dtar mvn package -DskipTests -Dtar -Dsnappy.lib=/usr/lib64/ -Dbundle.snappy

版权声明：本文内容由网络用户投稿，版权归原作者所有，本站不拥有其著作权，亦不承担相应法律责任。如果您发现本站中有涉嫌抄袭或描述失实的内容，请联系我们jiasou666@gmail.com 处理，核实后本网站将在24小时内删除侵权内容。