编译安装hadoop2.x

编译安装hadoop2.x

1、Requirements: * Unix System* JDK 1.7+* Maven 3.0 or later* Findbugs 1.3.9 (if running findbugs)* ProtocolBuffer 2.5.0* CMake 2.6 or newer (if compiling native code), must be 3.0 or newer on Mac* Zlib devel (if compiling native code)* openssl devel ( if compiling native hadoop-pipes )* Internet connection for first build (to fetch all Maven and Hadoop dependencies) 安装以上先决条件：yum groupinstall “Development tools”yum -y install cmake zlib openssl gcc zlib-devel openssl-devel make tar -zxvf protobuf-2.5.0.tar.gzcd protobuf-2.5.0./configuremake && make installprotoc --version tar  -zxvf findbugs-3.0.0.tar.gz -C /usr/local/findbugsmv /usr/local/findbugs-3.0.0 /usr/local/findbugs 设置环境变量：export FINDBUGS_HOME = /usr/local/findbugsexport PATH=$PATH:$FINDBUGS_HOME/bin 生效环境变量：source /etc/profile验证：findbugs -version 安装jdk，略。   2、安装mave： 1）解压缩maventar -zxvf apache-maven-3.3.3-bin.tar.gzmv apache-maven-3.3.3 /opt/service/maven-3.3.3chown -R hadoop:hadoop /opt/service/maven-3.3.3/ 2）配置maven环境变量：export MAVEN_HOME=/opt/service/maven-3.3.3export PATH=$MAVEN_HOME/bin:$PATH 3）本机编译环境如下：#java -versionjava version "1.7.0_67"Java(TM) SE Runtime Environment (build 1.7.0_67-b01)Java HotSpot(TM) 64-Bit Server VM (build 24.65-b04, mixed mode) # scala -versionScala code runner version 2.10.6 -- Copyright 2002-2013, LAMP/EPFL # mvn -versionApache Maven 3.3.3 (7994120775791599e205a5524ec3e0dfe41d4a06; 2015-04-22T19:57:37+08:00)Maven home: /opt/service/maven-3.3.3Java version: 1.7.0_67, vendor: Oracle CorporationJava home: /opt/service/jdk1.7.0_67/jreDefault locale: en_US, platform encoding: UTF-8OS name: "linux", version: "3.10.0-514.el7.x86_64", arch: "amd64", family: "unix" 4）修改maven仓库默认位置： vim conf/settings.xml       --添加如下内容：   /home/hadoop/.m2/repository/   5）修改maven镜像位置为阿里云的maven镜像仓库                     alimaven          central          aliyun maven                                repo1          central          Human Readable Name for this Mirror                                repo2          central          Human Readable Name for this Mirror                更改完成之后可以在命令行，执行命令验证一下：mvn help:system命令执行完成后我们可以在目录/home/hadoop/.m2/repository/下看到很多文件，这时表示maven已经配置成功了。$ ls /home/hadoop/.m2/repository/backport-util-concurrent  classworlds  com  commons-cli  commons-lang  jdom  jtidy  junit  org  plexus  xmlpull  xpp3 3、编译hadoopexport MAVEN_OPTS="-Xms256m -Xmx512m"mvn package -DskipTests -Pdist,native -Drequire.snappy 4、编译成功后的tar包存储在./hadoop-dist/target/hadoop-2.6.0.tar.gz 5、配置hadoop的native支持     用步骤6压缩包中lib/native目录下文件替换目标节点lib/native文件，然后通过以下指令检查hadoop库支持：$bin/ hadoop checknativeNative library checking:hadoop:  true /opt/cdh5/hadoop-2.6.0-cdh5.10.0/lib/native/libhadoop.sozlib:    true /lib64/libz.so.1snappy:  true /lib64/libsnappy.so.1lz4:     true revision:99bzip2:   true /lib64/libbz2.so.1openssl: true /lib64/libcrypto.so     至此，hadoop完全分布式安装成功。

版权声明：本文内容由网络用户投稿，版权归原作者所有，本站不拥有其著作权，亦不承担相应法律责任。如果您发现本站中有涉嫌抄袭或描述失实的内容，请联系我们jiasou666@gmail.com 处理，核实后本网站将在24小时内删除侵权内容。