Mac hadoop 运行权威指南天气例子

Mac hadoop 运行权威指南天气例子

1.修改环境变量 ~/bash_profile

export ANDROID_HOME=~/Library/Android/sdk # added by Anaconda3 5.1.0 installer export PATH="/Users/walle/anaconda3/bin:$PATH" # hadoop export HADOOP_HOME=/usr/local/Cellar/hadoop/3.1.0/libexec export PATH=$PATH:$HADOOP_HOME/bin:$HADOOP_HOME/sbin for f in $HADOOP_HOME/share/hadoop/common/hadoop-*.jar;do export CLASSPATH=$CLASSPATH:$f done for f in $HADOOP_HOME/share/hadoop/hdfs/hadoop-*.jar;do export CLASSPATH=$CLASSPATH:$f done for f in $HADOOP_HOME/share/hadoop/mapreduce/hadoop-*.jar;do export CLASSPATH=$CLASSPATH:$f done for f in $HADOOP_HOME/share/hadoop/yarn/hadoop-*.jar;do export CLASSPATH=$CLASSPATH:$f done export CLASSPATH=$CLASSPATH:$HADOOP_HOME/share/common/lib:$HADOOP_HOME/share/hdfs/lib:$HADOOP_HOME/share/mapreduce/lib:$HADOOP_HOME/share/tools/lib:$HADOOP_HOME/share/yarn/lib

2.编译java 为jar文件

javac *.java jar -cvf MaxTemperature.jar .

在src目录执行上面2行代码，得到MaxTemperature.jar

3. 拷贝文件到hfs

#创建几个目录 hadoop fs -mkdir /user/ hadoop fs -mkdir /user/walle #拷贝文件 hadoop fs -put ./input/1901 /user/walle hadoop fs -put ./input/1902 /user/walle

4.执行mapreduce

hadoop jar ./src/MaxTemperature.jar MaxTemperature /user/walle /user/walle/output

5.查看结果

hadoop fs -cat /user/walle/output/part-r-00000

版权声明：本文内容由网络用户投稿，版权归原作者所有，本站不拥有其著作权，亦不承担相应法律责任。如果您发现本站中有涉嫌抄袭或描述失实的内容，请联系我们jiasou666@gmail.com 处理，核实后本网站将在24小时内删除侵权内容。