Ambari部署文档

Ambari部署文档

一、配置说明

硬件环境

1. 硬件环境

2. 软件环境

二、搭建环境准备

1.

搭建虚拟机

·       搭建yum.hdp虚拟机，然后完成克隆，虚拟机搭建完毕。

3.1 搭建虚拟机

·      打开VMwear选择新建虚拟机

选择自定义安装

虚拟机兼容性选择

选择稍后安装操作系统

操作系统的选择

虚拟机位置与命名

处理器与内存的分配

网络连接类型的选择（NAT）

后两项按虚拟机默认即可

创建磁盘，选择容量

磁盘名称，默认即可

取消不需要的硬件，自定义硬件

选择不需要的硬件声卡、打印机等然后移除

在“我的计算机”下，新建文件夹Ambari管理集群服务器

安装CentOS，右击ambarihost，选择设置

开启虚拟机，安装操作系统，选择第一项，安装直接CentOS 7，回车

设置时间，时区选择上海

选择安装位置

开始安装

等待系统安装完毕，重启系统

登录，安装完毕

3.2 配置虚拟机

3.2.1 配置静态IP

打开“编辑-虚拟网络编辑器-更改设置”

在虚拟机中打开文件：vim /etc/sysconfig/network-scripts/ifcfg-ens33

修改文件如下：

重启网关：servicenetwork start

配置DNS

3.2.2 关闭防火墙和selinux

1. 防火墙设置

• # 查看防火墙状态• systemctl status firewalld• # 查看开机是否启动防火墙服务• systemctl is-enabled firewalld• # 关闭防火墙• systemctl stop firewalld• systemctl disable firewalld• # 再次查看防火墙状态和开机防火墙是否启动• systemctl status firewalld• systemctl is-enabled firewalld

2. 禁用selinux

• # 永久性关闭selinux（重启服务器生效）• sed -i 's/SELINUX=enforcing/SELINUX =disabled/' /etc/selinux/config• # 临时关闭selinux（立即生效，重启服务器失效）• setenforce 0• # 查看selinux状态• getenforce• # disabled为永久关闭，permissive为临时关闭，enforcing为开启

3.2.3修改主机名和hosts文件

• vim /etc/hostname # 该命令需要重启后生效• # 还有另外一种方式，执行该命令后立即生效，只不过需要重启Xshell连接• hostnamectl set-hostname node1.ambari.com

修改hosts文件

• vim /etc/hosts

3.2.5 安装时间同步服务（ntp）

​

• 安装：yum install -y ntp

启动并查看状态：systemctl start ntpd.servicesystemctl status ntpd.service

设置开机自启：systemctl enable ntpd.service

3.2.6 安装并配置JDK

下载链接: ​​j47n ；内有​​jdk-8u151-linux-x64.tar.gz​​和​​mysql-connector-java.jar​​文件。

mkdir /usr/java；将下载的压缩包上传到java文件夹内

解压压缩包：tar zxvf jdk-8u151-linux-x64.tar.gz

配置jdk环境变量：

​

• # 编辑/etc/profile,文末插入以下内容：• # set java• export JAVA_HOME=/usr/java/jdk1.8.0_151• export PATH=$JAVA_HOME/bin:$PATH• 使环境变量生效：source /etc/profile• 安装验证：java -version

3.3 克隆虚拟机

克隆虚拟机之前，保证当前虚拟机处于关闭状态，并修改镜像。

3.3.1 开始克隆

虚拟机-管理-克隆

下一步

虚拟机中的当前状态

创建完整克隆

修改虚拟机名称和存储位置，完成，关闭

按照相同步骤，克隆多台虚拟机

修改克隆出来的所有虚拟机mac地址

3.3.2 修改参数

启动克隆的虚拟机，并修改虚拟机的IP地址

​

• vim /etc/sysconfig/network-scripts/ ifcfg-eno16777736

重启network：service network start

3.3.3 在ambari虚拟机上安装mysql

mysql5.7 centos7:

​​检查本地资源库中是否有mysql的rpm包

• rpm -qa | grep mysql• # 删除相关rpm包• rpm -ev --nodeps

2. 搭建mysql5.7的yum源

• # 下载mysql5.7的rpm包• wget # 安装第一步下载的rpm文件，安装成功后/etc/yum.repos.d/目录下会增加两个文件• yum -y install mysql57-community-release-el7-11.noarch.rpm• # 查看mysql57的安装源是否可用，如不可用请自行修改配置文件（/etc/yum.repos.d/mysql-community.repo）使mysql57下面的enable=1• # 若有mysql其它版本的安装源可用，也请自行修改配置文件使其enable=0• yum repolist enabled | grep mysql

3. 安装mysql

• # 启动mysql服务• service mysqld start• # 查看root密码• grep "password" /var/log/mysqld.log• # 登陆mysql• mysql -u root -p• Enter password:• # 为了可以设置简单密码• set global validate_password_policy=0;• set global validate_password_length=4;• # 立即修改密码，执行其他操作报错：• SET PASSWORD FOR 'root'@'localhost' = PASSWORD('newpass');• # 我们创建密码为root123

5. 新增ambari用户并增加权限

• mysql -uroot -proot123• CREATE USER 'ambari'@'%' IDENTIFIED BY 'Aa-123456';• GRANT ALL PRIVILEGES ON *.* TO 'ambari'@'%';• CREATE USER 'ambari'@'localhost' IDENTIFIED BY 'Aa-123456';• GRANT ALL PRIVILEGES ON *.* TO 'ambari'@'localhost';• CREATE USER 'ambari'@'fct.hdp01' IDENTIFIED BY ' Aa-123456';• GRANT ALL PRIVILEGES ON *.* TO 'ambari'@' fct.hdp01'; //本地主机名• FLUSH PRIVILEGES;

6. 使用ambari用户登陆并创建数据库

• mysql -uambari -pambari• CREATE DATABASE ambari;• exit;

3.3.5 免密登录

各个主机均执行以下操作

• ## 生成密钥对• ssh-keygen -t rsa ## 一路回车即可• ## 进入.ssh目录，如果目录不存在则创建• cd ~/.ssh• ## 将公钥导入至authorized_keys• cat id_rsa.pub >> authorized_keys• ## 修改文件权限• chmod 700 ~/.ssh• chmod 600 authorized_keys

在ambari上执行

• ## 配置主从互相免密登陆• [root@node1 ~]# cat ~/.ssh/id_rsa.pub | ssh root@node2.ambari.com 'cat - >> ~/.ssh/authorized_keys'• [root@node1 ~]# cat ~/.ssh/id_rsa.pub | ssh root@node3.ambari.com 'cat - >> ~/.ssh/authorized_keys'• ssh node2.ambari.com • ssh node3.ambari.com # 验证主机点是否可以免密登陆从节点，执行exit命令退出即可。

在主机上安装

Ambari

4.1 安装yum相关工具

[root@yum ~]# yum install yum-utils -y[root@yum ~]# yum repolist[root@yum ~]# yum install createrepo -y

4.2 安装Apache ~]# yum install -y

安装完成后，会生成 /var/~]# mkdir /var/~]# mkdir /var/~]# mkdir /var/~]# tar -zxvf ambari-2.7.3.0-centos7.tar.gz -C /var/~]# tar -zxvf HDP-3.1.0.0-centos7-rpm.tar.gz -C /var/~]# tar -zxvf HDP-UTILS-1.1.0.22-centos7.tar.gz -C /var/systemctl start # 启动~]# systemctl status # 查看~]# systemctl enable # 设置配置本地 Repo

4.3.1 配置Ambari

• 下载• 安装wegt：yum -y install wget• wget -O /etc/yum.repos.d/ambari.repo /etc/yum.repos.d/ambari.repo

4.3.1 配置HDP和HDP-UTILS

​

创建配置文件：[root@yum yum.repos.d]# touch /etc/yum.repos.d/HDP.repo

添加内容#VERSION_NUMBER=3.1.0.0-78[HDP-3.1.0.0]name=HDP Version - HDP-3.1.0.0baseurl=Version - HDP-UTILS-1.1.0.22baseurl=分发Ambari.repo和HDP.repo

把ambari.repo HDP.repo分发到各个节点的相同目录下

[root@yum yum.repos.d]# scp ambari.repo HDP.repo nd-00.hdp:$PWD[root@yum yum.repos.d]# scp ambari.repo HDP.repo nd-01.hdp:$PWD[root@yum yum.repos.d]# scp ambari.repo HDP.repo nd-02.hdp:$PWD

4.3.3 生成本地源

使用createrepo命令，创建yum本地源（软件仓库），即为存放本地特定位置的众多rpm包建立索引，描述各包所需依赖信息，并形成元数据

​

[root@yum ~]# createrepo /var/worker 0 with 51 pkgsSpawning worker 1 with 50 pkgsSpawning worker 2 with 50 pkgsSpawning worker 3 with 50 pkgsWorkers FinishedSaving Primary metadataSaving file lists metadataSaving other metadataGenerating sqlite DBsSqlite DBs complete[root@yum ~]# createrepo /var/worker 0 with 4 pkgsSpawning worker 1 with 4 pkgsSpawning worker 2 with 4 pkgsSpawning worker 3 with 4 pkgsWorkers FinishedSaving Primary metadataSaving file lists metadataSaving other metadataGenerating sqlite DBsSqlite DBs complete

5.

安装

Ambari-Server

5.1  节点安装

安装ambari-server

[root@nd-00 ~]# yum install ambari-server[root@nd-00 ~]# ambari-server setupUsing python /usr/bin/pythonSetup ambari-serverChecking SELinux...SELinux status is 'disabled'Customize user account for ambari-server daemon [y/n] (n)? yEnter user account for ambari-server daemon (root):root # 用户Adjusting ambari-server permissions and ownership...Checking firewall status...Checking JDK...[1] Oracle JDK 1.8 + Java Cryptography Extension (JCE) Policy Files 8[2] Custom JDK==============================================================================Enter choice (1): 2 # 选择自定义jdkWARNING: JDK must be installed on all hosts and JAVA_HOME must be valid on all hosts.WARNING: JCE Policy files are required for configuring Kerberos security. If you plan to use Kerberos,please make sure JCE Unlimited Strength Jurisdiction Policy Files are valid on all hosts.Path to JAVA_HOME: /usr/java/jdk1.8.0_77 # jdk安装路径Validating JDK on Ambari Server...done.Check JDK version for Ambari Server...JDK version found: 8Minimum JDK version is 8 for Ambari. Skipping to setup different JDK for Ambari Server.Checking GPL software agreement...GPL License for LZO: Ambari Server to download and install GPL Licensed LZO packages [y/n] (n)? yCompleting setup...Configuring database...Enter advanced database configuration [y/n] (n)? yConfiguring database...==============================================================================Choose one of the following options:[1] - PostgreSQL (Embedded)[2] - Oracle[3] - MySQL / MariaDB[4] - PostgreSQL[5] - Microsoft SQL Server (Tech Preview)[6] - SQL Anywhere[7] - BDB==============================================================================Enter choice (1): 3 # 选择安装的mysqlHostname (localhost): nd-00 # 配置hostnamePort (3306): # 默认Database name (ambari): Username (ambari): Enter Database Password (bigdata): # 密码不显示Re-enter password: Configuring ambari database...Should ambari use existing default jdbc /usr/share/java/mysql-connector-java.jar [y/n] (y)? yConfiguring remote database connection properties...WARNING: Before starting Ambari Server, you must run the following DDL directly from the database shell to create the schema: /var/lib/ambari-server/resources/Ambari-DDL-MySQL-CREATE.sql # 此处需注意，启动ambari之前需要执行此句Proceed with configuring remote database connection properties [y/n] (y)? yExtracting system views...ambari-admin-2.7.3.0.139.jar....Ambari repo file contains latest json url updating stacks repoinfos with it...Adjusting ambari-server permissions and ownership...Ambari Server 'setup' completed successfully. # 安装成功

执行上面安装过程中给出的提示

# 使用root登录，设置允许远程登录mysql> set global validate_password_policy=0;Query OK, 0 rows affected (0.00 sec)mysql> set global validate_password_length=1;Query OK, 0 rows affected (0.00 sec)mysql> GRANT ALL PRIVILEGES ON ambari.* TO 'ambari'@'localhost' IDENTIFIED BY 'Ambari123';Query OK, 0 rows affected, 1 warning (0.03 sec)mysql> GRANT ALL PRIVILEGES ON ambari.* TO 'ambari'@'%' IDENTIFIED BY 'Ambari123';Query OK, 0 rows affected, 1 warning (0.00 sec)mysql> FLUSH PRIVILEGES;Query OK, 0 rows affected (0.00 sec)

使用ambari登录

[root@nd-00 ~]# mysql -u ambari -p -h nd-00Enter password: Welcome to the MySQL monitor. Commands end with ; or \g.Your MySQL connection id is 4Server version: 5.7.25 MySQL Community Server (GPL)Copyright (c) 2000, 2019, Oracle and/or its affiliates. All rights reserved.Oracle is a registered trademark of Oracle Corporation and/or itsaffiliates. Other names may be trademarks of their respectiveowners.Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.mysql> show databases;+--------------------+| Database |+--------------------+| information_schema || ambari |+--------------------+2 rows in set (0.03 sec)mysql> use ambari;Database changedmysql> source /var/lib/ambari-server/resources/Ambari-DDL-MySQL-CREATE.sql;

# 执行完，查看有无报错信息，并查看数据表mysql> show tables;+-------------------------------+| Tables_in_ambari |+-------------------------------+| ClusterHostMapping || QRTZ_BLOB_TRIGGERS || QRTZ_CALENDARS || QRTZ_CRON_TRIGGERS || QRTZ_FIRED_TRIGGERS || QRTZ_JOB_DETAILS || QRTZ_LOCKS || QRTZ_PAUSED_TRIGGER_GRPS || QRTZ_SCHEDULER_STATE || QRTZ_SIMPLE_TRIGGERS || QRTZ_SIMPROP_TRIGGERS || QRTZ_TRIGGERS || adminpermission || adminprincipal || adminprincipaltype || adminprivilege || adminresource || adminresourcetype || alert_current || alert_definition || alert_group || alert_group_target || alert_grouping || alert_history || alert_notice || alert_target || alert_target_states || ambari_configuration || ambari_operation_history || ambari_sequences || artifact || blueprint || blueprint_configuration || blueprint_setting || clusterconfig || clusters || clusterservices || clusterstate || confgroupclusterconfigmapping || configgroup || configgrouphostmapping || execution_command || extension || extensionlink || groups || host_role_command || host_version || hostcomponentdesiredstate || hostcomponentstate || hostconfigmapping || hostgroup || hostgroup_component || hostgroup_configuration || hosts || hoststate || kerberos_descriptor || kerberos_keytab || kerberos_keytab_principal || kerberos_principal || key_value_store || kkp_mapping_service || members || metainfo || permission_roleauthorization || remoteambaricluster || remoteambariclusterservice || repo_applicable_services || repo_definition || repo_os || repo_tags || repo_version || request || requestoperationlevel || requestresourcefilter || requestschedule || requestschedulebatchrequest || role_success_criteria || roleauthorization || servicecomponent_version || servicecomponentdesiredstate || serviceconfig || serviceconfighosts || serviceconfigmapping || servicedesiredstate || setting || stack || stage || topology_host_info || topology_host_request || topology_host_task || topology_hostgroup || topology_logical_request || topology_logical_task || topology_request || upgrade || upgrade_group || upgrade_history || upgrade_item || user_authentication || users || viewentity || viewinstance || viewinstancedata || viewinstanceproperty || viewmain || viewparameter || viewresource || viewurl || widget || widget_layout || widget_layout_user_widget |+-------------------------------+111 rows in set (0.00 sec)

5.2 启动Ambari-Server

如果启动失败，关闭服务【ambari-server stop】，重新启动

[root@nd-00 ~]# ambari-server startUsing python /usr/bin/pythonStarting ambari-serverAmbari Server running with administrator privileges.Organizing resource files at /var/lib/ambari-server/resources...Ambari database consistency check started...Server PID at: /var/run/ambari-server/ambari-server.pidServer out at: /var/log/ambari-server/ambari-server.outServer log at: /var/log/ambari-server/ambari-server.logWaiting for server start......................................................Server started listening on 8080DB configs consistency check: no errors and warnings were found.Ambari Server 'start' completed successfully.

5.3 安装Agent

所有节点安装ambari-agent

​

yum -y install ambari-agent

5.4 访问Ambari web页面

默认端口8080，Username：admin；Password：admin；​​~]# ambari-server setup --jdbc-db=mysql --jdbc-driver=/usr/share/java/mysql-connector-java.jar

等待启动完毕，不用担心警告，后期可以调整，搭建完成，可以在展示页面进行查看集群状态。

可以查看监控界面，可以看到大数据组件中出现错误，单个组件点开处理，由于本次搭建集群使用虚拟机，性能不好，可以少选择一些组件。

Ambari使用指南

中对应的地方翻译成中文即可。

（2）你可以使用ftp 工具进入将你安装好的ambari服务的/usr/lib/ambari-server/web/javascript/app.js文件下载到你本地。使用编辑工具打开，然后找到Em.I18n.translations 这个节点，然后翻译对应的英文。大概在58237行的地方，然后将文件替换回去，重启ambari

【问题描述】

IP地址从172.16.***.***转到了192.168.*.***。Hortonworks Sandbox 正常启动，进入Ambari浏览器页面(192.168.*.***:8080)，会出现如下问题:

黄色问号表示:HeartbeatLost

即所有的服务都心跳丢失了。。重启ambari-server和ambari-agent都不能解决。。

【问题解决】

本来没想到是IP的问题，以为ip换了，Sandbox会自动更新状态。第一次遇到该问题的时候，网上也没有找到有用的信息，因此选择了重装Sandbox。。直到第二次发生这个情况，才意识到可能是IP改变导致的。按此思路，也在网上找到了一点内容。

就是这样：

1.停止ambari-server和ambari-agent服务

2.修改/etc/hosts文件(打开文件的一刹那就直到这个方法有戏), 把host地址改成192.168.。。。。那个地址

3.启动ambari-server 和 ambari-agent

4.大功告成。进入页面会看到正常了。

Yum  install  xxx   (不加Y 会保存一下目录)

/usr/share/java

在后续安装 hive 时，测试连接存储 hive 元数据的数据库时，始终连不上。停掉 ambari-server 后，运行上面的命令后，才能连接成功。

ambari-server setup --jdbc-db=mysql--jdbc-driver=/usr/share/java/mysql-connector-java.jar

版权声明：本文内容由网络用户投稿，版权归原作者所有，本站不拥有其著作权，亦不承担相应法律责任。如果您发现本站中有涉嫌抄袭或描述失实的内容，请联系我们jiasou666@gmail.com 处理，核实后本网站将在24小时内删除侵权内容。