elk日志收集

elk日志收集

准备环境防火墙和selinux： 关闭主机名     elk-node1   elk-node2主机名解析192.168.227.128 elk-node1192.168.227.129 elk-node2master-slave模式：master收集到日志后，会把一部分数据碎片到slave上（随机的一部分数据）；同时，master和slave又都会各自做副本，并把副本放到对方机器上，这样就保证了数据不会丢失。如果master宕机了，那么客户端在日志采集配置中将elasticsearch主机指向改为slave，就可以保证ELK日志的正常采集和web展示。

ELasticsearch安装1、下载并安装GPG Key[root@elk-node1 ~]# rpm --import ~]# cd /etc/yum.repos.d[root@elk-node1  yum.repos.d]# vim elasticsearch.repo[elasticsearch-2.x]name=Elasticsearch repository for 2.x packagesbaseurl=~]# yum -y install elasticsearch4、安装java环境[root@elk-node1 ~]# yum -y install java[root@elk-node1 ~]# java -versionopenjdk version "1.8.0_102"OpenJDK Runtime Environment (build 1.8.0_102-b14)OpenJDK 64-Bit Server VM (build 25.102-b14, mixed mode)

elk-node1配置：1、修改配置文件[root@elk-node1 ~]#  mkdir -p /data/es-data[root@elk-node1 ~]#  vim /etc/elasticsearch/elasticsearch.ymlcluster.name: wingcluster         #组名 （同一个组，组名必须一致（自己定义））node.name: elk-node1             #节点名称，建议和主机名一致path.data:  /data/es-data                            #数据存放位置path.logs:   /var/log/elasticsearch/  #日志存放位置bootstrap.mlockall:  true           #锁住内存，不被使用到交换分区去network.host: 0.0.0.0              #网络设置（0.0.0.0表示监听所有网卡）9200                       #端口2、启动并查看[root@elk-node1 ~]#  chown elasticsearch.elasticsearch  /data/[root@elk-node1 ~]#  systemctl  start  elasticsearch[root@elk-node1 ~]#  systemctl  status elasticsearch CGroup: /system.slice/elasticsearch.service           └─3005 /bin/java -Xms256m -Xmx1g -Djava.awt.headless=true -XX:+UseParNewGC -XX:+UseConcMarkSweepGC -XX:CMSI...注意：上面可以看出elasticsearch设置的内存最小256m，最大1g[root@elk-node1 ~]#  netstat  -antlp  |egrep  “9200|9300”tcp6       0      0 :::9200                 :::*                    LISTEN      3005/java           tcp6       0      0 :::9300                 :::*                    LISTEN      3005/java通过web访问测试curl -i -XGET '-d '{"query":{"match_all":{}}}'HTTP/1.1 200 OKContent-Type: application/json; charset=UTF-8Content-Length: 95{"count" : 0,"_shards" : {"total" : 0,"successful" : 0,"failed" : 0}}elk-node2配置操作同elk-node1(配置文件稍微不同)[root@elk-node2 ~]#  vim /etc/elasticsearch/elasticsearch.ymlcluster.name: wingcluster        node.name: elk-node2path.data: /data/es-data path.logs: /var/log/elasticsearch/ bootstrap.mlockall: true network.host: 0.0.0.0  9200        discovery.zen.ping.multicast.enabled: falsediscovery.zen.ping.unicast.hosts: ["172.16.113.155", "172.16.113.156"]

安装插件例如安装head插件a)插件安装方法一[root@elk-node1 ~]#  /usr/share/elasticsearch/bin/plugin install mobz/elasticsearch-head[root@elk-node1 ~]#  chown -R elasticsearch:elasticsearch /usr/share/elasticsearch/plugins[root@elk-node1 ~]#  systemctl restart elasticsearchb)插件安装方法二在/usr/share/elasticsearch/plugins目录下创建head目录下载head插件到/usr/local/src/目录下（下载地址Key[root@elk-node1 ~]# rpm --import ~]# cd /etc/yum.repos.d/[root@elk-node1  yum.repos.d]# vim logstash.repo[logstash-2.1]name=Logstash repository for 2.1.x packagesbaseurl=~]#  yum -y install logstash[root@elk-node1 ~]#  systemctl  restart elasticsearch测试1）基本的输入输出[root@elk-node1 ~]# /opt/logstash/bin/logstash -e 'input { stdin{} } output { stdout{} }'Settings: Default filter workers: 1Logstash startup completedhello                                                                                     #输入这个2016-11-11T06:41:07.690Z elk-node1 hello                        #输出这个wangshibo                                                                            #输入这个2016-11-11T06:41:10.608Z elk-node1 wangshibo               #输出这个

[root@elk-node1 yum.repos.d]# /opt/logstash/bin/logstash -e 'input { stdin{} } output { stdout{} }'Settings: Default filter workers: 2Logstash startup completedwing2018-07-09T07:13:50.851Z elk-node1 wing你的标准输入是什么，就打印它到标准输出2018-07-09T07:14:16.819Z elk-node1 你的标准输入是什么，就打印它到标准输出

kibana安装

kibana安装配置1）、kibana的安装：1）kibana的安装：[root@elk-node1 ~]# cd /usr/local/src[root@elk-node1 src]# wget src]# tar zxf kibana-4.3.1-linux-x64.tar.gz[root@elk-node1 src]# mv kibana-4.3.1-linux-x64 /usr/local/[root@elk-node1 src]# ln -s /usr/local/kibana-4.3.1-linux-x64/ /usr/local/kibana2）修改配置文件：[root@elk-node1 config]# pwd/usr/local/kibana/config[root@elk-node1 config]# cp kibana.yml kibana.yml.bak[root@elk-node1 config]# vim kibana.yml server.port: 5601server.host: "0.0.0.0"elasticsearch.url: "".kibana"因为它一直运行在前台，所以我们要么选择重开一个窗口，要么选择使用screen安装并使用screen[root@elk-node1 ~]# yum -y install screen[root@elk-node1 ~]# screen                          ＃这样就另开启了一个终端窗口[root@elk-node1 ~]# /usr/local/kibana/bin/kibana  log   [18:23:19.867] [info][status][plugin:kibana] Status changed from uninitialized to green - Ready   log   [18:23:19.911] [info][status][plugin:elasticsearch] Status  changed from uninitialized to yellow - Waiting for Elasticsearch  log   [18:23:19.941] [info][status][plugin:kbn_vislib_vis_types] Status changed from uninitialized to green - Ready  log   [18:23:19.953] [info][status][plugin:markdown_vis] Status changed from uninitialized to green - Ready  log   [18:23:19.963] [info][status][plugin:metric_vis] Status changed from uninitialized to green - Ready  log   [18:23:19.995] [info][status][plugin:spyModes] Status changed from uninitialized to green - Ready  log   [18:23:20.004] [info][status][plugin:statusPage] Status changed from uninitialized to green - Ready  log   [18:23:20.010] [info][status][plugin:table_vis] Status changed from uninitialized to green - Ready

然后按ctrl+a+d组合键，暂时断开screen会话这样在上面另启的screen屏里启动的kibana服务就一直运行在前台了....[root@elk-node1 ~]# screen -lsThere is a screen on:        15041.pts-0.elk-node1   (Detached)1 Socket in /var/run/screen/S-root.

注：screen重新连接会话    下例显示当前有两个处于detached状态的screen会话，你可以使用screen -r 重新连接上：    [root@tivf18 root]# screen –ls    There are screens on:            8736.pts-1.tivf18       (Detached)            8462.pts-0.tivf18       (Detached)    2 Sockets in /root/.screen.         [root@tivf18 root]# screen -r 87363)、访问kibana测试    http://192.168.227.128:5601

版权声明：本文内容由网络用户投稿，版权归原作者所有，本站不拥有其著作权，亦不承担相应法律责任。如果您发现本站中有涉嫌抄袭或描述失实的内容，请联系我们jiasou666@gmail.com 处理，核实后本网站将在24小时内删除侵权内容。