「开源摘星计划」Loki实现Harbor日志的高效管理

「开源摘星计划」Loki实现Harbor日志的高效管理

本文已参与「​​开源摘星计划​​​」，欢迎正在阅读的你加入。​​​​​​​​

前言

在Harbor的

使用Loki实现Harbor日志的管理，在Grafana页面查询Harbor的日志。

环境：

已通过Helm在Kubernetes集群中安装Harbor，Harbor服务部署在Harbor命名空间中。

Helm部署Harbor的文档请阅读：

Helm部署Harbor,实现高可用的镜像仓库（超详细分享）~后附踩坑记录

​​harbor]# kubectl -n harbor get podsNAME READY STATUS RESTARTS AGEharbor-chartmuseum-68f68d88ff-nq97s 1/1 Running 0 2m52sharbor-core-7565bfb9cc-w5k29 1/1 Running 0 2m52sharbor-database-0 1/1 Running 0 2m52sharbor-jobservice-6747b486df-vm9zs 1/1 Running 0 2m52sharbor-nginx-956fff988-r8dpx 1/1 Running 0 2m52sharbor-notary-server-776f85f9c6-4nlsn 1/1 Running 0 2m52sharbor-notary-signer-7f895d59d6-k5957 1/1 Running 0 2m52sharbor-portal-5766b784c7-sndmz 1/1 Running 0 2m52sharbor-redis-0 1/1 Running 0 2m52sharbor-registry-6b88cfb465-fcxvn 2/2 Running 0 2m52sharbor-trivy-0 1/1 Running 0 2m52s[root@master1 harbor]# kubectl -n harbor get svcNAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGEharbor NodePort 10.98.71.137 80:30002/TCP,4443:30004/TCP 3m5sharbor-chartmuseum ClusterIP 10.102.194.148 80/TCP 3m5sharbor-core ClusterIP 10.100.87.174 80/TCP 3m5sharbor-database ClusterIP 10.100.79.72 5432/TCP 3m5sharbor-jobservice ClusterIP 10.111.33.230 80/TCP 3m5sharbor-notary-server ClusterIP 10.97.144.222 4443/TCP 3m5sharbor-notary-signer ClusterIP 10.110.131.62 7899/TCP 3m5sharbor-portal ClusterIP 10.105.213.145 80/TCP 3m5sharbor-redis ClusterIP 10.110.18.210 6379/TCP 3m5sharbor-registry ClusterIP 10.110.129.16 5000/TCP,8080/TCP 3m5sharbor-trivy ClusterIP 10.109.0.155 8080/TCP 3m5s

Loki简介

Loki是 Grafana Labs 团队最新的开源项目，是一个水平可扩展，高可用性，多租户的日志聚合系统。它的设计非常经济高效且易于操作，因为它不会为日志内容编制索引，而是为每个日志流编制一组标签。

与其他日志系统不同，Loki 是围绕仅索引有关日志的元数据的想法构建的：标签（就像 Prometheus 标签一样）。然后，日志数据本身会被压缩并以块的形式存储在对象存储（例如 S3 或 GCS）中，甚至本地存储在文件系统中。小索引和高度压缩的块简化了操作并显着降低了 Loki 的成本。

使用Helm部署Loki日志平台

环境准备

1)创建命名空间

[root@lidabai-master1 ~]# kubectl create namespace loki

2）安装Helm

如果你已经安装好helm了，可以忽略本步骤。

[root@lidabai-master1 ~]# wget ~]# tar zxvf helm-v3.7.2-linux-amd64.tar.gz[root@lidabai-master1 ~]# cp linux-amd64/helm /usr/local/bin/[root@lidabai-master1 ~]# helm versioversion.BuildInfo{Version:"v3.7.2", GitCommit:"663a896f4a815053445eec4153677ddc24a0a361", GitTreeState:"clean", GoVersion:"go1.16.10"}

3）添加Chart仓库

[root@lidabai-master1 ~]# helm repo add grafana ~]# helm repo listNAME URLgrafana ~]# helm repo update

4)搜索chart

[root@master1 ~]# helm search repo grafanaNAME CHART VERSION APP VERSION DESCRIPTION grafana/grafana 6.32.1 9.0.2 The leading tool for querying and visualizing t...grafana/grafana-agent-operator 0.2.2 0.25.1 A Helm chart for Grafana Agent Operator grafana/enterprise-logs 2.2.2 v1.4.1 Grafana Enterprise Logs grafana/enterprise-logs-simple 1.2.0 v1.4.0 DEPRECATED Grafana Enterprise Logs (Simple Scal...grafana/enterprise-metrics 1.9.0 v1.7.0 DEPRECATED Grafana Enterprise Metrics grafana/fluent-bit 2.3.1 v2.1.0 Uses fluent-bit Loki go plugin for gathering lo...grafana/loki 2.12.2 v2.5.0 Loki: like Prometheus, but for logs. grafana/loki-canary 0.8.1 2.5.0 Helm chart for Grafana Loki Canary grafana/loki-distributed 0.50.0 2.5.0 Helm chart for Grafana Loki in microservices mode grafana/loki-simple-scalable 1.6.1 2.5.0 Helm chart for Grafana Loki in simple, scalable...grafana/loki-stack 2.6.5 v2.4.2 Loki: like Prometheus, but for logs. grafana/mimir-distributed 2.1.0 2.1.0 Grafana Mimir grafana/mimir-openshift-experimental 2.1.0 2.0.0 Grafana Mimir on OpenShift Experiment grafana/oncall 1.0.2 v1.0.3 Developer-friendly incident response with brill...grafana/promtail 6.0.2 2.5.0 Promtail is an agent which ships the contents o...grafana/rollout-operator 0.1.2 v0.1.1 Grafana rollout-operator grafana/tempo 0.15.4 1.4.1 Grafana Tempo Single Binary Mode grafana/tempo-distributed 0.20.3 1.4.1 Grafana Tempo in MicroService mode grafana/tempo-vulture 0.2.0 1.3.0 Grafana Tempo Vulture - A tool to monitor Tempo...

grafana/loki-stack: 将多个进程封装到一个Pod内，未做数据持久化，适用于测试环境,单体模式。 grafana/loki-canary: 金丝雀更新模式； grafana/loki-distributed:  微服务模式，适合生产较大规模场景； grafana/loki-simple-scalable: 读写分离模式，简单可扩展；

下载并安装Loki Chart

下载解压chart

[root@master1 ~]# helm pull grafana/loki-stack --untar[root@master1 ~]# cd loki-stack/[root@master1 loki-stack]# lscharts Chart.yaml README.md requirements.lock requirements.yaml templates values.yaml

--untar：下载并解压Chart包；

修改values.yaml配置

[root@master1 loki-stack]# vim values.yaml grafana: enabled: true #启用Grafana组件 sidecar: datasources: enabled: true maxLines: 1000 image: tag: 8.3.5

安装

[root@master1 loki-stack]# helm install loki-stack . -n lokiNAME: loki-stackLAST DEPLOYED: Thu Jul 7 13:31:02 2022NAMESPACE: lokiSTATUS: deployedREVISION: 1NOTES:The Loki stack has been deployed to your cluster.Loki can now be added as a datasource in Grafana.See for more detail.[root@master1 loki-stack]# helm -n loki ls

查看验证服务

[root@master1 loki-stack]# kubectl -n loki get podsNAME READY STATUS RESTARTS AGEloki-stack-0 1/1 Running 0 74sloki-stack-grafana-69966bcf94-lp4z5 2/2 Running 0 74sloki-stack-promtail-ckpsh 1/1 Running 0 74sloki-stack-promtail-g7kft 1/1 Running 0 74sloki-stack-promtail-gf86b 1/1 Running 0 74sloki-stack-promtail-jnznc 1/1 Running 0 74snfs-provisioner-56fd47fc9c-lktrv 1/1 Running 3 23h[root@master1 loki-stack]# kubectl -n loki get svcNAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGEloki-stack ClusterIP 10.102.81.250 3100/TCP 76sloki-stack-grafana ClusterIP 10.103.62.31 80/TCP 76sloki-stack-headless ClusterIP None 3100/TCP 76sloki-stack-memberlist ClusterIP None 7946/TCP 76s

Pod都部署成功！

修改Grafana服务访问方式为NodePort

[root@master1 loki-stack]# kubectl -n loki edit svc loki-stack-grafanaspec: type: NodePort

Grafana配置

登录Grafana

[root@master1 loki-stack]# kubectl -n loki get svc loki-stack-grafana NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGEloki-stack-grafana NodePort 10.103.62.31 80:32555/TCP 4m9s

查看Grafana密码

$ kubectl -n loki get secret loki-stack-grafana -o jsonpath="{.data.admin-password}" | base64 --decode ; echofPPZ19cAnP0J7MYPiU4vkWIcMDlbxlgGJYe4jKYT

浏览器登录Grafana

登录地址： k8s节点IP:NodePort用户名：admin密码： 刚才查看到的secret解析串

Grafana查看Loki日志

然后在输入： {namespace="harbor"} 即可查看Harbor命名空间的所有日志。

版权声明：本文内容由网络用户投稿，版权归原作者所有，本站不拥有其著作权，亦不承担相应法律责任。如果您发现本站中有涉嫌抄袭或描述失实的内容，请联系我们jiasou666@gmail.com 处理，核实后本网站将在24小时内删除侵权内容。