kubernetes（k8s）安装 Prometheus + Grafana

组件说明

MetricServer：是kubernetes集群资源使用情况的聚合器，收集数据给kubernetes集群内使用，如 kubectl,hpa,scheduler等。

PrometheusOperator：是一个系统监测和警报工具箱，用来存储监控数据。

NodeExporter：用于各node的关键度量指标状态数据。

KubeStateMetrics：收集kubernetes集群内资源对象数据，制定告警规则。

Prometheus：采用pull方式收集apiserver，scheduler，controller-manager，kubelet组件数据，通过http协议传输。

Grafana：是可视化数据统计和监控平台。

kubernetesk8s怎么使用,kubernetesk8s标签详解

克隆代码

root@hello:~#gitclone-brelease-0.10https://github.com/prometheus-operator/kube-prometheus.git
Cloninginto'kube-prometheus'...
remote:Enumeratingobjects:16026,done.
remote:Countingobjects:100%(2639/2639),done.
remote:Compressingobjects:100%(165/165),done.
remote:Total16026(delta2524),reused2485(delta2470),pack-reused13387
Receivingobjects:100%(16026/16026),7.81MiB|2.67MiB/s,done.
Resolvingdeltas:100%(10333/10333),done.
root@hello:~#

进入目录修改镜像地址

若访问Google畅通无阻即可无需修改，跳过即可

root@hello:~#cdkube-prometheus/manifests
root@hello:~/kube-prometheus/manifests#sed-i"s#quay.io/prometheus/#registry.cn-hangzhou.aliyuncs.com/chenby/#g"*.yaml
root@hello:~/kube-prometheus/manifests#sed-i"s#quay.io/brancz/#registry.cn-hangzhou.aliyuncs.com/chenby/#g"*.yaml
root@hello:~/kube-prometheus/manifests#sed-i"s#k8s.gcr.io/prometheus-adapter/#registry.cn-hangzhou.aliyuncs.com/chenby/#g"*.yaml
root@hello:~/kube-prometheus/manifests#sed-i"s#quay.io/prometheus-operator/#registry.cn-hangzhou.aliyuncs.com/chenby/#g"*.yaml
root@hello:~/kube-prometheus/manifests#sed-i"s#k8s.gcr.io/kube-state-metrics/#registry.cn-hangzhou.aliyuncs.com/chenby/#g"*.yaml
root@hello:~/kube-prometheus/manifests#

修改svc为NodePort

root@hello:~/kube-prometheus/manifests#sed-i"/ports:/i\type:NodePort"grafana-service.yaml
root@hello:~/kube-prometheus/manifests#sed-i"/targetPort:http/i\nodePort:31100"grafana-service.yaml
root@hello:~/kube-prometheus/manifests#catgrafana-service.yaml
apiVersion:v1
kind:Service
metadata:
labels:
app.kubernetes.io/component:grafana
app.kubernetes.io/name:grafana
app.kubernetes.io/part-of:kube-prometheus
app.kubernetes.io/version:8.3.3
name:grafana
namespace:monitoring
spec:
type:NodePort
ports:
-name:http
port:3000
nodePort:31100
targetPort:http
selector:
app.kubernetes.io/component:grafana
app.kubernetes.io/name:grafana
app.kubernetes.io/part-of:kube-prometheus
root@hello:~/kube-prometheus/manifests#sed-i"/ports:/i\type:NodePort"prometheus-service.yaml
root@hello:~/kube-prometheus/manifests#sed-i"/targetPort:web/i\nodePort:31200"prometheus-service.yaml
root@hello:~/kube-prometheus/manifests#sed-i"/targetPort:reloader-web/i\nodePort:31300"prometheus-service.yaml
root@hello:~/kube-prometheus/manifests#catprometheus-service.yaml
apiVersion:v1
kind:Service
metadata:
labels:
app.kubernetes.io/component:prometheus
app.kubernetes.io/instance:k8s
app.kubernetes.io/name:prometheus
app.kubernetes.io/part-of:kube-prometheus
app.kubernetes.io/version:2.32.1
name:prometheus-k8s
namespace:monitoring
spec:
type:NodePort
ports:
-name:web
port:9090
nodePort:31200
targetPort:web
-name:reloader-web
port:8080
nodePort:31300
targetPort:reloader-web
selector:
app.kubernetes.io/component:prometheus
app.kubernetes.io/instance:k8s
app.kubernetes.io/name:prometheus
app.kubernetes.io/part-of:kube-prometheus
sessionAffinity:ClientIP
root@hello:~/kube-prometheus/manifests#sed-i"/ports:/i\type:NodePort"alertmanager-service.yaml
root@hello:~/kube-prometheus/manifests#sed-i"/targetPort:web/i\nodePort:31400"alertmanager-service.yaml
root@hello:~/kube-prometheus/manifests#sed-i"/targetPort:reloader-web/i\nodePort:31500"alertmanager-service.yaml
root@hello:~/kube-prometheus/manifests#catalertmanager-service.yaml
apiVersion:v1
kind:Service
metadata:
labels:
app.kubernetes.io/component:alert-router
app.kubernetes.io/instance:main
app.kubernetes.io/name:alertmanager
app.kubernetes.io/part-of:kube-prometheus
app.kubernetes.io/version:0.23.0
name:alertmanager-main
namespace:monitoring
spec:
type:NodePort
ports:
-name:web
port:9093
nodePort:31400
targetPort:web
-name:reloader-web
port:8080
nodePort:31500
targetPort:reloader-web
selector:
app.kubernetes.io/component:alert-router
app.kubernetes.io/instance:main
app.kubernetes.io/name:alertmanager
app.kubernetes.io/part-of:kube-prometheus
sessionAffinity:ClientIP

执行部署

root@hello:~#
root@hello:~#kubectlcreate-f/root/kube-prometheus/manifests/setup
customresourcedefinition.apiextensions.k8s.io/alertmanagerconfigs.monitoring.coreos.comcreated
customresourcedefinition.apiextensions.k8s.io/alertmanagers.monitoring.coreos.comcreated
customresourcedefinition.apiextensions.k8s.io/podmonitors.monitoring.coreos.comcreated
customresourcedefinition.apiextensions.k8s.io/probes.monitoring.coreos.comcreated
customresourcedefinition.apiextensions.k8s.io/prometheuses.monitoring.coreos.comcreated
customresourcedefinition.apiextensions.k8s.io/prometheusrules.monitoring.coreos.comcreated
customresourcedefinition.apiextensions.k8s.io/servicemonitors.monitoring.coreos.comcreated
customresourcedefinition.apiextensions.k8s.io/thanosrulers.monitoring.coreos.comcreated
namespace/monitoringcreated
root@hello:~#
root@hello:~#
root@hello:~#
root@hello:~#kubectlcreate-f/root/kube-prometheus/manifests/
alertmanager.monitoring.coreos.com/maincreated
networkpolicy.networking.k8s.io/alertmanager-maincreated
poddisruptionbudget.policy/alertmanager-maincreated
prometheusrule.monitoring.coreos.com/alertmanager-main-rulescreated
secret/alertmanager-maincreated
service/alertmanager-maincreated
serviceaccount/alertmanager-maincreated
servicemonitor.monitoring.coreos.com/alertmanager-maincreated
clusterrole.rbac.authorization.k8s.io/blackbox-exportercreated
clusterrolebinding.rbac.authorization.k8s.io/blackbox-exportercreated
configmap/blackbox-exporter-configurationcreated
deployment.apps/blackbox-exportercreated
networkpolicy.networking.k8s.io/blackbox-exportercreated
service/blackbox-exportercreated
serviceaccount/blackbox-exportercreated
servicemonitor.monitoring.coreos.com/blackbox-exportercreated
secret/grafana-configcreated
secret/grafana-datasourcescreated
configmap/grafana-dashboard-alertmanager-overviewcreated
configmap/grafana-dashboard-apiservercreated
configmap/grafana-dashboard-cluster-totalcreated
configmap/grafana-dashboard-controller-managercreated
configmap/grafana-dashboard-grafana-overviewcreated
configmap/grafana-dashboard-k8s-resources-clustercreated
configmap/grafana-dashboard-k8s-resources-namespacecreated
configmap/grafana-dashboard-k8s-resources-nodecreated
configmap/grafana-dashboard-k8s-resources-podcreated
configmap/grafana-dashboard-k8s-resources-workloadcreated
configmap/grafana-dashboard-k8s-resources-workloads-namespacecreated
configmap/grafana-dashboard-kubeletcreated
configmap/grafana-dashboard-namespace-by-podcreated
configmap/grafana-dashboard-namespace-by-workloadcreated
configmap/grafana-dashboard-node-cluster-rsrc-usecreated
configmap/grafana-dashboard-node-rsrc-usecreated
configmap/grafana-dashboard-nodescreated
configmap/grafana-dashboard-persistentvolumesusagecreated
configmap/grafana-dashboard-pod-totalcreated
configmap/grafana-dashboard-prometheus-remote-writecreated
configmap/grafana-dashboard-prometheuscreated
configmap/grafana-dashboard-proxycreated
configmap/grafana-dashboard-schedulercreated
configmap/grafana-dashboard-workload-totalcreated
configmap/grafana-dashboardscreated
deployment.apps/grafanacreated
networkpolicy.networking.k8s.io/grafanacreated
prometheusrule.monitoring.coreos.com/grafana-rulescreated
service/grafanacreated
serviceaccount/grafanacreated
servicemonitor.monitoring.coreos.com/grafanacreated
prometheusrule.monitoring.coreos.com/kube-prometheus-rulescreated
clusterrole.rbac.authorization.k8s.io/kube-state-metricscreated
clusterrolebinding.rbac.authorization.k8s.io/kube-state-metricscreated
deployment.apps/kube-state-metricscreated
networkpolicy.networking.k8s.io/kube-state-metricscreated
prometheusrule.monitoring.coreos.com/kube-state-metrics-rulescreated
service/kube-state-metricscreated
serviceaccount/kube-state-metricscreated
servicemonitor.monitoring.coreos.com/kube-state-metricscreated
prometheusrule.monitoring.coreos.com/kubernetes-monitoring-rulescreated
servicemonitor.monitoring.coreos.com/kube-apiservercreated
servicemonitor.monitoring.coreos.com/corednscreated
servicemonitor.monitoring.coreos.com/kube-controller-managercreated
servicemonitor.monitoring.coreos.com/kube-schedulercreated
servicemonitor.monitoring.coreos.com/kubeletcreated
clusterrole.rbac.authorization.k8s.io/node-exportercreated
clusterrolebinding.rbac.authorization.k8s.io/node-exportercreated
daemonset.apps/node-exportercreated
networkpolicy.networking.k8s.io/node-exportercreated
prometheusrule.monitoring.coreos.com/node-exporter-rulescreated
service/node-exportercreated
serviceaccount/node-exportercreated
servicemonitor.monitoring.coreos.com/node-exportercreated
clusterrole.rbac.authorization.k8s.io/prometheus-k8screated
clusterrolebinding.rbac.authorization.k8s.io/prometheus-k8screated
networkpolicy.networking.k8s.io/prometheus-k8screated
poddisruptionbudget.policy/prometheus-k8screated
prometheus.monitoring.coreos.com/k8screated
prometheusrule.monitoring.coreos.com/prometheus-k8s-prometheus-rulescreated
rolebinding.rbac.authorization.k8s.io/prometheus-k8s-configcreated
rolebinding.rbac.authorization.k8s.io/prometheus-k8screated
rolebinding.rbac.authorization.k8s.io/prometheus-k8screated
rolebinding.rbac.authorization.k8s.io/prometheus-k8screated
role.rbac.authorization.k8s.io/prometheus-k8s-configcreated
role.rbac.authorization.k8s.io/prometheus-k8screated
role.rbac.authorization.k8s.io/prometheus-k8screated
role.rbac.authorization.k8s.io/prometheus-k8screated
service/prometheus-k8screated
serviceaccount/prometheus-k8screated
servicemonitor.monitoring.coreos.com/prometheus-k8screated
apiservice.apiregistration.k8s.io/v1beta1.metrics.k8s.iocreated
clusterrole.rbac.authorization.k8s.io/prometheus-adaptercreated
clusterrole.rbac.authorization.k8s.io/system:aggregated-metrics-readercreated
clusterrolebinding.rbac.authorization.k8s.io/prometheus-adaptercreated
clusterrolebinding.rbac.authorization.k8s.io/resource-metrics:system:auth-delegatorcreated
clusterrole.rbac.authorization.k8s.io/resource-metrics-server-resourcescreated
configmap/adapter-configcreated
deployment.apps/prometheus-adaptercreated
networkpolicy.networking.k8s.io/prometheus-adaptercreated
poddisruptionbudget.policy/prometheus-adaptercreated
rolebinding.rbac.authorization.k8s.io/resource-metrics-auth-readercreated
service/prometheus-adaptercreated
serviceaccount/prometheus-adaptercreated
servicemonitor.monitoring.coreos.com/prometheus-adaptercreated
clusterrole.rbac.authorization.k8s.io/prometheus-operatorcreated
clusterrolebinding.rbac.authorization.k8s.io/prometheus-operatorcreated
deployment.apps/prometheus-operatorcreated
networkpolicy.networking.k8s.io/prometheus-operatorcreated
prometheusrule.monitoring.coreos.com/prometheus-operator-rulescreated
service/prometheus-operatorcreated
serviceaccount/prometheus-operatorcreated
servicemonitor.monitoring.coreos.com/prometheus-operatorcreated
root@hello:~#
root@hello:~#

查看验证

root@hello:~#kubectlgetpod-nmonitoring
NAMEREADYSTATUSRESTARTSAGE
alertmanager-main-02/2Running069s
alertmanager-main-12/2Running069s
alertmanager-main-22/2Running069s
blackbox-exporter-6c559c5c66-kw6vd3/3Running083s
grafana-7fd69887fb-jmpmp1/1Running081s
kube-state-metrics-867b64476b-h84g43/3Running081s
node-exporter-576bm2/2Running080s
node-exporter-94gn92/2Running080s
node-exporter-cbjqk2/2Running080s
node-exporter-mhlh72/2Running080s
node-exporter-pdc6k2/2Running080s
node-exporter-pqqds2/2Running080s
node-exporter-s9cz42/2Running080s
node-exporter-tdlnt2/2Running081s
prometheus-adapter-8f88b5b45-rrsh41/1Running078s
prometheus-adapter-8f88b5b45-wh6pf1/1Running078s
prometheus-k8s-02/2Running068s
prometheus-k8s-12/2Running068s
prometheus-operator-7f9d9c77f8-h5gkt2/2Running078s
root@hello:~#
root@hello:~#kubectlgetsvc-nmonitoring
NAMETYPECLUSTER-IPEXTERNAL-IPPORT(S)AGE
alertmanager-mainNodePort10.103.47.160<none>9093:31400/TCP,8080:31500/TCP92s
alertmanager-operatedClusterIPNone<none>9093/TCP,9094/TCP,9094/UDP78s
blackbox-exporterClusterIP10.102.108.160<none>9115/TCP,19115/TCP92s
grafanaNodePort10.106.2.21<none>3000:31100/TCP90s
kube-state-metricsClusterIPNone<none>8443/TCP,9443/TCP90s
node-exporterClusterIPNone<none>9100/TCP90s
prometheus-adapterClusterIP10.108.65.108<none>443/TCP87s
prometheus-k8sNodePort10.100.227.174<none>9090:31200/TCP,8080:31300/TCP88s
prometheus-operatedClusterIPNone<none>9090/TCP77s
prometheus-operatorClusterIPNone<none>8443/TCP87s
root@hello:~#
http://192.168.1.81:31400/
http://192.168.1.81:31200/
http://192.168.1.81:31100/

一条命令执行

cd/root;gitclone-brelease-0.10https://github.com/prometheus-operator/kube-prometheus.git;cdkube-prometheus/manifests;sed-i"s#quay.io/prometheus/#registry.cn-hangzhou.aliyuncs.com/chenby/#g"*.yaml;sed-i"s#quay.io/brancz/#registry.cn-hangzhou.aliyuncs.com/chenby/#g"*.yaml;sed-i"s#k8s.gcr.io/prometheus-adapter/#registry.cn-hangzhou.aliyuncs.com/chenby/#g"*.yaml;sed-i"s#quay.io/prometheus-operator/#registry.cn-hangzhou.aliyuncs.com/chenby/#g"*.yaml;sed-i"s#k8s.gcr.io/kube-state-metrics/#registry.cn-hangzhou.aliyuncs.com/chenby/#g"*.yaml;sed-i"/ports:/i\type:NodePort"grafana-service.yaml;sed-i"/targetPort:http/i\nodePort:31100"grafana-service.yaml;sed-i"/ports:/i\type:NodePort"prometheus-service.yaml;sed-i"/targetPort:web/i\nodePort:31200"prometheus-service.yaml;sed-i"/targetPort:reloader-web/i\nodePort:31300"prometheus-service.yaml;sed-i"/ports:/i\type:NodePort"alertmanager-service.yaml;sed-i"/targetPort:web/i\nodePort:31400"alertmanager-service.yaml;sed-i"/targetPort:reloader-web/i\nodePort:31500"alertmanager-service.yaml;kubectlcreate-f/root/kube-prometheus/manifests/setup;kubectlcreate-f/root/kube-prometheus/manifests/;sleep30;kubectlgetpod-nmonitoring;kubectlgetsvc-nmonitoring;

https://www.oiox.cn/

https://www.chenby.cn/

https://cby-chen.github.io/

https://blog.csdn.net/qq_33921750

https://my.oschina.net/u/3981543

https://www.zhihu.com/people/chen-bu-yun-2

https://segmentfault.com/u/hppyvyv6/articles

https://juejin.cn/user/3315782802482007

https://cloud.tencent.com/developer/column/93230

https://www.jianshu.com/u/0f894314ae2c

https://www.toutiao.com/c/user/token/MS4wLjABAAAAeqOrhjsoRZSj7iBJbjLJyMwYT5D0mLOgCoo4pEmpr4A/

CSDN、GitHub、知乎、开源中国、思否、掘金、简书、腾讯云、*今条头日**、个人博客、全网可搜《小陈运维》

文章主要发布于微信公众号：《Linux运维交流社区》

kubernetesk8s使用教程 (prometheus k8s)

kubernetes（k8s） 安装 Prometheus + Grafana