Prometheus和Openshift

  1. prometheus.yml配置中绑定alertmanager服务

    1
    2
    3
    4
    5
    6
    7
    ......
    alerting:
      alertmanagers:
      - scheme: http
        static_configs:
        - targets:
          - "localhost:9093"

  2. prometheus.rules设置prometheus告警规则

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    ...
    rules:
    - alert: TooManyPods
      expr: kuberlet_running_pod_count > 10
      for: 2m
      labels:
        team: node
      annotations:
        summary: "{{$labels.instance}}: has {{$value}} pods"
        description: "{{$labels.instance}} be cateful"

    规则：kuberlet_running_pod_count指标持续2分钟超过10，则发出告警。

  3. AlertManager中设置告警接收器alertmanager.yml

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    global:
      smtp_smarthost: 'mail.xx.com:25'
      smtp_from: 'service@xx.com'
      smtp_auth_username: 'service@xx.com'
      smtp_auth_password: 'password'
      smtp_require_tls: false
    route:
      group_by: ['alertname']
      receiver: alert-email
    receivers:
    - name: alert-email
      email_configs:
      - to: 'pan@xx.com'
        send_resolved: true