alertmanger HA 方法

alertmanager 启动时添加mesh相关配置

1
2
3
./alertmanager -config.file simple.yml -mesh.peer=am-1:6783 -mesh.peer=am-2:6783 -mesh.peer=am-3:6783
./alertmanager -config.file simple.yml -mesh.peer=am-1:6783 -mesh.peer=am-2:6783 -mesh.peer=am-3:6783
./alertmanager -config.file simple.yml -mesh.peer=am-1:6783 -mesh.peer=am-2:6783 -mesh.peer=am-3:6783

prometheus 节点添加alertmanager相关配置

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
#在prome-1上编辑prometheus配置并启动
cat > prometheus.yml << EOF
global:
external_labels:
dc: cn1
alerting:
alert_relabel_configs:
- source_labels: [dc] #将容易产生报警区别的label置成相同label
regex: (.+)\d+
target_label: dc
alertmanagers:
- static_configs:
- targets: ['am-1:9093', 'am-2:9093', am-3:6783]
# 其他配置和正常prometheus配置相同
EOF
./prometheus
#在prome-2上编辑prometheus配置并启动
cat > prometheus.yml << EOF
global:
external_labels:
dc: cn2
alerting:
alert_relabel_configs:
- source_labels: [dc]
regex: (.+)\d+
target_label: dc
alertmanagers:
- static_configs:
- targets: ['am-1:9093', 'am-2:9093', am-3:6783]
# 其他配置和正常prometheus配置相同
EOF
./prometheus
#在prome-3上编辑prometheus配置并启动
cat > prometheus.yml << EOF
global:
external_labels:
dc: cn3
alerting:
alert_relabel_configs:
- source_labels: [dc]
regex: (.+)\d+
target_label: dc
alertmanagers:
- static_configs:
- targets: ['am-1:9093', 'am-2:9093', am-3:6783]
# 其他配置和正常prometheus配置相同
EOF
./prometheus

启动后查看各alertmanager后台,会发现报警信息已同步。

1
2
3
am-1:9093/#/alerts
am-2:9093/#/alerts
am-3:9093/#/alerts