【网络架构】keepalive

一、keepalive基础

1.1 作用

1.2 原理

1.3 功能

二、keepalive安装

2.1 yum安装

2.2 编译安装

三、配置文件

3.1 keepalived相关文件

3.2 主配置的组成

3.2.1 全局配置

3.2.2 配置虚拟路由器

四、实际操作

4.1 lvs+keepalived高可用群集

4.2 keepalived+nginx

一、keepalive基础

1.1 作用

用于解决系统的单点故障

1.2 原理

根据vrrp协议的工作原理，虚拟路由冗余协议

多个路由器组成一个路由器组，即虚拟路由器。
虚拟路由器使用虚拟IP与外网交互，虚拟MAC与内网交互
获得VIP的路由器为主路由器（Master状态），其他路由器为备份路由器(Backup状态)
主路由器每隔 advertisement_intervl 秒（对应图中配置项 advert_int），向组内其他路由器发送通告消息（即心跳），告知本路由器的优先级等信息；
组播的方式发送，组播地址224.0.0.18
只有主路由器会响应ARP请求，组内其他路由器会丢弃ARP请求；
只有主路由器会响应对虚拟IP请求，组内其他路由器会丢弃对虚拟IP的请求；

1.3 功能

- 基于vrrp协议完成地址流动
- 为vip地址所在的节点生成ipvs规则(在配置文件中预先定义)
- 为ipvs集群的各RS做健康状态检测
- 基于脚本调用接口完成脚本中定义的功能，进而影响集群事务，以此支持nginx、haproxy等服务

二、keepalive安装

2.1 yum安装

yum install -y keepalived

2.2 编译安装

yum install gcc curl openssl-devel libnl3-devel net-snmp-devel  -y
# 安装依赖包环境https://keepalived.org/download.html
# 官网下载安装包
wget https://keepalived.org/software/keepalived-2.2.2.tar.gztar  xf   keepalived-2.2.2.tar.gz
cd   keepalived-2.2.2/./configure --prefix=/usr/local/keepalived
make   &&  make  install自动生成service文件#编译好后起不来原因是 没有配置文件 /etc/keepalived/keepalived.confmkdir    /etc/keepalived 
cp /usr/local/keepalived/etc/keepalived/keepalived.conf  /etc/keepalived/keepalived.conf 需要修改配置文件的21行  网卡名称， 否则起不来sed  -i   's/eth0/ens33/'

三、配置文件

3.1 keepalived相关文件

软件包名：keepalived
主程序文件：/usr/sbin/keepalived
主配置文件：/etc/keepalived/keepalived.conf
配置文件示例：/usr/share/doc/keepalived/
Unit File：/lib/systemd/system/keepalived.service
Unit File的环境配置文件：
/etc/sysconfig/keepalived CentOS

3.2 主配置的组成

/etc/keepalived/keepalived.conf配置组成

GLOBAL CONFIGURATION

Global definitions（全局配置）：定义邮件配置，route_id ，vrrp配置，组播地址等

VRRP CONFIGURATION

VRRP instance(s)：定义vrrp协议中的每个vrrp虚拟路由器的规则，基本信息

LVS CONFIGURATION（lvs调度服务器的规则设置）

Virtual server group(s)

Virtual server(s)：LVS集群的VS和RS

3.2.1 全局配置

#/etc/keepalived/keepalived.conf 
global_defs {notification_email {root@localhost#keepalived 发生故障切换时邮件发送的目标邮箱，可以按行区分写多个root@localhost360601212@qq.com }notification_email_from keepalived@localhost  #发邮件的地址smtp_server 127.0.0.1     #邮件服务器地址smtp_connect_timeout 30   #邮件服务器连接timeoutrouter_id   LVS01#每个keepalived主机唯一标识，建议使用当前主机名，但多节点重名不影响vrrp_skip_check_adv_addr  #对所有通告报文都检查，会比较消耗性能，启用此配置后，如果收到的通告报文和上一个报文是同一个路由器，则跳过检查，默认值为全检查
vrrp_strict 
#严格遵守VRRP协议,启用此项后以下状况将无法启动服务:1.无VIP地址 2.配置了单播邻居 3.在VRRP版本2中有IPv6地址，开启动此项并且没有配置vrrp_iptables时会自动开启iptables防火墙规则，默认导致VIP无法访问,建议不加此项配置。vrrp_garp_interval 0 #gratuitous ARP messages 免费ARP报文发送延迟，0表示不延迟vrrp_gna_interval 0 #unsolicited NA messages （不请自来）消息发送延迟vrrp_mcast_group4 225.0.0.18#指定组播IP地址范围：224.0.0.0到239.255.255.255,默认值：224.0.0.18 vrrp_iptables        #此项和vrrp_strict同时开启时，则不会添加防火墙规则,如果无配置vrrp_strict项,则无需启用此项配置
}

3.2.2 配置虚拟路由器

vrrp_instance <STRING> {
#<String>为vrrp的实例名,一般为业务名称配置参数......}
#配置参数：
state MASTER|BACKUP
#当前节点在此虚拟路由器上的初始状态，状态为MASTER或者BACKUP
interface IFACE_NAME 
#绑定为当前虚拟路由器使用的物理接口，如：eth0,bond0,br0,可以和VIP不在一个网卡
virtual_router_id VRID 
#每个虚拟路由器惟一标识，范围：0-255，每个虚拟路由器此值必须唯一，否则服务无法启动，同属一个虚拟路由器的多个keepalived节点必须相同,务必要确认在同一网络中此值必须唯一
priority 100 
#当前物理节点在此虚拟路由器的优先级，范围：1-254，值越大优先级越高,每个keepalived主机节点此值不同advert_int 1
#vrrp通告的时间间隔，默认1sauthentication { 
#认证机制auth_type AH|PASS   #AH为IPSEC认证(不推荐),PASS为简单密码(建议使用)auth_pass <PASSWORD> #预共享密钥，仅前8位有效，同一个虚拟路由器的多个keepalived节点必须一样
}
include    /etc/keealived/conf.d/*.conf
virtual_ipaddress { 
#虚拟IP,生产环境可能指定上百个IP地址<IPADDR>/<MASK> brd <IPADDR> dev <STRING> scope <SCOPE> label <LABEL>192.168.200.100 #指定VIP，不指定网卡，默认为,注意：不指定/prefix,默认为/32192.168.200.101/24 dev eth1   #指定VIP的网卡，建议和interface指令指定的岗卡不在一个网卡192.168.200.102/24 dev eth2 label eth2:1 #指定VIP的网卡label   ifconfig eth2:1   192.168.200.102/24
}
track_interface { 
#配置监控网络接口，一旦出现故障，则转为FAULT状态实现地址转移eth0eth1…
}

配置范例：

[root@localhost keepalived]#vim keepalived.conf
global_defs {notification_email {sysadmin@firewall.loc}notification_email_from Alexandre.Cassen@firewall.locsmtp_server 127.0.0.1  #邮箱地址写成自己smtp_connect_timeout 30router_id lvs_01   #修改成主机名vrrp_skip_check_adv_addr#vrrp_strict   注释严格模式vrrp_garp_interval 0vrrp_gna_interval 0
}
include   /etc/keepalived/conf.d/*.conf  #添加子配置文件[root@localhost keepalived]#mkdir  conf.d
[root@localhost conf.d]# vim  conf.d/test.conf
vrrp_instance VI_1 {state MASTERinterface ens33   #修改网卡virtual_router_id 51  #路由器id 需要主从保持一致priority 100   advert_int 1authentication {auth_type PASSauth_pass 1111}virtual_ipaddress {192.168.91.188/24   #设置虚拟ip}
}将配置文件复制到从节点
scp  keepalived.conf    192.168.91.11:/etc/keepalived/
scp  keepalived.conf    192.168.91.11:/etc/conf.d/test.conf修改配置文件
global_defs {notification_email {sysadmin@firewall.loc}notification_email_from Alexandre.Cassen@firewall.locsmtp_server 127.0.0.1  smtp_connect_timeout 30router_id lvs_02  #修改成主机名vrrp_skip_check_adv_addr#vrrp_strict   注释严格模式vrrp_garp_interval 0vrrp_gna_interval 0
}
include   /etc/keepalived/conf.d/*.conf  #添加子配置文件vrrp_instance VI_1 {state BACKUP       #修改类型为备节点interface ens33  virtual_router_id 51  #路由器id 需要主从保持一致   不用修改priority 80           #优先级调小advert_int 1authentication {auth_type PASSauth_pass 1111}virtual_ipaddress {192.168.91.188/24   #设置虚拟ip}
}

四、实际操作

4.1 lvs+keepalived高可用群集

环境配置

主keepalive：192.168.254.10 MASTER

从keepalive：192.168.254.20 SLAVE

web1：192.168.254.30

web2：192.168.254.40

vip：192.168.254.188

客户：192.168.254.50

主keepalive配置

systemctl stop firewalld
setenforce 0yum install -y ipvsadm keepalived
#安装lvs工具和keepalivedipvsadm-save > /etc/sysconfig/ipvsadm
vim /etc/keepalived/keepalived.confglobal_defs {notification_email {acassen@firewall.locfailover@firewall.locsysadmin@firewall.loc}notification_email_from Alexandre.Cassen@firewall.locsmtp_server 127.0.0.1#修改邮箱指向自己（10行）smtp_connect_timeout 30router_id LVS_01#指定服务器名称主备需要不一样（12行）vrrp_skip_check_adv_addr#vrrp_strict#14行需要注释否则服务启动有问题vrrp_garp_interval 0vrrp_gna_interval 0
}vrrp_instance VI_1 {state MASTER#指定服务器类型MASTER为主 BACKUP为备（20行）interface ens33#修改网卡名称为ens33（21）virtual_router_id 10#指定虚拟路由器的ID号主备需要一致#nopreempt #非抢占模式两个节点都需要配置去掉注释priority 100#设定优先级数字越大优先级越高，准备需要不一样advert_int 1#通告间隔（查看是否存活）authentication {auth_type PASS#认证类型auth_pass 123456#修改验证密码，主备需要一样（27行）}virtual_ipaddress {192.168.254.188#指定群集vip地址}
}
virtual_server 192.168.254.188 80 {delay_loop 6#健康间隔时间6秒lb_algo rr#调度算法轮询lb_kind DR#lvs模式为DR persistence_timeout 0#连接保持时间改为0 否则 无法体现效果protocol TCP#采用协议real_server 192.168.254.30 80 {#43行修改地址为真实主机地址weight 1#45行删除#节点权重TCP_CHECK {connect_port 80#检查目标端口connect_timeout 3#连接超时 nb_get_retry 3#重试次数delay_before_retry 3#重试间隔时间}}real_server 192.168.254.40 80 {#第二个weight 1TCP_CHECK{connect_port 80connect_timeout 3nb_get_retry 3delay_before_retry 3}}systemctl start keepalivedvim /etc/sysctl.conf
net.ipv4.conf.all.send_redirects = 0
net.ipv4.conf.default.send_redirects = 0
net.ipv4.conf.ens33.send_redirects = 0 sysctl -p

从keepalive配置

systemctl stop firewalld
setenforce 0yum install -y ipvsadm keepalived
#安装lvs工具和keepalivedipvsadm-save > /etc/sysconfig/ipvsadm
vim /etc/keepalived/keepalived.confglobal_defs {notification_email {acassen@firewall.locfailover@firewall.locsysadmin@firewall.loc}notification_email_from Alexandre.Cassen@firewall.locsmtp_server 127.0.0.1#修改邮箱指向自己（10行）smtp_connect_timeout 30router_id LVS_02#指定服务器名称主备需要不一样（12行）vrrp_skip_check_adv_addr#vrrp_strict#14行需要注释否则服务启动有问题vrrp_garp_interval 0vrrp_gna_interval 0
}vrrp_instance VI_1 {state BACKUP#指定服务器类型MASTER为主 BACKUP为备（20行）interface ens33#修改网卡名称为ens33（21）virtual_router_id 10#指定虚拟路由器的ID号主备需要一致#nopreempt #非抢占模式两个节点都需要配置去掉注释priority 80#设定优先级数字越大优先级越高，准备需要不一样advert_int 1#通告间隔（查看是否存活）authentication {auth_type PASS#认证类型auth_pass 123456#修改验证密码，主备需要一样（27行）}virtual_ipaddress {192.168.254.188#指定群集vip地址}
}
virtual_server 192.168.254.188 80 {delay_loop 6#健康间隔时间6秒lb_algo rr#调度算法轮询lb_kind DR#lvs模式为DR persistence_timeout 0#连接保持时间改为0 否则 无法体现效果protocol TCP#采用协议real_server 192.168.254.30 80 {#43行修改地址为真实主机地址weight 1#45行删除#节点权重TCP_CHECK {connect_port 80#检查目标端口connect_timeout 3#连接超时 nb_get_retry 3#重试次数delay_before_retry 3#重试间隔时间}}real_server 192.168.254.40 80 {#第二个weight 1TCP_CHECK{connect_port 80connect_timeout 3nb_get_retry 3delay_before_retry 3}}systemctl start keepalivedvim /etc/sysctl.conf
net.ipv4.conf.all.send_redirects = 0
net.ipv4.conf.default.send_redirects = 0
net.ipv4.conf.ens33.send_redirects = 0 sysctl -p

web1：

systemctl stop firewalld
setenforce 0yum install -y httpd
echo "web1" > /var/www/html/index.htmlifconfig lo:0 192.168.254.188 netmask 255.255.255.255vim /etc/sysctl.conf
net.ipv4.conf.all.arp_ignore = 1
net.ipv4.conf.all.arp_announce = 2
net.ipv4.conf.default.arp_ignore = 1
net.ipv4.conf.default.arp_announce = 2
net.ipv4.conf.lo.arp_ignore = 1
net.ipv4.conf.lo.arp_announce = 2sysctl -proute add -host 192.168.254.188 dev lo:0

web2：

systemctl stop firewalld
setenforce 0yum install -y httpd
echo "web2" > /var/www/html/index.htmlifconfig lo:0 192.168.254.188 netmask 255.255.255.255vim /etc/sysctl.conf
net.ipv4.conf.all.arp_ignore = 1
net.ipv4.conf.all.arp_announce = 2
net.ipv4.conf.default.arp_ignore = 1
net.ipv4.conf.default.arp_announce = 2
net.ipv4.conf.lo.arp_ignore = 1
net.ipv4.conf.lo.arp_announce = 2sysctl -proute add -host 192.168.254.188 dev lo:0

客户机：

curl 192.168.254.188web1
web2
web1
web2
...

4.2 keepalived+nginx

环境配置：

主keepalived+nginx：192.168.254.10

从keepalived+nginx：192.168.254.20

web1：192.168.254.30

web2：192.168.254.40

客户机：192.168.254.50

主keepalived+nginx

systemctl stop firewalld
setenforce 0yum install -y epel-release.noarch nginx keepalivedvim /etc/keepalived/keepalived.conf
global_defs {notification_email {acassen@firewall.locfailover@firewall.locsysadmin@firewall.loc}notification_email_from Alexandre.Cassen@firewall.locsmtp_server 178.168.200.1smtp_connect_timeout 30router_id LVS_01vrrp_skip_check_adv_addr#vrrp_strictvrrp_garp_interval 0vrrp_gna_interval 0
}vrrp_script check_nginx {script "/etc/keepalived/nginx.sh"interval 1weight -30fall 3rise 2timeout 2
}vrrp_instance VI_1 {state MASTERinterface ens33virtual_router_id 51priority 100advert_int 1authentication {auth_type PASSauth_pass 1111}virtual_ipaddress {192.168.254.188}track_script {check_nginx
}
}vim /etc/keepalived/nginx.sh
#!/bin/bash
killall -0 nginx
if [ $(echo $0) == 1 ]
then echo "请开启nginx"
else echo "已检测nginx"
fivim /etc/nginx/nginx.conf
#在http模块中，添加
upstream web {
server 192.168.254.30;
server 192.168.254.40;
}
#在server模块中，添加
location / {
proxy_pass http://web;
}systemctl start keepalived nginx

从keepalived+nginx

systemctl stop firewalld
setenforce 0yum install -y epel-release.noarch nginx keepalivedvim /etc/keepalived/keepalived.conf
global_defs {notification_email {acassen@firewall.locfailover@firewall.locsysadmin@firewall.loc}notification_email_from Alexandre.Cassen@firewall.locsmtp_server 178.168.200.1smtp_connect_timeout 30router_id LVS_01vrrp_skip_check_adv_addr#vrrp_strictvrrp_garp_interval 0vrrp_gna_interval 0
}vrrp_script check_nginx {script "/etc/keepalived/nginx.sh"interval 1weight -30fall 3rise 2timeout 2
}vrrp_instance VI_1 {state BACKUPinterface ens33virtual_router_id 51priority 100advert_int 1authentication {auth_type PASSauth_pass 1111}virtual_ipaddress {192.168.254.188}track_script {check_nginx
}
}vim /etc/keepalived/nginx.sh
#!/bin/bash
killall -0 nginx
if [ $(echo $0) == 1 ]
then echo "请开启nginx"
else echo "已检测nginx"
fivim /etc/nginx/nginx.conf
#在http模块中，添加
upstream web {
server 192.168.254.30;
server 192.168.254.40;
}
#在server模块中，添加
location / {
proxy_pass http://web;
}systemctl start keepalived nginx

web1 && web2

web1：

systemctl stop firewalld
setenforce 0yum install -y httpd
echo web1 > /var/www/html/index.htmlsystemctl start httpd

web2:

systemctl stop firewalld
setenforce 0yum install -y httpd
echo web2 > /var/www/html/index.htmlsystemctl start httpd

客户机：

curl 192.168.254.188web1
web2
web1
web2
web1
web2
...