Nginx+Keepalived实现高可用案例

image.png

背景

实现高可用，防止单点发生故障导致服务下线影响业务正常进行

Keepalived软件主要是通过VRRP协议实现高可用功能的。VRRP是Virtual Router RedundancyProtocol(虚拟路由器冗余协议）的缩写,VRRP出现的目的就是为了解决静态路由单点故障问题的，它能够保证当个别节点宕机时，整个网络可以不间断地运行。（当一个节点挂掉，走另外一个节点）

搭建

1.下载keepalived

https://www.keepalived.org/index.html

1
2
3
4
5
6
7
8
9
10
wget https://www.keepalived.org/software/keepalived-2.2.7.tar.gz --no-check-certificate
tar -xzvf keepalived-2.2.7.tar.gz
mv keepalived-2.2.7 keepalived
cd keepalived
yum -y install gcc openssl openssl-devel
./configure --prefix=/usr/local/keepalived
make install
echo 'export PATH=$PATH:/usr/local/keepalived/sbin/' >> ~/.bash_profile
keepalived -v
keepalived -h

2.下载nginx

http://nginx.org/

1
2
3
4
5
6
7
8
9
wget https://nginx.org/download/nginx-1.22.1.tar.gz --no-check-certificate
tar zxvf nginx-1.22.1.tar.gz 
cd nginx-1.22.1/
./configure --prefix=/usr/local/nginx
make
make install
./nginx -v
echo 'export PATH=$PATH:/usr/local/nginx/sbin/' >> ~/.bash_profile
nginx

机器配置

关闭防火墙和enforce

1
2
setenforce 0
systemctl disable --now firewalld

3.配置keepalived

1
vim /usr/local/etc/keepalived/keepalived.conf

主服务器

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
global_defs {
   router_id LVS_DEVEL
}

vrrp_instance VI_1 {
    state MASTER
    interface ens160
    virtual_router_id 51
    priority 100
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass 1111
    }
    virtual_ipaddress {
        192.168.100.100
    }
}

virtual_server 192.168.100.100 80 {
    delay_loop 6
    lb_algo rr
    lb_kind DR
    persistence_timeout 50
    protocol TCP

    real_server 192.168.100.21 80 {
        weight 1
        TCP_CHECK {
            connect_timeout 3
            retry 3
            delay_before_retry 3
        }
    }
    real_server 192.168.100.22 80 {
        weight 1
        TCP_CHECK {
            connect_timeout 3
            retry 3
            delay_before_retry 3
        }
    }
}

备服务器

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
global_defs {
   router_id LVS_DEVEL_SALVE
}

vrrp_instance VI_1 {
    state MASTER
    interface ens160
    virtual_router_id 51
    priority 80
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass 1111
    }
    virtual_ipaddress {
        192.168.100.100
    }
}

virtual_server 192.168.100.100 80 {
    delay_loop 6
    lb_algo rr
    lb_kind DR
    persistence_timeout 50
    protocol TCP

    real_server 192.168.100.21 80 {
        weight 1
        TCP_CHECK {
            connect_timeout 3
            retry 3
            delay_before_retry 3
        }
    }
    real_server 192.168.100.22 80 {
        weight 1
        TCP_CHECK {
            connect_timeout 3
            retry 3
            delay_before_retry 3
        }
    }
}

启动keepalived

1
systemctl enable --now keepalived

查看vip

1
ip a

我们配置的vip

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: ens160: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
    link/ether 00:0c:29:c2:de:91 brd ff:ff:ff:ff:ff:ff
    inet 192.168.100.21/24 brd 192.168.100.255 scope global noprefixroute ens160
       valid_lft forever preferred_lft forever
    inet 192.168.100.100/32 scope global ens160 (VIP)
       valid_lft forever preferred_lft forever
    inet6 fe80::20c:29ff:fec2:de91/64 scope link noprefixroute 
       valid_lft forever preferred_lft forever
3: tunl0@NONE: <NOARP> mtu 1480 qdisc noop state DOWN group default qlen 1000
    link/ipip 0.0.0.0 brd 0.0.0.0
4: dummy0: <BROADCAST,NOARP> mtu 1500 qdisc noop state DOWN group default qlen 1000
    link/ether e2:d1:8c:f6:aa:e8 brd ff:ff:ff:ff:ff:ff
5: kube-ipvs0: <BROADCAST,NOARP> mtu 1500 qdisc noop state DOWN group default 
    link/ether 0a:bd:4c:73:69:b2 brd ff:ff:ff:ff:ff:ff
    inet 10.98.13.147/32 scope global kube-ipvs0
       valid_lft forever preferred_lft forever
    inet 10.98.126.203/32 scope global kube-ipvs0
       valid_lft forever preferred_lft forever
    inet 10.108.37.26/32 scope global kube-ipvs0
       valid_lft forever preferred_lft forever
    inet 10.96.0.1/32 scope global kube-ipvs0
       valid_lft forever preferred_lft forever
    inet 10.96.0.10/32 scope global kube-ipvs0
       valid_lft forever preferred_lft forever
    inet 10.109.253.213/32 scope global kube-ipvs0
       valid_lft forever preferred_lft forever
    inet 10.100.200.149/32 scope global kube-ipvs0
       valid_lft forever preferred_lft forever
    inet 10.105.133.95/32 scope global kube-ipvs0
       valid_lft forever preferred_lft forever
    inet 10.109.192.172/32 scope global kube-ipvs0
       valid_lft forever preferred_lft forever

测试

默认正常走master节点

image.png

down掉master节点，VIP会飘到salve节点

image.png

重启master节点后VIP会飘会master节点

image.png

资料

官网：https://www.keepalived.org/
github：https://github.com/acassen/keepalived

中间件篇
Nginx+Keepalived实现高可用案例
https://mikeygithub.github.io/2022/11/06/yuque/Nginx+Keepalived 实现高可用案例/
作者
Mikey
发布于
2022年11月6日
许可协议