keepalived 简介
Keepalived is a routing software written in C. The main goal of this project is to provide simple and robust facilities for loadbalancing and high-availability to Linux system and Linux based infrastructures. Loadbalancing framework relies on well-known and widely used Linux Virtual Server (IPVS) kernel module providing Layer4 loadbalancing. Keepalived implements a set of checkers to dynamically and adaptively maintain and manage loadbalanced server pool according their health. On the other hand high-availability is achieved by VRRP protocol. VRRP is a fundamental brick for router failover. In addition, Keepalived implements a set of hooks to the VRRP finite state machine providing low-level and high-speed protocol interactions. Keepalived frameworks can be used independently or all together to provide resilient infrastructures. keepalived
Keepalived是一个由C编写的项目,主要目标是为Linux系统和基于Linux的架构提供高可用和负载均衡服务。负载均衡框架主要依赖于众所周知的**Linux Virtual Server(IPVS)内核提供的四层负载均衡。Keepalived可以根据他们的健康状态动态检测来管理负载均衡服务。 另一方面,keepalived的高可用是 VRRP(virtual route redundancy protocol)**协议的实现。 关于VRRP,参考H3C技术白皮书
keepalived架构图
keepalived 的工作原理
Keepalived 是一个基于VRRP协议来实现LVS服务高可用的方案。
一个LVS服务会有2台服务器运行keepalived 一台为主服务器MASTER一台为备份服务器BACKUP,对外只表现为一个虚拟IP;主服务器会发送特定的消息给备份服务器,当备份服务器收不到这个消息的时候即主服务器宕机的时候,备份服务器会直接接管虚拟IP继续提供服务从而保证了高可用性。
实验拓扑
主机规划
主机名
IP地址
角色
Director1
192.168.1.21
Direcotr
Director1
192.168.1.22
Direcotr
Rs1
192.168.1.23
Real Server
Rs2
192.168.1.24
Real Server
VIP1:192.168.1.41
VIP2:192.168.1.42
注意:所有节点iptables和selinux关闭
配置Keepalived-1
各节点时间同步,博主这里使用的是ansible自动化工具批量处理
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 root@director1 ~] 192.168.1.23 | SUCCESS => { "changed" : false , "ping" : "pong" } 192.168.1.24 | SUCCESS => { "changed" : false , "ping" : "pong" } 192.168.1.22 | SUCCESS => { "changed" : false , "ping" : "pong" } 192.168.1.21 | SUCCESS => { "changed" : false , "ping" : "pong" } [root@director1 ~] 192.168.1.22 | SUCCESS | rc=0 >> 25 Dec 15:54:02 ntpdate[2805]: step time server 202.112.29.82 offset -51.455956 sec 192.168.1.24 | SUCCESS | rc=0 >> 25 Dec 15:54:02 ntpdate[2449]: step time server 202.112.29.82 offset -86450.516769 sec 192.168.1.21 | SUCCESS | rc=0 >> 25 Dec 15:54:02 ntpdate[2894]: step time server 202.112.29.82 offset -51.292752 sec 192.168.1.23 | SUCCESS | rc=0 >> 25 Dec 15:54:03 ntpdate[2425]: step time server 202.112.29.82 offset 86408.418292 sec [root@director1 ~]
####实现Director的VIP互为主从
Direcotr1配置
1 2 3 [root@director1 ~] [root@director1 ~] [root@director1 ~]
Director1 keepalived配置
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 [root@director1 ~]# vim /etc/keepalived/keepalived.conf ! Configuration File for keepalived global_defs { notification_email { root@localhost } notification_email_from Alexandre.Cassen@firewall.loc smtp_server 127.0 .0 .1 smtp_connect_timeout 30 router_id LVS_DEVEL } vrrp_script chk_mt { script "[[ -f /etc/keepalived/down ]] && exit 1 || exit 0" interval 1 weight -2 } vrrp_instance VI_1 { state MASTER interface eno16777736 virtual_router_id 51 priority 100 advert_int 1 authentication { auth_type PASS auth_pass Daniel } virtual_ipaddress { 192.168 .1 .41 /32 } track_script { chk_mt } } vrrp_instance VI_2 { state BACKUP interface eno16777736 virtual_router_id 61 priority 99 advert_int 1 authentication { auth_type PASS auth_pass Daniel } virtual_ipaddress { 192.168 .1 .42 /32 } track_script { chk_mt } } virtual_server 192.168 .1 .41 80 { delay_loop 6 lb_algo wrr lb_kind DR nat_mask 255.255 .255 .255 protocol TCP sorry_server 127.0 .0 .1 80 real_server 192.168 .1 .23 80 { weight 1 HTTP_GET { url { path / status_code 200 } connect_timeout 3 nb_get_retry 3 delay_before_retry 3 } } real_server 192.168 .1 .24 80 { weight 2 HTTP_GET { url { path / status_code 200 } connect_timeout 3 nb_get_retry 3 delay_before_retry 3 } } } virtual_server 192.168 .1 .42 80 { delay_loop 6 lb_algo wrr lb_kind DR nat_mask 255.255 .255 .255 protocol TCP sorry_server 127.0 .0 .1 80 real_server 192.168 .1 .23 80 { weight 1 HTTP_GET { url { path / status_code 200 } connect_timeout 3 nb_get_retry 3 delay_before_retry 3 } } real_server 192.168 .1 .24 80 { weight 2 HTTP_GET { url { path / status_code 200 } connect_timeout 3 nb_get_retry 3 delay_before_retry 3 } } } ## 上述keepalived配置是双主模型Director完整配置
Derector2配置
1 2 3 [root@director2 ~] [root@director2 ~] [root@director2 ~]
Director2 keepalived配置
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 ! Configuration File for keepalived global_defs { notification_email { root@localhost } notification_email_from Alexandre.Cassen@firewall.loc smtp_server 127.0.0.1 smtp_connect_timeout 30 router_id LVS_DEVEL } vrrp_script chk_mt { script "[[ -f /etc/keepalived/down ]] && exit 1 || exit 0" interval 1 weight -2 } vrrp_instance VI_1 { state BACKUP interface eno16777736 virtual_router_id 51 priority 99 advert_int 1 authentication { auth_type PASS auth_pass Daniel } virtual_ipaddress { 192.168.1.41/32 } track_script { chk_mt } } vrrp_instance VI_1 { state MASTER interface eno16777736 virtual_router_id 61 priority 100 advert_int 1 authentication { auth_type PASS auth_pass Daniel } virtual_ipaddress { 192.168.1.42/32 } track_script { chk_mt } } virtual_server 192.168.1.41 80 { delay_loop 6 lb_algo wrr lb_kind DR nat_mask 255.255.255.255 protocol TCP sorry_server 127.0.0.1 80 real_server 192.168.1.23 80 { weight 1 HTTP_GET { url { path / status_code 200 } connect_timeout 3 nb_get_retry 3 delay_before_retry 3 } } real_server 192.168.1.24 80 { weight 2 HTTP_GET { url { path / status_code 200 } connect_timeout 3 nb_get_retry 3 delay_before_retry 3 } } } virtual_server 192.168.1.42 80 { delay_loop 6 lb_algo wrr lb_kind DR nat_mask 255.255.255.255 protocol TCP sorry_server 127.0.0.1 80 real_server 192.168.1.23 80 { weight 1 HTTP_GET { url { path / status_code 200 } connect_timeout 3 nb_get_retry 3 delay_before_retry 3 } } real_server 192.168.1.24 80 { weight 2 HTTP_GET { url { path / status_code 200 } connect_timeout 3 nb_get_retry 3 delay_before_retry 3 } } }
同时在Director1和Director2 上启动keepalived服务
1 2 [root@director1 ~] [root@director2 ~]
测试
默认情况下director1 和director2 的ip如下
将director1 的keepalived服务停掉,IP自动转移到director2
再次启动director1 的keepalived服务,IP再次回到director1
配置LVS
这里我们使用DR模型进行实验, 因为keepalived可以通过调用ipvs的接口来自动生成规则, 所以我们这里无需ipvsadm, 但是我们要通过ipvsadm命令来查看一下ipvs规则
上述keepalived配置文件中对real server 的定义已经加进去,此时我们重启下keepalived服务,查看生成的ipvs规则
1 2 3 4 5 6 7 8 9 10 11 [root@director1 keepalived] IP Virtual Server version 1.2.1 (size=4096) Prot LocalAddress:Port Scheduler Flags -> RemoteAddress:Port Forward Weight ActiveConn InActConn TCP 192.168.1.41:80 wrr -> 192.168.1.23:80 Route 1 0 0 -> 192.168.1.24:80 Route 2 0 0 TCP 192.168.1.42:80 wrr -> 192.168.1.23:80 Route 1 0 0 -> 192.168.1.24:80 Route 2 0 0 [root@director1 keepalived]
RS1配置
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 [root@rs1 ~] [root@rs1 ~] <h1>This is Rs1.Service</h1> [root@rs1 ~] [root@rs1 ~] #!/bin/bash case $1 in start) echo 1 > /proc/sys/net/ipv4/conf/all/arp_ignore echo 1 > /proc/sys/net/ipv4/conf/lo/arp_ignore echo 2 > /proc/sys/net/ipv4/conf/all/arp_announce echo 2 > /proc/sys/net/ipv4/conf/lo/arp_announce ifconfig lo:0 192.168.1.41/32 broadcast 192.168.1.41 up ifconfig lo:1 192.168.1.42/32 broadcast 192.168.1.42 up ;; stop) echo 0 > /proc/sys/net/ipv4/conf/all/arp_ignore echo 0 > /proc/sys/net/ipv4/conf/lo/arp_ignore echo 0 > /proc/sys/net/ipv4/conf/all/arp_announce echo 0 > /proc/sys/net/ipv4/conf/lo/arp_announce ifconfig lo:0 192.168.1.41/32 broadcast 192.168.1.41 down ifconfig lo:1 192.168.1.42/32 broadcast 192.168.1.42 down ;; esac [root@rs1 ~] [root@rs1 ~]
RS2配置
1 2 3 4 5 [root@rs2 ~] [root@rs2 ~] <h1>This is Rs2.Service</h1> [root@rs2 ~] [root@rs2 ~]
测试LVS
测试 Director1 和Director2
当我们关闭rs1的web服务, 会自动检查健康状态并删除
当我们同时关闭rs1和rs2的web服务, 会自动启用sorry server
至此 LVS+Keepalived双主模型已然实现,这次实验做了一天,中间遇到很多坑,总算是做好。记录下来方便以后忘记时翻阅。