Paul Hirose
2009-11-25 19:58:25 UTC
Keepalived 1.1.19 running on CentOS 5.4 using HAProxy 1.3.22 as the service traffic load balancer. This much works fine :) I have two LBs both running KA, and it fails over wonderfully when I pull the plug on one LB or the other.
I was trying to use the vrrp_script to watch the haproxy software running on the active/master load-balancer:
vrrp_script chk_haproxy { # Requires keepalived-1.1.13
script "killall -0 haproxy" # cheaper than pidof
interval 2 # check every 2 seconds
}
track_script {
chk_haproxy
}
I found this works fine as long as haproxy is running on the master.
When I manually kill haproxy to test it, Keepalived switches to the FAULT state. I thought (incorrectly I guess?) this would tell the LB in the BACKUP state to switch to the MASTER state (like it would, if the first LB actually went down.) But it doesn't. The LB in BACKUP state, stays there. So now I have one Keepalived system on the Backup state and another in the Fault state, and none in the Master state.
First, is this correct and expected? Or am I missing something that should have made the Backup go into the Master state, when the current Master goes into the Fault state?
I also use the following:
notify_master /opt/keepalived/bin/notify_master.sh
notify_backup /opt/keepalived/bin/notify_backup.sh
notify_fault /opt/keepalived/bin/notify_backup.sh
The notify_master.sh script just runs haproxy. So when a system goes from whatever state to Master state, it should start the haproxy process. This works :) When a system goes from whatever state to the Backup state, it kills the haproxy process. I wasn't sure what Fault state really was (I'm still not), so I run the same script when a system goes into Fault state, and kill the haproxy.
Is the only way a system goes into a Fault state when the vrrp_script fails? The only way that script fails is if haproxy isn't running. So if that causes the system to enter the Fault state, I could create a notify_fault.sh which would basically be the same as notify_master.sh (eg: it would (re)execute the haproxy program.) I'd then have to modify notify_master.sh to check for the existence of haproxy before trying to run it (probably a good idea anyway).
I know I could eliminate all this by simply not running HAProxy and simply using the full ability of Keepalived (I'm essentially only using it in VRRP mode, since I have no virtual_servers defined.) I just found haproxy easier to get going than the virtual_server part of the Keepalived stuff (not necessarily because of Keepalived itself, but because of the whole NAT/DR/TUN type stuff and I'm not sure how my network is laid-out.) But my network layout, I think, is still one single network, all using real/routable IP addresses, all with single interfaces, etc. I tried going through the LVS-DR document about using only one interface, but that didn't work out :)
If anyone can help me out with this whole fault/backup/master state stuff and transitioning between them, and/or how to get an existing backup-state system to go to master when the current master goes into fault, etc, I'd greatly appreciate it.
Thank you,
PH
Paul Hirose : ***@ucdavis.edu : Sysadm Motto: rm -fr /MyLife
I was trying to use the vrrp_script to watch the haproxy software running on the active/master load-balancer:
vrrp_script chk_haproxy { # Requires keepalived-1.1.13
script "killall -0 haproxy" # cheaper than pidof
interval 2 # check every 2 seconds
}
track_script {
chk_haproxy
}
I found this works fine as long as haproxy is running on the master.
When I manually kill haproxy to test it, Keepalived switches to the FAULT state. I thought (incorrectly I guess?) this would tell the LB in the BACKUP state to switch to the MASTER state (like it would, if the first LB actually went down.) But it doesn't. The LB in BACKUP state, stays there. So now I have one Keepalived system on the Backup state and another in the Fault state, and none in the Master state.
First, is this correct and expected? Or am I missing something that should have made the Backup go into the Master state, when the current Master goes into the Fault state?
I also use the following:
notify_master /opt/keepalived/bin/notify_master.sh
notify_backup /opt/keepalived/bin/notify_backup.sh
notify_fault /opt/keepalived/bin/notify_backup.sh
The notify_master.sh script just runs haproxy. So when a system goes from whatever state to Master state, it should start the haproxy process. This works :) When a system goes from whatever state to the Backup state, it kills the haproxy process. I wasn't sure what Fault state really was (I'm still not), so I run the same script when a system goes into Fault state, and kill the haproxy.
Is the only way a system goes into a Fault state when the vrrp_script fails? The only way that script fails is if haproxy isn't running. So if that causes the system to enter the Fault state, I could create a notify_fault.sh which would basically be the same as notify_master.sh (eg: it would (re)execute the haproxy program.) I'd then have to modify notify_master.sh to check for the existence of haproxy before trying to run it (probably a good idea anyway).
I know I could eliminate all this by simply not running HAProxy and simply using the full ability of Keepalived (I'm essentially only using it in VRRP mode, since I have no virtual_servers defined.) I just found haproxy easier to get going than the virtual_server part of the Keepalived stuff (not necessarily because of Keepalived itself, but because of the whole NAT/DR/TUN type stuff and I'm not sure how my network is laid-out.) But my network layout, I think, is still one single network, all using real/routable IP addresses, all with single interfaces, etc. I tried going through the LVS-DR document about using only one interface, but that didn't work out :)
If anyone can help me out with this whole fault/backup/master state stuff and transitioning between them, and/or how to get an existing backup-state system to go to master when the current master goes into fault, etc, I'd greatly appreciate it.
Thank you,
PH
Paul Hirose : ***@ucdavis.edu : Sysadm Motto: rm -fr /MyLife