HAproxy – KeepAlived important notes.

  • Multicast must be enabled for fail over to work. Otherwise, If the haproxy service stops but the keepalived service is still running, the VirtualIP will still be pingable however any requests made to that IP will not be forwarded via HAproxy simply because HAproxy is no longer running. The keepalived will switch to the standby server only when the keepalived service it self stops from ServerA. Then ServerB will automatically take over.  In other words, failure of HAproxy does not mean Keepalive will automatically switch to the standby server.
  • If for whatever reason you wish to run a HA service with out SSL then your HAproxy config files should listen to the virtual IP address.
Share Button

Centos 7 – Part 4 – HAProxy Standby with SSL support combined with NGINX Load Balancing

These instructions expand on the previous post. The previous post shows how to implement HAPROXY with SSL in front of two NGINX load balancers with NGINX servers having Fail Over enabled.  This post will show how to create add another HAPROXY server in order to have fail over enabled,

As explained on the previous post, HAPROXY and keepalived needs to be installed.

HAProxy

Configure HAProxy.

The configuration file for server HAPROXY2 is the same as with the configuration file with server HAPROXY1, minus of course the IP address that we bind. Important: You must copy the ssl certificate files from HAPROXY1 to HAPROXY2 server under the directory specified in the config file.

global
        log 127.0.0.1   local0
        log 127.0.0.1   local1 debug
        maxconn   45000 # Total Max Connections.
        daemon
        nbproc      1 # Number of processing cores.
defaults
        timeout server 86400000
        timeout connect 86400000
        timeout client 86400000
        timeout queue   1000s

frontend https_frontend
  bind 10.0.0.53:443 ssl crt /etc/ssl/haproxy1.sfentona.lol/haproxy1.pem
  mode http
  option httpclose
  option forwardfor
  reqadd X-Forwarded-Proto:\ https
  default_backend web_server

backend web_server
  mode http
  balance roundrobin
  cookie SERVERID insert indirect nocache
  server wordpressvirtip 10.0.0.44:80

Configure Keepalived

Keep in mind that we already have keepalived running for the two NGINX load balancers. We have designated them in the keepalived.cfg as virtual_router_id 51. For the HAproxy servers we are going to assign them a different id. Servers HAPROXY1 and HAPROXY2 will now be designated as virtual_router_id 52 .

Keepalived config file for HAPROXY1

vrrp_script chk_haproxy {           # Requires keepalived-1.1.13
script "killall -0 haproxy"     # cheaper than pidof
interval 2                      # check every 2 seconds
weight 2                        # add 2 points of prio if OK
}
vrrp_instance VI_1 {
interface ens192
state MASTER
virtual_router_id 52
priority 101                    # 101 on master, 100 on backup
virtual_ipaddress {
10.0.0.54
}
track_script {
chk_haproxy
}
}

Keepalived config file for HAPROXY2

vrrp_script chk_haproxy {           # Requires keepalived-1.1.13
script "killall -0 haproxy"     # cheaper than pidof
interval 2                      # check every 2 seconds
weight 2                        # add 2 points of prio if OK
}
vrrp_instance VI_1 {
interface ens192
state MASTER
virtual_router_id 52
priority 100                    # 101 on master, 100 on backup
virtual_ipaddress {
10.0.0.54
}
track_script {
chk_haproxy
}
}

Configure your Firewall
The following IPTABLE rules should be running on both HAPROXY1 and HAPROXY2. Copy and paste the following rules in a text file and import them to your firewall table.

# Generated by iptables-save v1.4.21 on Thu Oct  8 15:18:59 2015
*filter
:INPUT ACCEPT [26988:2784395]
:FORWARD ACCEPT [0:0]
:OUTPUT ACCEPT [35111:2263400]
-A INPUT -p tcp -m tcp --dport 80 -j ACCEPT
-A INPUT -p vrrp -j ACCEPT
-A INPUT -d 224.0.0.0/8 -j ACCEPT
COMMIT
# Completed on Thu Oct  8 15:18:59 2015
iptables-restore < /root/firewall.fw
Share Button

Centos 7 – Part 1 – NGINX – Apache – Load Balancing High Availability

These instructions show how to setup a web Load Balancer by using two NGINX servers as the Load Balancers and two Apache servers.

2 – Centos 7  servers running NGINX will be used as Load Balancers.

2 – Cent0s 7 serves running Apache will be used to serve web pages via virtual hosts.

The NGINX servers will:
– Determine the appropriate destination service based on the method chosen; in this case it will be Round Robin, which is the default option.
– Will use KeepAlive in order to create a virtual IP address. The IP address will based on the already NICs of the Load Balancers.

The Apache servers will:
– Act as your normal every day web servers with virtual hosts.
– Will log the IP of the actual client.

nginx-HA

Setting up the Apache web Servers.

  • Create the directories where the content of you virtual hosts will be placed.
mkdir /etc/httpd/vhosts.d
mkdir -p /sites/wordpress/
chown -R apache:apache /sites/wordpress/
chmod 755 /sites/wordpress/

 

  • Instruct Apache to look into the directory you created for your virtual hosts.
vim /etc/httpd/conf.d/vhosts.conf
IncludeOptional vhosts.d/*.conf

 

  • Create the config file for the corresponding web site.
vim /etc/httpd/vhosts.d/wordpress.sfentona.lol.conf
ServerAdmin webmaster@dummy-host.example.com
DocumentRoot /sites/wordpress/
ServerName wordpress.sfentona.lol
ServerAlias www.wordpress.sfentona.lol

Directory "/sites/wordpress";
DirectoryIndex index.html index.php
Options FollowSymLinks
AllowOverride All
Require all granted

 

  • Set up log x-fowarded-for in order to get the the IP of the actual client who visited the site and not the IP of the load balancer in your logs.
vi /etc/httpd/conf/httpd.conf
LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"" combined
LogFormat "%h %l %u %t \"%r\" %>s %b" common
LogFormat "%{X-Forwarded-For}i %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"" forwarded

 

Configure NGINX as a load balancer (view lines 34-38)

vi /etc/nginx/nginx.conf
# For more information on configuration, see:
#   * Official English Documentation: http://nginx.org/en/docs/
#   * Official Russian Documentation: http://nginx.org/ru/docs/

user nginx;
worker_processes auto;
error_log /var/log/nginx/error.log;
pid /run/nginx.pid;

events {
    worker_connections 1024;
}

http {
    log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
                      '$status $body_bytes_sent "$http_referer" '
                      '"$http_user_agent" "$http_x_forwarded_for"';

    access_log  /var/log/nginx/access.log  main;

    sendfile            on;
    tcp_nopush          on;
    tcp_nodelay         on;
    keepalive_timeout   65;
    types_hash_max_size 2048;

    include             /etc/nginx/mime.types;
    default_type        application/octet-stream;

    # Load modular configuration files from the /etc/nginx/conf.d directory.
    # See http://nginx.org/en/docs/ngx_core_module.html#include
    # for more information.
    include /etc/nginx/conf.d/*.conf;
    proxy_set_header        X-Real-IP $remote_addr;
    proxy_set_header        X-Forwarded-For $proxy_add_x_forwarded_for;
    upstream wordpress {
    server 10.0.0.42:80;
    server 10.0.0.43:80;
}


        server {
        listen       80;
        server_name  www.wordpress.sfentona.lol;

        # Load configuration files for the default server block.
        include /etc/nginx/default.d/*.conf;

        location / {
                proxy_pass http://wordpress;
        }

        error_page 404 /404.html;
            location = /40x.html {
        }

        error_page 500 502 503 504 /50x.html;
            location = /50x.html {
        }
    }
}

Allow NGINX to bind to a non-local shared ip

vi /etc/sysctl.conf
net.ipv4.ip_nonlocal_bind=1
sysctl -p

Set up your firewall in order for Multicast and VRRP to work correctly.

iptables -I INPUT -d 224.0.0.0/8 -j ACCEPT
iptables -I INPUT -p vrrp -j ACCEPT

 

Configure Keep Alive.

vi/etc/keepalived/keepalived.conf

This is for the Master Load Banacer LB1

 notification_email {
     sysadmin@mydomain.com
     support@mydomain.com
   }
   notification_email_from lb1@mydomain.com
   smtp_server localhost
   smtp_connect_timeout 30
}

vrrp_instance VI_1 {
    state MASTER
    interface ens192
    virtual_router_id 51
    priority 101
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass 1111
    }
    virtual_ipaddress {
        10.0.0.44
    }
}

This is for the Master Load Banacer LB2. Priority on the slave is a lower number.

global_defs {
   notification_email {
     sysadmin@mydomain.com
     support@mydomain.com
   }
   notification_email_from lb2@mydomain.com
   smtp_server localhost
   smtp_connect_timeout 30
}

vrrp_instance VI_1 {
    state MASTER
    interface ens192
    virtual_router_id 51
    priority 100
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass 1111
    }
    virtual_ipaddress {
        10.0.0.44
    }
}
Share Button