Multicast must be enabled for fail over to work. Otherwise, If the haproxy service stops but the keepalived service is still running, the VirtualIP will still be pingable however any requests made to that IP will not be forwarded via HAproxy simply because HAproxy is no longer running. The keepalived will switch to the standby server only when the keepalived service it self stops from ServerA. Then ServerB will automatically take over. In other words, failure of HAproxy does not mean Keepalive will automatically switch to the standby server.
If for whatever reason you wish to run a HA service with out SSL then your HAproxy config files should listen to the virtual IP address.