This is a top hit on google so I figured I'd post the solution I found here. I also ran the issue of vms losing network access when the host machine joins a docker swarm. I had to turn on ip forwarding and then vms got their network access again. Add these lines to /etc/sysctl.conf...
