There are many questions like this one, i.e. why hosts on the VPN LAN cannot be pinged, and I went over dozens of them, but in each instance the difference is that they forgot to open their firewalls or something silly along those lines, which I had eliminated along the way of setting things up.
In the process of setting up an edge router in a remote office, I configured the NAT and firewall to allow me to both administer the router through its web UI, by accessing it through the public IP, and to SSH into a Linux jump box on the LAN behind the router. So the basic admin tasks are working.
Next step, I wanted to make the administration of that remote office more secure, and for that I initially brought up an OpenVPN instance and created a user for it. For this task, I used the vendor’s guide and followed all of the steps, i.e. created the instance, saved, applied, created the user, saved, applied, created the 2x firewall rules for the access from the Internet and for the access from the VPN to the LAN, and finally I downloaded the ZIPped bundle of the cert/key/ovpn config, and imported it into Network Manager on my office machine. This did not work at all, in a sense that the network manager applet only displayed a flashing lock and timed out after one minute or so.
Next, I ran openvpn with that OVPN file from the bundle, and it connected successfully, but there was still a problem: even though the client was logged in and assigned a correct VPN LAN’s IP address from the pool, it could ping to or request HTTP from neither of the edge router or any hosts on the remote LAN, needless to say the Internet. While connected, I also lost access to the Internet on the client.
On my client side, the tap0 adapter was created, and the VPN LAN was added to the end of the routing table, but it was not made default.
Next, I shut down the OpenVPN instance and set up a new Wireguard instance. For that, I also followed the vendor’s guide and went down the very similar path of creating a new adapter, a new instance, a new client, an outbound NAT entry, and 2x firewall rules. All was saved and applied, and then I got the client definition, imported it into the WG client, and imported it into the Network Manager. Unlike with OpenVPN, I could immediately connect from the network manager applet, but I had the same problem: not pinging anything on the VPN side.
The only difference with WG was that its wg0 adapter became the 1st default route for the duration of the VPN connection, whereas the physical eth0 became the 2nd default route for the local LAN. Unlike with OpenVPN, my Internet access on the client continued to function normally.
To troubleshoot, I completely dropped the local firewall on my office machine. It made no difference. I also logged connections to the remote office on my local office’s edge router. They only showed single UDP packets going to the remote the moment the VPN connected but nothing further, so it seems that the pings were not being sent across. The hosts on the remote LAN that I tried to ping or to access over HTTP all allow such packets to pass.
I also logged all of the rules that I had created for each of the OpenVPN and WG on the remote office’s router, and the log showed the same thing: only the initial connection. Next, I created much broader rules that allowed the traffic to flow, but they showed nothing new in the log.
Previously, I used OpenVPN on my office’s edge router to allow staff to WFH, and none had any issues. Once any of us imported the bundle into our clients, it simply worked for host-to-LAN and to the outside Internet. The only difference now is that I am connecting from behind my local router.
The topology of both sites is very simple: there is a cable modem connected to the wall, an edge router’s WAN port plugged into the modem, a switch plugged into the router’s LAN port, and a bunch of mostly Linux devices connected to the switch. All of them are on the wired Ethernet, and there is no WiFi involved in any of that.
What other troubleshooting steps should I take to find and fix the root cause of the packets not flowing?