How to funnel local network traffic through local cache proxy?

Sorry if this might seem obvious but I can’t seem to figure how to set this up.

I would like all devices connected to my local network and accessing Internet to do so through a local cache proxy which I configured in a Proxmox box.

I would have liked a result alike setting up a DNS for the local network. I just had to configure the router to distribute the setting to all devices and they are all using the local DNS first. I understand that they can by pass this setting but it is the default behavior.

How to force all traffic to go through the proxy as opposed to set each device individually to do so?