Why am I having VLAN connection issues on Raspberry Pi?

I try to connect my Raspberry Pi to a network configured with VLAN and SSH into a device in the network.

When I try using the same ethernet port using a Windows PC everything just works.

When I try to do the same on my Raspberry and ping the device I can see with Wireshark, that it seems like the device is only answering, when I ping it without any VLAN tag. But then responds with a package tagged VLAN 1.

So for me it looks like a miscommunication between my Raspberry Pi and the device.

The device answers in VLAN 1 but only responds to untagged packages and my Raspberry Pi can only do either one. Send/receive tagged or untagged but can‘t mix both.

Does this make sense? And if yes, can I configure the pi to handle this properly?