Zend certified PHP/Magento developer

Magento 2.3.2 Autoscaling and User Sessions

We have magento running on AWS Fargate behind a load balancer. It works great when it is not scaled(i.e only one task is running), however when it scales to 2 or more tasks I am not able to login as a customer or administrator and we aren’t getting any login errors, it just redirects to itself. I can only login successfully when only one task is running. We’re running magento 2.3.2 on AWS Fargate behind a load balancer. Any ideas why this might be happening?