ffmpeg using multiple GPUs of same model and performance, but different usage

OS Ubuntu 22.04.1
ffmpeg version N-109649-gab8cde6efa

I installed the GPU drivers from an official source.

wget https://international.download.nvidia.com/XFree86/Linux-x86_64/525.78.01/NVIDIA-Linux-x86_64-525.78.01.run
chmod +x NVIDIA-Linux-x86_64-525.78.01.run
sudo ./NVIDIA-Linux-x86_64-525.78.01.run

And compiled ffmpeg from source as well to enable the hardware encoding.

git clone https://git.ffmpeg.org/ffmpeg.git ffmpeg/
cd ffmpeg
./configure --enable-libfribidi --enable-libfontconfig --enable-filter=drawtext --enable-gpl --enable-libx264 --enable-libfreetype --enable-nonfree --enable-cuda-nvcc --enable-libnpp --extra-cflags=-I/usr/local/cuda/include --extra-ldflags=-L/usr/local/cuda/lib64
make -j $(nproc)
sudo mv ffmpeg /usr/bin

This is the output of nvtop

The GPU usage of GPU 0 is much lower than that of GPU 1 despite having the same command and using the same file. GPU 0 is plugged into the first PCI-E 16x slot and GPU 1 is plugged into the next PCI-E 16x slow.

The Input video is a H.264 1080P 30FPS video. This is the command that I’m using to encode the videos.

  ffmpeg -hide_banner -y 
    -hwaccel cuda 
    -hwaccel_device $hwaccel_device 
    -i tmp/videos/$2/$1.mp4 
    -filter:v fps=30 
    -threads 16 
    -c:a aac 
    -ar 48000 
    -c:v h264_nvenc 
    -profile:v main 
    -crf 20 
    -sc_threshold 0 
    -vf scale=w=1920:h=1080:force_original_aspect_ratio=decrease 
    -b:v 5000k 
    -maxrate 4500000 
    -bufsize 4500000 
    -b:a 192k 
    tmp/videos/$2/$1_1080p.mp4

Why is the GPU 0 usage much lower than GPU 1? Also important to note, the CPU does have an onboard graphics and that’s what I’m using to drive the monitor, so the GPUs should be dedicated to encoding.