I have created a web app which allows users to stream IP cams publicly via a code snippet. The problem is I run into performance issue at scale. I currently have 18 cameras all streaming 1080p via rtsp. I need these in rtmp for my NGINX server to work. My goal is to push this to 100-500 cameras continously streaming.
Here’s my stack:
Node JS
PM2
NGINX (with rtmp module)
fluent-ffmpeg
And here is my current server:
8 Core vCPU
12gb RAM
Ubuntu 22
about 100mpbs up and down
I use Node js to trigger an ffmpeg script each time a user creates a camera. PM2 works to keep track of each process on the server.
Here is an example FFMPEG command:
ffmpeg -hwaccel vdpau -hwaccel_device /dev/dri/renderD128 -rtsp_transport tcp -rtsp_flags prefer_tcp -i ‘rtsp://xxxxxxx’ -acodec copy -vcodec copy -max_delay 5000000 -bufsize 4M -preset fast -tune zerolatency -f flv ‘rtmp://xxxxxxx’
I get severe latency and streaming quality issue at around 10 cameras continuously streaming. When I run htop all 8 CPU cores bounce between 50-70% utilization. RAM is only 1.8/12gb
I figure simply copying the video and audio codecs I should be able to get more performance than this. I am wondering whether I am limited by the amount of cores on my server.
I need to know the true limits of the system so I can add more hardware, or can I make this more efficient somehow.
If you have any suggestions apart from boosting hardware please let me know!
Thanks y’all
