Problem: File transfer protocols on my NAS box running Unraid (SMB,FTP,SFTP,SCP) won’t maintain high transfer speeds. Often but not always start high, ~800 MB/s, and then after 1-4 GB of transfer falls to around 100 MB/s or less. Common correlation is slow down occurs after free memory is filled, seems the memory won’t flush as fast as should.
I have been troubleshooting this for a number of days, I will try to be brief and avoid a wall of text. This is related to Unraid OS for my NAS. No luck yet from support forum on this, may have hit dead end, maybe need different expertise and hoping some folks here know more about this.
Environment (PC <–> S2500 <–> NAS):
- 10gbe fiber network (Aruba S2500), Mellanox NIC’s on both ends.
- NAS: NVMe SSD cache, Corsair Force MP510 480GB. data written here first
- NAS: 8GB DDR4 2666 (Ordered more to test with dual channel)
- NAS: Intel i3-8100T, ASRock B365M PRO4-F
- Jumbo packets (9kb) configured on NIC’s and switch ports alike
- Tested file transfers (Windows 10 -> Unraid) with SMB, FTP, SCP, SFTP. Generally focused on SMB and FTP (vsftpd, no encryption).
Results from multiple tests/troubleshooting:
- iperf3: ~9.5 Gbps sustained throughput PC to NAS over network.
- dd (using openssl to generate cipher data off of /dev/zero): ~920 MB/s Write IO to cache SSD
- Low CPU utilization when transfers slow to 100 MB/s, plenty of room to do more.
- Transfers rapid roughly until memory filled, then slow while flushing to disk.
- Transfers over network to /dev/shm also slow down (this is a ram disk as I understand it).
- Copied large file from /dev/shm/ to /mnt/cache/Media/ very fast, 1.0 GB/s no slowdown(?!) (Memory -> SSD without file transfer protocols).
- I’ve conducted iperf3 + dd tests concurrently, slight performance impact, no significant slowdown. Don’t see any PCIe related throughput problems.
Summary: Hardware components can operate fast, things only slow down when a file transfer software is involved. I’ve not been able to identify the exact bottleneck. I’ve done tests that show the memory, storage and network are all capable of ~1.0 GB/s, and yet transfers always eventually fall to ~100 MB/s. I haven’t come across any performance tuning for FTP/SMB that could improve on this.
Can anyone give me some ideas/guidance on what else I can do to figure this out? 🙂
