Very slow write speeds - how to troubleshoot


rob
6 months ago 0 rob 1

Hi All,

i am working on testing out Lizardfs. I have a setup with 8 machines that have at least 128GB of ram and 30TB of storage. Setting up the system was straight forward. However, if i write to the storage i get only about 30MB/s write speeds. I looked into the network (10GBE) and it works just fine. I used NFS over the same network and i get 400-500MB write speeds. So i thought of removing all bottlenecks and trying to think of the simplest setup to test.


I came up with running everything on one machine and storing all data and metadata in a ramdisk. So the lizardfs-master, lizardfs-chunkserver are set up to store data/metadata completely in a ramdisk. Since everything is connected through localhost there should not be any networking issue as well. I can start the setup and i can use the lizardfs mount normally, but i still get only 30MB/s write speeds. I have not changed any settings beyond path of the data directories to point to a ramdisk. So it is a vanilla CenttOS using EPEL repositories for lizardfs 3.12. I use a single chunkserver with a single 4GB file that i am copying from a ramdisk... i should get speeds of several hundered MB/s (if not more), but i am getting only 30MB/s (the same as over the network). I thought there might be throttling in play, but at that does not seem to be activated. I am at a loss at what else i could check or how i could go about to find the limiting factor.


Does anybody know how what i could do to speed this up or how i could go about further troubleshooting it?


Thanks,

Rob.


Reading this thread: