btrfs
btrfs copied to clipboard
maharmstone
Performance - NTFS vs WinBtrfs
I couldn't find any benchmarks or any information about the performance of WinBtrfs, so I made a quick test with CrystalDiskMark 5. It seems that winbtrfs has better random writes/reads but lower seq. write/read performance.
Have you made similar observations?
OS : Windows 10 Professional [10.0 Build 16299] (x64) WinBtrfs v1.0.1 on HDD
WinBTRFS with LZO Sequential Read (Q= 32,T= 1) : 119.619 MB/s Sequential Write (Q= 32,T= 1) : 93.616 MB/s Random Read 4KiB (Q= 32,T= 1) : 1.381 MB/s [ 337.2 IOPS] Random Write 4KiB (Q= 32,T= 1) : 1.874 MB/s [ 457.5 IOPS] Sequential Read (T= 1) : 117.870 MB/s Sequential Write (T= 1) : 122.909 MB/s Random Read 4KiB (Q= 1,T= 1) : 0.672 MB/s [ 164.1 IOPS] Random Write 4KiB (Q= 1,T= 1) : 4.304 MB/s [ 1050.8 IOPS]
NTFS Sequential Read (Q= 32,T= 1) : 152.757 MB/s Sequential Write (Q= 32,T= 1) : 145.346 MB/s Random Read 4KiB (Q= 32,T= 1) : 0.951 MB/s [ 232.2 IOPS] Random Write 4KiB (Q= 32,T= 1) : 1.098 MB/s [ 268.1 IOPS] Sequential Read (T= 1) : 154.369 MB/s Sequential Write (T= 1) : 146.380 MB/s Random Read 4KiB (Q= 1,T= 1) : 0.395 MB/s [ 96.4 IOPS] Random Write 4KiB (Q= 1,T= 1) : 1.153 MB/s [ 281.5 IOPS]
Thank you, that's interesting to know!
My guess is that the sequential reads and writes are slower because unlike NTFS Btrfs does checksumming. Have you tried running it in a directory with the nocsum flag turned on?
I can't find such a flag in btrfs properties. I guess I need btrfs-tools for that!?
And CrystalDiskMark creates a random folder in the root dir everytime... can I set nocsum for the whole drive?
If you have the shell extension installed, it should be a checkbox on the Btrfs properties tab. And yes, from memory I think you can set it on the root directory.
I can see Disable Copy-on-Write (= nocsum?).
I used dd now for testing seq speed
dd if=/dev/zero of=asd bs=1M count=2024 conv=fdatasync,notrunc
With cow disabled the speed is more or less the same. With cow enabled it is about 30-40mb/s slower.
@theChaosCoder what about disabling lzo but leaving cow? lzo makes btrfs be more cpu dependent and takes speed away (how much depends on your cpu)
SSD - WinBtrfs 1.0.1 (Debug) @maharmstone Could the debug version of winbtrfs affect the write rate significantly?
actually the compression should make the /dev/zero test much FASTER, as zeroes compress extremely well
Could the debug version of winbtrfs affect the write rate significantly? - idk, but it could definitely have a significant impact on the compression
@maharmstone are you sure its called nocsum ? because in linux its called nodatasum
@theChaosCoder Cow seems to have a big impact on write performance The native btrfs of the Linux kernel may perform better? May NTFS compression be worse than Btrfs's lzo performance? CrystalDiskMark may be able to work on Linux mono
According to this benchmark https://quixdb.github.io/squash-benchmark/ the ms-compress:LZNT1 is indeed slower then lzo. Since windows8.1 you can compress files also with xpress and lzx. But I couldn't find any usefull benchmarks.
Wimlib implemented both of them https://wimlib.net/compression.html
@theChaosCoder It's interesting! I saw lz4 with amazing compression speed and pressure! In terms of revenue, this is far more applicable to transparent compression of file systems than Lzo.
I saw lz4 with amazing compression speed and pressure! In terms of revenue, this is far more applicable to transparent compression of file systems than Lzo.
But they don't seem to care lz4😥
Btrfs seems to be showing poor 4K write performance. Is this perhaps a reason for non-native support?
It would be also cool to test how performance compares with the "original" linux.
I was just testing git status (using cygwin git), or a pretty large repo:
NTFS: git status -s 4.40s user 23.98s system 257% cpu 11.034 total
BTRFS 1.1: git status -s 6.84s user 417.06s system 104% cpu 6:45.84 total
So approximately 40x slower. BTRFS seems to be highly CPU limited. The git process has many threads all in sync primitives, for example:
0 ntoskrnl.exe!KeSynchronizeExecution+0x5be6
1 ntoskrnl.exe!KeWaitForSingleObject+0x12e6
2 ntoskrnl.exe!KeWaitForSingleObject+0xadb
3 ntdll.dll!ZwClose+0x14
4 KERNELBASE.dll!CloseHandle+0x62
5 cygwin1.dll!setpassent+0xa87
6 cygwin1.dll!umask+0xf97
7 cygwin1.dll!acl_get_perm+0x930a
or
0 ntoskrnl.exe!KeSynchronizeExecution+0x5be6
1 ntoskrnl.exe!KeWaitForSingleObject+0x12e6
2 ntoskrnl.exe!KeWaitForSingleObject+0xadb
3 ntoskrnl.exe!KeWaitForSingleObject+0x1ff
4 ntoskrnl.exe!IofCompleteRequest+0x748
5 ntoskrnl.exe!KeLeaveCriticalRegion+0x37
6 btrfs.sys+0xa502
7 ntoskrnl.exe!IofCallDriver+0x59
8 FLTMGR.SYS!FltIsCallbackDataDirty+0x2e7
9 FLTMGR.SYS!FltDecodeParameters+0x406
10 ntoskrnl.exe!IofCallDriver+0x59
11 cbfs6.sys+0x2e3b
12 cbfs6.sys+0x10da
13 ntoskrnl.exe!IofCallDriver+0x59
14 ntoskrnl.exe!SeQueryInformationToken+0x2e14
15 ntoskrnl.exe!NtDeviceIoControlFile+0x1e40
16 ntoskrnl.exe!ObfDereferenceObjectWithTag+0xc6
17 ntoskrnl.exe!SeLockSubjectContext+0x899
18 ntoskrnl.exe!NtClose+0xcd
19 ntoskrnl.exe!setjmpex+0x6f83
20 ntdll.dll!ZwClose+0x14
21 KERNELBASE.dll!CloseHandle+0x62
22 cygwin1.dll!setpassent+0xa87
23 cygwin1.dll!umask+0xf97
24 cygwin1.dll!acl_get_perm+0x930a
I haven't yet dug into the code enough to understand why this would be so much worse than NTFS, but it's bad enough that I'll have to at least investigate before using it for day-to-day work.
That cbfs6.sys looked suspicious to me, so I disabled it. I'm not sure how I ended up with it installed.
The status time was reduced a lot: git status -s 4.48s user 64.51s system 121% cpu 56.931 total. Now it's only 6x slower. There was no noticeable effect on NTFS.
On NTFS, git uses ~22% of a 16-thread ryzen CPU, but on BTRFS it uses ~7% (one core). I'm still getting similar call stacks:
ntoskrnl.exe!KeSynchronizeExecution+0x5be6
ntoskrnl.exe!KeWaitForSingleObject+0x12e6
ntoskrnl.exe!KeWaitForSingleObject+0xadb
ntoskrnl.exe!KeWaitForSingleObject+0x1ff
ntoskrnl.exe!KeCheckProcessorGroupAffinity+0x98a
ntoskrnl.exe!ExIsResourceAcquiredSharedLite+0x7cb
ntoskrnl.exe!KiCheckForKernelApcDelivery+0x27
ntoskrnl.exe!KeLeaveCriticalRegion+0x37
btrfs.sys+0xa502
ntoskrnl.exe!IofCallDriver+0x59
FLTMGR.SYS!FltIsCallbackDataDirty+0x2e7
FLTMGR.SYS!FltDecodeParameters+0x406
ntoskrnl.exe!IofCallDriver+0x59
ntoskrnl.exe!SeQueryInformationToken+0x2e14
ntoskrnl.exe!NtDeviceIoControlFile+0x1e40
ntoskrnl.exe!ObfDereferenceObjectWithTag+0xc6
ntoskrnl.exe!SeLockSubjectContext+0x899
ntoskrnl.exe!NtClose+0xcd
ntoskrnl.exe!setjmpex+0x6f83
ntdll.dll!ZwClose+0x14
KERNELBASE.dll!CloseHandle+0x62
cygwin1.dll!setpassent+0xa87
cygwin1.dll!umask+0xf97
cygwin1.dll!acl_get_perm+0x930a
Thanks David. btrfs.sys+0xa502 is the call to FsRtlExitFileSystem at the end of drv_close, which makes sense given the call stack.
One characteristic of Git is that it has lots of small files... my hunch is that you have a lot of data stored as inline files, and it's choking on the per-subvolume metadata lock. I'll give it a proper look when I get the chance.
Yeah, I'm running git status on a clean working tree with 174k files. So it should just be a ton of metadata reads.
Could you point me at the lock you're describing? I'm guessing somewhere under drv_create?
Well, it turned out to be a whole number of things. It's nowhere near done, and it still needs a lot of testing, but on my machine the most recent patches bring git status on the Linux kernel down from over 15 minutes to under 8 seconds. That's the same as doing the same thing on Linux, so it's probably more or less optimal speed-wise.
Wow, that's awesome. I'll try to find some time to build master (I assume this is the stuff you pushed recently) and run my benchmarks again.
Thanks.
That sounds amazing, thank you for your work!
Have you also tested how sequential performance with relatively few files was affected?
@maharmstone I gave it a try. I was able to create a new volume, subvolume, directory, snapshot, but the first time I tried to write something to a file I got a crash:
FYI I built it with VS2017, so this may not be relevant to you.
*******************************************************************************
* *
* Bugcheck Analysis *
* *
*******************************************************************************
KERNEL_SECURITY_CHECK_FAILURE (139)
A kernel component has corrupted a critical data structure. The corruption
could potentially allow a malicious user to gain control of this machine.
Arguments:
Arg1: 0000000000000003, A LIST_ENTRY has been corrupted (i.e. double remove).
Arg2: ffff94855b0ed940, Address of the trap frame for the exception that caused the bugcheck
Arg3: ffff94855b0ed898, Address of the exception record for the exception that caused the bugcheck
Arg4: 0000000000000000, Reserved
Debugging Details:
------------------
KEY_VALUES_STRING: 1
STACKHASH_ANALYSIS: 1
TIMELINE_ANALYSIS: 1
DUMP_CLASS: 1
DUMP_QUALIFIER: 401
BUILD_VERSION_STRING: 17134.1.amd64fre.rs4_release.180410-1804
SYSTEM_MANUFACTURER: MSI
SYSTEM_PRODUCT_NAME: MS-7A33
SYSTEM_SKU: To be filled by O.E.M.
SYSTEM_VERSION: 3.0
BIOS_VENDOR: American Megatrends Inc.
BIOS_VERSION: 5.50
BIOS_DATE: 09/20/2017
BASEBOARD_MANUFACTURER: MSI
BASEBOARD_PRODUCT: X370 GAMING PLUS (MS-7A33)
BASEBOARD_VERSION: 3.0
DUMP_TYPE: 1
BUGCHECK_P1: 3
BUGCHECK_P2: ffff94855b0ed940
BUGCHECK_P3: ffff94855b0ed898
BUGCHECK_P4: 0
TRAP_FRAME: ffff94855b0ed940 -- (.trap 0xffff94855b0ed940)
NOTE: The trap frame does not contain all registers.
Some register values may be zeroed or incorrect.
rax=ffffbf07850ba2e8 rbx=0000000000000000 rcx=0000000000000003
rdx=ffffbf0784f89220 rsi=0000000000000000 rdi=0000000000000000
rip=fffff805b5500194 rsp=ffff94855b0edad0 rbp=0000000000000080
r8=ffffbf0784cda7a8 r9=000000000000001b r10=ffffe303c99024c0
r11=ffffbf0784fa5180 r12=0000000000000000 r13=0000000000000000
r14=0000000000000000 r15=0000000000000000
iopl=0 nv up ei pl nz ac pe cy
btrfs!RtlFailFast+0x5 [inlined in btrfs!reap_fcb+0x374]:
fffff805`b5500194 cd29 int 29h
Resetting default scope
EXCEPTION_RECORD: ffff94855b0ed898 -- (.exr 0xffff94855b0ed898)
ExceptionAddress: fffff805b5500194 (btrfs!RtlFailFast+0x0000000000000005)
ExceptionCode: c0000409 (Security check failure or stack buffer overrun)
ExceptionFlags: 00000001
NumberParameters: 1
Parameter[0]: 0000000000000003
Subcode: 0x3 FAST_FAIL_CORRUPT_LIST_ENTRY
CPU_COUNT: 10
CPU_MHZ: bb8
CPU_VENDOR: AuthenticAMD
CPU_FAMILY: 17
CPU_MODEL: 1
CPU_STEPPING: 1
BLACKBOXBSD: 1 (!blackboxbsd)
BLACKBOXPNP: 1 (!blackboxpnp)
BUGCHECK_STR: 0x139
PROCESS_NAME: System
CURRENT_IRQL: 0
DEFAULT_BUCKET_ID: FAIL_FAST_CORRUPT_LIST_ENTRY
ERROR_CODE: (NTSTATUS) 0xc0000409 - The system detected an overrun of a stack-based buffer in this application. This overrun could potentially allow a malicious user to gain control of this application.
EXCEPTION_CODE: (NTSTATUS) 0xc0000409 - The system detected an overrun of a stack-based buffer in this application. This overrun could potentially allow a malicious user to gain control of this application.
EXCEPTION_CODE_STR: c0000409
EXCEPTION_PARAMETER1: 0000000000000003
ANALYSIS_SESSION_HOST: DAVIDM
ANALYSIS_SESSION_TIME: 04-15-2019 10:44:02.0736
ANALYSIS_VERSION: 10.0.17763.1 amd64fre
LAST_CONTROL_TRANSFER: from fffff80354a63f69 to fffff80354a53380
STACK_TEXT:
ffff9485`5b0ed618 fffff803`54a63f69 : 00000000`00000139 00000000`00000003 ffff9485`5b0ed940 ffff9485`5b0ed898 : nt!KeBugCheckEx
ffff9485`5b0ed620 fffff803`54a64310 : 00000000`00000000 00000000`00000000 ffff3ef1`fccc8d42 00000000`00000000 : nt!KiBugCheckDispatch+0x69
ffff9485`5b0ed760 fffff803`54a6291f : 00000000`00000000 ffffe303`c9799840 ffff9680`4e11c140 fffff803`54b942d4 : nt!KiFastFailDispatch+0xd0
ffff9485`5b0ed940 fffff805`b5500194 : ffffbf07`84fa5358 ffffbf07`84fa53b0 00000000`00000080 ffffbf07`84fa53c0 : nt!KiRaiseSecurityCheckFailure+0x2df
ffff9485`5b0edad0 fffff805`b5500225 : ffffbf07`850ba2f8 00000000`00000000 ffffbf07`845f45a8 fffff803`548dc56a : btrfs!reap_fcb+0x374 [c:\users\davidm\src\btrfs\src\btrfs.c @ 1539]
ffff9485`5b0edb00 fffff805`b55595a4 : ffffe303`c9901498 00000000`00000080 00000000`00000000 ffffe303`c9901ed8 : btrfs!reap_fcbs+0x35 [c:\users\davidm\src\btrfs\src\btrfs.c @ 1566]
ffff9485`5b0edb30 fffff805`b552231c : ffffe303`c9900180 ffffffff`00000000 ffffe303`c9901498 00000000`00000000 : btrfs!free_trees+0x294 [c:\users\davidm\src\btrfs\src\treefuncs.c @ 830]
ffff9485`5b0edbb0 fffff803`5491d4f7 : 00000000`00000080 fffff805`b55221e0 ffffe303`c9799700 ffffe303`c9900030 : btrfs!flush_thread+0x13c [c:\users\davidm\src\btrfs\src\flushthread.c @ 7417]
ffff9485`5b0edc10 fffff803`54a5a7f6 : ffff9680`4eb48180 ffffe303`c9799700 fffff803`5491d4b0 00000000`00000000 : nt!PspSystemThreadStartup+0x47
ffff9485`5b0edc60 00000000`00000000 : ffff9485`5b0ee000 ffff9485`5b0e8000 00000000`00000000 00000000`00000000 : nt!KiStartSystemThread+0x16
THREAD_SHA1_HASH_MOD_FUNC: 818543b709ef8d2737be9d90f2fa14bbad26c6c9
THREAD_SHA1_HASH_MOD_FUNC_OFFSET: 6344d02533c08a46dd2133fb140b0c34a3378718
THREAD_SHA1_HASH_MOD: ed006da84d64fd1acd771fbd8008f29200185557
FOLLOWUP_IP:
btrfs!reap_fcb+374 [c:\users\davidm\src\btrfs\src\btrfs.c @ 1539]
fffff805`b5500194 cd29 int 29h
FAULT_INSTR_CODE: 8b4829cd
FAULTING_SOURCE_LINE: c:\users\davidm\src\btrfs\src\btrfs.c
FAULTING_SOURCE_FILE: c:\users\davidm\src\btrfs\src\btrfs.c
FAULTING_SOURCE_LINE_NUMBER: 1539
FAULTING_SOURCE_CODE:
11060: )
11061:
11062: {
11063:
>11064: __fastfail(Code);
11065: }
11066:
11067: #endif // _MSC_VER
11068:
11069: //
SYMBOL_STACK_INDEX: 4
SYMBOL_NAME: btrfs!reap_fcb+374
FOLLOWUP_NAME: MachineOwner
MODULE_NAME: btrfs
IMAGE_NAME: btrfs.sys
DEBUG_FLR_IMAGE_TIMESTAMP: 5cb480a6
STACK_COMMAND: .thread ; .cxr ; kb
BUCKET_ID_FUNC_OFFSET: 374
FAILURE_BUCKET_ID: 0x139_3_CORRUPT_LIST_ENTRY_btrfs!reap_fcb
BUCKET_ID: 0x139_3_CORRUPT_LIST_ENTRY_btrfs!reap_fcb
PRIMARY_PROBLEM_CLASS: 0x139_3_CORRUPT_LIST_ENTRY_btrfs!reap_fcb
TARGET_TIME: 2019-04-15T13:37:01.000Z
OSBUILD: 17134
OSSERVICEPACK: 0
SERVICEPACK_NUMBER: 0
OS_REVISION: 0
SUITE_MASK: 272
PRODUCT_TYPE: 1
OSPLATFORM_TYPE: x64
OSNAME: Windows 10
OSEDITION: Windows 10 WinNt TerminalServer SingleUserTS
OS_LOCALE:
USER_LCID: 0
OSBUILD_TIMESTAMP: 2018-09-14 23:18:09
BUILDDATESTAMP_STR: 180410-1804
BUILDLAB_STR: rs4_release
BUILDOSVER_STR: 10.0.17134.1.amd64fre.rs4_release.180410-1804
ANALYSIS_SESSION_ELAPSED_TIME: 818
ANALYSIS_SOURCE: KM
FAILURE_ID_HASH_STRING: km:0x139_3_corrupt_list_entry_btrfs!reap_fcb
FAILURE_ID_HASH: {e9082920-3c50-4ffc-06c2-baea55bd1ea1}
Followup: MachineOwner
---------
NTFS
CrystalDiskMark 6.0.2 x64 (C) 2007-2018 hiyohiyo Crystal Dew World : https://crystalmark.info/
-
MB/s = 1,000,000 bytes/s [SATA/600 = 600,000,000 bytes/s]
-
KB = 1000 bytes, KiB = 1024 bytes
Sequential Read (Q= 32,T= 1) : 2815.107 MB/s Sequential Write (Q= 32,T= 1) : 1615.286 MB/s Random Read 4KiB (Q= 8,T= 8) : 1231.131 MB/s [ 300569.1 IOPS] Random Write 4KiB (Q= 8,T= 8) : 970.043 MB/s [ 236826.9 IOPS] Random Read 4KiB (Q= 32,T= 1) : 439.156 MB/s [ 107215.8 IOPS] Random Write 4KiB (Q= 32,T= 1) : 225.975 MB/s [ 55169.7 IOPS] Random Read 4KiB (Q= 1,T= 1) : 44.427 MB/s [ 10846.4 IOPS] Random Write 4KiB (Q= 1,T= 1) : 110.803 MB/s [ 27051.5 IOPS]
Test : 500 MiB [C: 58.7% (55.4/94.4 GiB)] (x1) [Interval=5 sec] Date : 2020/03/28 23:28:04 OS : Windows 10 Enterprise [10.0 Build 18363] (x64)
BTRFS
CrystalDiskMark 6.0.2 x64 (C) 2007-2018 hiyohiyo Crystal Dew World : https://crystalmark.info/
-
MB/s = 1,000,000 bytes/s [SATA/600 = 600,000,000 bytes/s]
-
KB = 1000 bytes, KiB = 1024 bytes
Sequential Read (Q= 32,T= 1) : 2302.475 MB/s Sequential Write (Q= 32,T= 1) : 130.982 MB/s Random Read 4KiB (Q= 8,T= 8) : 727.899 MB/s [ 177709.7 IOPS] Random Write 4KiB (Q= 8,T= 8) : 1.350 MB/s [ 329.6 IOPS] Random Read 4KiB (Q= 32,T= 1) : 399.801 MB/s [ 97607.7 IOPS] Random Write 4KiB (Q= 32,T= 1) : 2.295 MB/s [ 560.3 IOPS] Random Read 4KiB (Q= 1,T= 1) : 40.229 MB/s [ 9821.5 IOPS] Random Write 4KiB (Q= 1,T= 1) : 1.204 MB/s [ 293.9 IOPS]
Test : 500 MiB [F: 26.5% (206.7/781.3 GiB)] (x1) [Interval=5 sec] Date : 2020/03/28 23:30:32 OS : Windows 10 Enterprise [10.0 Build 18363] (x64)
Same drive two different partitions, may be unfair. But 30-99% slower? There are much more serious issues than speed however. This is with default settings.
XPG SX8200 1tb i76700k 2900mhz ram
Thanks for this. I've not done any rigorous testing yet, but like you I found that write speeds on Btrfs were worse than read speeds, whereas on NTFS they are about the same.
The difference in your read speeds is probably due to the checksumming that Btrfs does.
Having had a look at what it's doing, I'm not sure that the figures for random writes tell you anything meaningful at all - certainly it's not indicative of a bug that needs to be fixed. It's slow because it has to traverse a list of thousands of extents, whereas NTFS does everything in place.
The only time you're likely to encounter random writes in practice is with databases and VM images, and the official advice for these has always been to make sure they're marked nocow.
Different software, different run, much faster but innacurate[?] disks wouldn't let me run a write test while the drive was active...a Fio script I was recommended gave errors creating a tmp dir to benchmark... But this read speed is much more accurate to the device's specifications than in windows.
Adding to the performance testimonies, here are some crystaldiskmark results on a 960 EVO NVMe with 2 separate partitions.
NTFS
------------------------------------------------------------------------------
CrystalDiskMark 8.0.1 x64 (C) 2007-2021 hiyohiyo
Crystal Dew World: https://crystalmark.info/
------------------------------------------------------------------------------
* MB/s = 1,000,000 bytes/s [SATA/600 = 600,000,000 bytes/s]
* KB = 1000 bytes, KiB = 1024 bytes
[Read]
SEQ 1MiB (Q= 8, T= 1): 2375.063 MB/s [ 2265.0 IOPS] < 3447.45 us>
SEQ 1MiB (Q= 1, T= 1): 1388.444 MB/s [ 1324.1 IOPS] < 754.82 us>
RND 4KiB (Q= 32, T= 1): 620.412 MB/s [ 151467.8 IOPS] < 204.52 us>
RND 4KiB (Q= 1, T= 1): 46.323 MB/s [ 11309.3 IOPS] < 88.29 us>
[Write]
SEQ 1MiB (Q= 8, T= 1): 1510.362 MB/s [ 1440.4 IOPS] < 5539.39 us>
SEQ 1MiB (Q= 1, T= 1): 1507.275 MB/s [ 1437.4 IOPS] < 695.16 us>
RND 4KiB (Q= 32, T= 1): 525.530 MB/s [ 128303.2 IOPS] < 242.68 us>
RND 4KiB (Q= 1, T= 1): 203.812 MB/s [ 49758.8 IOPS] < 20.00 us>
Profile: Default
Test: 1 GiB (x5) [C: 67% (86/127GiB)]
Mode: [Admin]
Time: Measure 5 sec / Interval 5 sec
Date: 2021/02/03 19:26:04
OS: Windows 10 [10.0 Build 19041] (x64)
BTRFS (xxHash, no compression)
------------------------------------------------------------------------------
CrystalDiskMark 8.0.1 x64 (C) 2007-2021 hiyohiyo
Crystal Dew World: https://crystalmark.info/
------------------------------------------------------------------------------
* MB/s = 1,000,000 bytes/s [SATA/600 = 600,000,000 bytes/s]
* KB = 1000 bytes, KiB = 1024 bytes
[Read]
SEQ 1MiB (Q= 8, T= 1): 2030.902 MB/s [ 1936.8 IOPS] < 4120.17 us>
SEQ 1MiB (Q= 1, T= 1): 852.324 MB/s [ 812.8 IOPS] < 1229.54 us>
RND 4KiB (Q= 32, T= 1): 772.471 MB/s [ 188591.6 IOPS] < 164.25 us>
RND 4KiB (Q= 1, T= 1): 41.672 MB/s [ 10173.8 IOPS] < 98.17 us>
[Write]
SEQ 1MiB (Q= 8, T= 1): 697.329 MB/s [ 665.0 IOPS] < 11957.07 us>
SEQ 1MiB (Q= 1, T= 1): 341.835 MB/s [ 326.0 IOPS] < 3064.89 us>
RND 4KiB (Q= 32, T= 1): 4.059 MB/s [ 991.0 IOPS] < 32151.50 us>
RND 4KiB (Q= 1, T= 1): 5.613 MB/s [ 1370.4 IOPS] < 729.17 us>
Profile: Default
Test: 1 GiB (x5) [E: 64% (68/105GiB)]
Mode: [Admin]
Time: Measure 5 sec / Interval 5 sec
Date: 2021/02/03 19:32:40
OS: Windows 10 [10.0 Build 19041] (x64)
BTRFS (xxHash, zstd:3 compression)
------------------------------------------------------------------------------
CrystalDiskMark 8.0.1 x64 (C) 2007-2021 hiyohiyo
Crystal Dew World: https://crystalmark.info/
------------------------------------------------------------------------------
* MB/s = 1,000,000 bytes/s [SATA/600 = 600,000,000 bytes/s]
* KB = 1000 bytes, KiB = 1024 bytes
[Read]
SEQ 1MiB (Q= 8, T= 1): 2024.762 MB/s [ 1931.0 IOPS] < 4133.89 us>
SEQ 1MiB (Q= 1, T= 1): 992.458 MB/s [ 946.5 IOPS] < 1056.03 us>
RND 4KiB (Q= 32, T= 1): 758.915 MB/s [ 185282.0 IOPS] < 167.19 us>
RND 4KiB (Q= 1, T= 1): 41.257 MB/s [ 10072.5 IOPS] < 99.15 us>
[Write]
SEQ 1MiB (Q= 8, T= 1): 230.245 MB/s [ 219.6 IOPS] < 36239.66 us>
SEQ 1MiB (Q= 1, T= 1): 338.777 MB/s [ 323.1 IOPS] < 3093.65 us>
RND 4KiB (Q= 32, T= 1): 5.213 MB/s [ 1272.7 IOPS] < 25038.44 us>
RND 4KiB (Q= 1, T= 1): 5.735 MB/s [ 1400.1 IOPS] < 713.74 us>
Profile: Default
Test: 1 GiB (x5) [L: 64% (68/105GiB)]
Mode: [Admin]
Time: Measure 5 sec / Interval 5 sec
Date: 2021/02/03 19:46:45
OS: Windows 10 [10.0 Build 19041] (x64)
Read speeds are still lower overall, but very usable, write speeds however are somewhat of a letdown, and compression totally destroys performance, but that was to be expected.
All in all those are trade-offs that I'm willing to make since my workload of choice will be gaming so this shouldn't matter much. My steam library sits on a ntfs volume striped across 2 SATA SSDs, mounting this on linux requires libldm and I'd like to get rid of it, I'll give raid0 btrfs a try and see how that goes.
Does that benchmark take into account the fact that compressed data results in more "raw" (uncompressed) data wrote/read ?
Paragon's ntfs driver doesn't require any extra dependencies, on top of possibly being even faster than ext4.
Does that benchmark take into account the fact that compressed data results in more "raw" (uncompressed) data wrote/read ?
That is a good question I'm afraid I can't answer. Will see how real world workloads fare, that's what matters after all.
Paragon's ntfs driver doesn't require any extra dependencies, on top of possibly being even faster than ext4.
The whole point is to get rid of NTFS and use a superior filesystem (checksum, native raid, better transparent compression). Speed is hardly the deciding factor for me, and I'd rather have windows lag behind than having linux being limited by windows. Plus the paragon driver is still relatively new and hasn't landed in the kernel yet, so there's that.
That is a good question I'm afraid I can't answer. Will see how real world workloads fare, that's what matters after all.
Maybe if you can measure the time to open/write a very big file (set) (be cautious about caching and so on, it can bias the result, a lot) you can have an estimation 🤔
I'm asking because I'm surprised that compression really destroy the performances, on my setup when I compared btrfs with and without compression (on linux) I saw no impact of compression on read and write speed (but thanks to the compression the "real" speed was better).