Re: Low NTFS read performance

Diomidis Spinellis wrote in freebsd-stable@:
I can't get an Ultrium-2 LTO drive to stream, and (I think) I've traced the problem to the read performance of the USB2-attached NTFS disk, and specifically the NTFS filesystem. I'm reading a single 190GB file, and the throughput I'm getting is 5.4MB/s:

$ dd if=ad2c.dump of=/dev/null bs=1M
^Τ
load: 0.04 cmd: dd 1434 [biord] 0.00u 4.78s 2% 1672k
610+0 records in
610+0 records out
639631360 bytes transferred in 117.937613 secs (5423472 bytes/sec)

The is an old but relatively fast machine

CPU: Intel(R) Pentium(R) 4 CPU 2.40GHz (2407.18-MHz 686-class CPU)

running 7.0-RC1:

$ uname -a
FreeBSD icarian.dmst.aueb.gr 7.0-RC1 FreeBSD 7.0-RC1 #0: Mon Dec 24 12:18:24 UTC 2007 root@xxxxxxxxxxxxxxxxxxxxx:/usr/obj/usr/src/sys/GENERIC i386

and the load during the read remains comfortably low:

$ uptime
6:34PM up 3:50, 3 users, load averages: 0.09, 0.04, 0.04

Reading from the raw device tripples the performance:

# dd if=/dev/da0s1 bs=1m of=/dev/null
533725184 bytes transferred in 34.777460 secs (15346871 bytes/sec)

bringing it on par with what I get from Windows (on a different machine):

F:\>dd if=other.20051007.tgz of=/dev/null bs=1M
1231030919 bytes (1.2 GB) copied, 82.845 s, 14.9 MB/s

[...]

I'd appreciate any suggestions you may have.

Today I upgraded the machine to 8.0-CURRENT, and obtained similar results:
- 5372041 bytes/sec reading an NTFS file
- 15369758 bytes/sec reading from the raw device

The raw/FS read performance difference with UFS is a lot lower:
- 33162654 bytes/sec reading a UFS file
- 47221133 bytes/sec reading from the raw device

I'd welcome any ideas on why the NTFS layer makes reading three time slower than reading from the raw device and suggestions for improving the performance. I'd also be happy to work on improving the situation, if I get any pointers on where to start.

Diomidis Spinellis - http://www.spinellis.gr

_______________________________________________
freebsd-current@xxxxxxxxxxx mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscribe@xxxxxxxxxxx"

Relevant Pages

  • Re: Bad clusters on Harddrive?
    ... If not NTFS, you can use DOS mode Scandisk and surface scan. ... "fix" bad sectors (thus hiding impending failure from you). ... on - by reading SMART's logs of what has been "fixed" at the HD ... rather than looking only at SMART logs. ...
    (microsoft.public.windowsxp.hardware)
  • Re: Bad clusters on Harddrive?
    ... If not NTFS, you can use DOS mode Scandisk and surface scan. ... "fix" bad sectors (thus hiding impending failure from you). ... on - by reading SMART's logs of what has been "fixed" at the HD ... rather than looking only at SMART logs. ...
    (microsoft.public.windowsxp.perform_maintain)
  • Re: Maxtor 6B200P0 lost file system
    ... Pro, ... I disconnect the USB and shut down both the HD and the PC ... the PC can see the drive, recognise the model (probably reading the HD ... the HD lost the file system, no NTFS anymore.. ...
    (microsoft.public.windowsxp.perform_maintain)
  • Re: Bad XP problem
    ... I have couple more options and some reading to do. ... The next time I build a custom high end PC for someone I will be charging quite ... would doing this on a clone cause NTFS put a stop to it? ...
    (sci.electronics.repair)
  • Re: Low NTFS read performance
    ... - 5372041 bytes/sec reading an NTFS file ... 15369758 bytes/sec reading from the raw device ... I'd welcome any ideas on why the NTFS layer makes reading three time slower than reading from the raw device and suggestions for improving the performance. ...
    (freebsd-current)