Software RAID5 archived ~37 MB/sec resync speed with two pieces of 3ware 9550SX-8LP

# mdadm -Cv /dev/md7 -l5 -n15 -x1 -c128 /dev/sd[c-r]1
Personalities : [raid1] [raid5]
md7 : active raid5 sdq1[15] sdr1[16](S) sdp1[13] sdo1[12] sdn1[11] sdm1[10] sdl1[9] sdk1[8] sdj1[7] sdi1[6] sdh1[5] sdg1[4] sdf1[3] sde1[2] sdd1[1] sdc1[0]
4102468608 blocks level 5, 128k chunk, algorithm 2 [15/14] [UUUUUUUUUUUUUU_]
[>....................] recovery = 0.6% (1908296/293033472) finish=131.1min speed=36998K/sec

3ware 9000 Storage Controller device driver for Linux v2.26.04.006.
Firmware FE9X 3.01.01.028, BIOS BE9X 3.01.00.024.

Setting 'max_queue' options for sx8.ko increases RAID5 recovery speed to ~23 MB/sec

Unload sx8 module. Add
options sx8 max_queue=24
to /etc/modules.conf and reload sx8. Now the recovery speed with a single adaptor and 8 drives is increased to 23000K/sec.

The effect of max_queue=32 has not much difference but in the comment of sx8.c version 1.0 it is claimed that problems seem to occur when the value exceeds ~30.

Thanks to Jeff Garzik's ultra-prompt reply in five minutes.

Linux software RAID5 using sx8.c driver = only ~9000 K/sec recovery speed......

Kernel 2.6.14.2 SMP, sx8.c version 1.0

mdadm -Cv /dev/md7 -l5 -n7 -x1 /dev/sx8/[0-7]p1 -c 128



And if two SX8 adapters are used, raid5 recovery speed dropped under 500 K/sec if more than 8 drives are included.

A fileserver of 4.2 TB in size drowned Zippy 650W power supply

The specification of fileserver:

• Supermicro PDSME Motherboard
  
• Intel Pentium-4 820 Dual Core / Dual 1MB L2 cache / 800 MHz FSB
• 4 x 1GB DDR2-533 (Transcend TS128MLQ64V5J)
• 2 x Promise SATAII-150SX8 Controller (8 SATA ports each)
  
• 16 x 3.5" Seagate 7200rpm 300GB SATA HDD with 8MB cache (ST3300831AS)
• 2 x 2.5" Seagate 5400rpm 80GB SATA HDD with 8MB cache (ST98823AS)
• 1 x Intel Pro/1000MT Dual Port Gigabit Controller (PWLA8492MT)
• 1 x Intel Pro/100M Single Port FastEthernet Controller (PILA8460M)

The 650W power supply (P2L-6650P, +12V=34A, +5V=40A) chocked with only the above four items in red installed. I had to detatch 4 HDs to make the CPU fan spin.

Changing to one of the following is under consideration:

• P2M6600P(600W) +12V=42A +5V=35A
• P2M6601P(600W) +12V=48A +5V=30A
• H3M6600P(600W) +12V=48A +5V=30A

NWChem 4.7 out of memory/swap problem with GOTO 1.0 in EM64T

A simple DFT task of 480 basis functions failed due to "out of memory". All of the system memory and swap space were occupied (3GB RAM, 15GB swap and counting......) no matter what number had been set to "memory total" keyword. Switching linking BLAS library from libgoto_prescott64p-r1.00.so to MKL 7.2c solved the problem, and this task actually only require ~60MB of RAM.

Grand opening on Oct 25, 2005.
Ok, as you may have guessed, this is a test.