|
![]() July 20, 2007 12 Bay Infiniband Multilane SATA Enclosure A Review of the Norco DS-1240 High Performance Triple Multilane Interface Enclosure By Arthur Whalem ![]() NORCO is now shipping the NORCO DS-1240 12 Bay Infiniband Multilane SATA Hard Drive Storage Array ($639.99). This 12-bay 3U SATA direct connect hard drive storage subsystem features twelve hot swap SATA hard drive bays enclosed in a black heavy duty rack mountable 1.2mm steel case. The enclosure utilizes three Infiniband multilane connections on the rear of the case to support all twelve hard drive bays. The DS-1240 supports both SATA I and SATA II hard drives. The enclosure measures 19.75" deep, 5.9" high and 19" wide. The enclosure weighs 32 pounds empty and approximately 50 pounds when twelve SATA hard drives are installed. The Norco DS-1240 is compatible with Macintosh, Windows and Linux computers that have a compatible SATA host adapter installed. The purpose of this review is to determine how the NORCO DS-1240 will perform with an Apple Mac Pro.System Requirements Supports Mac OS X 10.4.x or higher, Windows and Linux. Supports up to 12 external SATA 3.5" hard drives. Utilizes three Infiniband multilane external connectors. Optional NORCO-4154 Multilane to 4 SATA adapter is available. What's Included? The Norco DS-1240 includes a 3U rack mountable case, twelve 3.5" hard drive trays, drive mounting screws, a small screwdriver, power cord and a users guide. ![]() Rear Connections The Norco DS-1240 has a power cord outlet on the right rear side of the case with a power on/off switch just below it. A power standby switch is provided in the top right corner. Both switches must be on to power hard drives mounted inside the enclosure. On the bottom left side, three Infiniband multilane connectors are mounted below three 70mm case fans which keep the hard drives cool. The Infiniband1 data port is configured with bays 9-12. The Infiniband2 data port is connected to bays 5-8 and the Infiniband3 data port works with hard drives mounted in bays 1-4. ![]() Interface The Norco DS-1240 utilizes a circuit board on the rear of the drive tray. To mount a SATA hard drive into the drive tray, users will need to slide the SATA hard drive data and power connections onto the rear circuit board. Placing the hard drive into the tray opening and pulling back on it will snap the connectors in place. Next, use two screws on each side of the tray to secure the hard drive. The screws need to be tightened sufficiently so that the head of the screw is flush or below the blue plastic side panels. This is required to allow the tray to easily slide into the enclosure. ![]() The stock DS-1240 screws have a large head that may occasionally interfere with SATA hot swap tray clearance. The clearance issue caused me to tighten the stock screws a little too tight. When I removed the hard drives two of the NORCO screw heads broke off leaving them in the disk drives. As you can imagine getting the screw shaft out of the hard drive is no easy task. To avoid this problem, I substituted the stock screws with a screw design that has a rounded head and a slightly shorter shaft. This alternative screw system provides better clearance for the hard drive trays and over tightening is no longer a concern. The screws that come with the DS-1240 will work in most cases. However, if tray clearance becomes a problem do not over tighten the screws. Once the hard drives are mounted in the Norco DS-1240 drive trays users can insert them into the twelve available hot swap bays. To insert a Norco DS-1240 tray into the enclosure, simply push it all the way in while lining it up on the right side and then push in the blue tray tab. The drive tray will lock into place with a gentle push. ![]() To eject a drive tray from the Norco DS-1240 enclosure, insert a small Phillips screwdriver (included) into the hole that is located below the blue locking tab of the drive tray. A slight clicking noise will be heard as the drive tray unlocks. Pulling on the blue tab will eject the drive tray. The screwdriver that is supplied with the Norco DS-1240 works well. However, I found that the WiebeTech SilverSATA white keys work great too. The WiebeTech white plastic key is a lot smaller than the screwdriver. Users can put a string or a chain through the hole in the back of the white plastic key and attach it to the handle of the Norco DS-1240. While the NORCO supplied screwdriver and the WiebeTech white keys work well for ejecting hard drives, users can also improvise and use a paper clip or any thin object that is about an inch long to eject the hard drive trays.![]() Lack of Activity Lights While the Norco DS-1220 twelve bay SATA PM enclosure includes activity lights, the Norco DS-1240 does not. The yellow and green tray lights turn on when a hard drive is inserted but they do not blink to indicate hard drive activity. This may be a problem when a hard drive failure occurs. With no hard drive activity lights JBOD users may have to eject all of the hard drives and re-insert them one at a time to identify a failed disk drive. In the case of a large RAID configuration, detection of the failed hard drive may be more difficult unless the user plans ahead. The HighPoint and the Areca web interfaces both provide a list of hard drives and identify which port they are mounted on. If during the initial setup the user matches the web interface port numbers with the DS-1240 marked bay numbers, discovery of a failed hard drive can be easily accomplished. Without advanced planning, identifying and replacing a failed hard drive within a RAID set may be difficult. ![]() Design The exterior design of the Norco DS-1240 provides the capability to utilize the unit in a U3 rack mount configuration. A nice black enamel finish is found on all sides of the enclosure and the heavy steel 1.2mm case construction provides strength and weight to the Norco DS-1240 design. Four 1" round rubber feet are mounted on the bottom of the enclosure to provide airflow, an easy way to get your fingers under the unit to pick it up and to protect the enclosures finish when sliding it into place. The blue plastic drive tray locking tabs add a quality appearance to the design of Norco DS-1240 along with the aluminum handles. Each drive bay is numbered for easy identification. Ventilation air slots are provided in the front of each drive tray to allow cool air to enter the enclosure as the three 70mm exhaust fans push air out of the rear of the enclosure. While the Norco DS-1240 is designed to be inserted into an equipment rack, I have found that it can also be placed on its side next to a PowerMac or a Mac Pro. For users that do not have an equipment rack to install the Norco DS-1240 into, this option is worth considering. Placing the unit on its side is not as stable as it would be when sitting horizontally on its feet, but AMUG had no problems using the DS-1240 in this position while 12 hard drives were mounted and working as fast as they could. If you mount the enclosure on its side, make sure the unit does not rock in that position. Placing a small cloth under the rear of the enclosure can help balance the weight. In my opinion, the Norco DS-1240 looks very nice sitting next to an Apple computer. ![]() Internal Design The top cover of the Norco DS-1240 comes off with two screws on the back, one screw on each side near the front and one screw in the front top section of the cover. Inside the Norco DS-1240, the front compartment houses twelve SATA hard drive bays. A steel cage holds the 16" long SATA backplane circuit board in place. In the rear of the enclosure twelve SATA bays are connected to three Infiniband multilane connectors. On the right rear side the power supply is mounted. ![]() The Norco DS-1240 provides a 360 watt power supply model DS-360 which is modified from a standard ATX power supply. The 12V and 5V outputs are enhanced to 22A and 19A. The 3.3V and other outputs are adjusted to be able to handle no load without shutting down the power supply. Five 4-pin molex connectors are provided from the power supply. Four of the power connectors plug into the backplane to provide power to the SATA hard drives and three 70mm exhaust fans. A fifth power adapter connector is available for future expansion inside the enclosure. Exposed Backplane View ![]() If all twelve hard drive trays are removed from the Norco DS-1240, the backplane mounting connectors that supply SATA data and power connections are visible. As you can see in the image above air ventilation is provided below the backplane and where holes are cut into the circuit board near the SATA power connections. ![]() The Areca ARC-1231ML is a great controller to utilize with the Norco DS-1240. It provides twelve ports with RAID levels 0, 1, 3, 5, 6, 10 or JBOD support. The ARC-1231ML features a PCIe eight lane interface, an 800MHz Intel IOP341 I/O processor, 256MB of on-board DDR2 memory (expandable to 2GB) and an optimized RAID engine to support extreme RAID 6 performance. It features online RAID capacity expansion, RAID level migration, hot swap, global online spare, automatic failed drive rebuilding and background RAID rebuilding. ![]() The Areca ARC-1231ML PCI-Express x8 SATA II Controller Card design uses internal Mini SAS SFF-8087 connectors. Three model iSAS-7370/1m one meter cables were utilized to connect the ARC-1231ML (SFF-8087) connectors to the three Infiniband multilane (SFF-8470) connectors on the DS-1240 enclosure. Once the ARC-1231ML is installed in slot 4 of the Mac Pro for 8x PCIe support, the cables can be attached to the card and run out of an open PCI slot. The internal Mini SAS SFF-8087 connector farthest inside the Mac Pro will be connected to the Infiniband3 labeled connector on the DS-1240. The middle Mini SAS SFF-8087 connector is attached to the Infiniband2 labeled connector on the DS-1240 and the closest Mini SAS SFF-8087 connector to the Mac Pro door is connected to the Infiniband1 port on the DS-1240. ![]() You might wonder why it matters which cable goes to which connector on the DS-1240? From a simple usage stand point it does not matter. However, this is the configuration that allowed the ARC-1231ML port numbers to match the Norco DS-1240 drive bay numbers. Users will not realize how important this is, until they have a hard disk failure in a RAID set and cannot figure out which hard disk is down, as the DS-1240 has no activity lights. Setting up the ports on the ARC-1231ML to match the numbered bays on the DS-1240 eliminates this problem. With port 1 assigned to bay 1, port 2 assigned to bay 2 and so on, the user simply looks at the Areca web interface to determine which drive failed and replace the hard drive in the corresponding bay number. Users should double check that this setup is working by setting up a test RAID and pulling hard drives. Make sure that when the hard drive in bay one is pulled, the web interface displays that the drive on port 1 has failed. Do this until all ports are confirmed to correspond with the proper hard drive bay. The lack of activity lights on Norco DS-1240 will not matter once the ARC-1231ML controller ports and the enclosure drive bay numbers are configured to match. Energy Usage According to the Kill-a-Watt electricity usage monitor, the Norco DS-1240 enclosure uses 3 watts when it is in standby mode. Standby mode is in effect when the enclosure is turned off but the bottom power supply switch is turned on. Turning both power switches off reduces the power usage to zero. Turning on both DS-1240 power switches with no hard drives installed uses 24 watts. Mounting a single Seagate 320GB model 7200.10 SATA hard drive inside the Norco DS-1240 increases the energy usage to 32 watts while idle. Once twelve Seagate 320GB SATA hard drives are mounted inside the enclosure the power usage increases to 125 watts while idle. When the twelve drive striped RAID set begins copying lots of data the power usage rises to between 144-148 watts. Stability The Areca ARC-1231ML SATA host adapter features RAID 6 capabilities which provide twice the redundancy of RAID 5. High Performance can be achieved when using a twelve drive RAID 6 configuration, while also providing hard drive failure protection. When using the Areca web interface with a RAID setup users will not want to hot swap the volume. Shutting down the computer first and then re-inserting another set of hard disks will work. However, hot swapping RAID sets with the ARC-1231ML will usually damage the RAID structure. Up to two hard drives can be removed from a RAID 6 configuration. Removing more than two hard drives will cause the RAID 6 to fail.Individual Drives If the ARC-1231ML controller is setup in JBOD mode, individual hard drives or RAID sets configured with Disk Utility can be hot swapped into the enclosure. Usually, each hard drive mounts 10 seconds after the last hard drive mounted. Mounting all twelve hard drives at once can take over 100 seconds. CoolingThe Norco DS-1240 cooling system includes a quiet power supply fan and three aggressive Colorful 70mm rear exhaust fans model CF-12725B. The three 70mm fans have an RPM of 3500 and can move as much as 33.5 cubic feet of air per minute each. These fans are rated at a noise level of 33.5 dBA. In the stock cooling configuration, the Norco DS-1240 keeps the hard drives very cool. However, this 12-bay enclosure creates substantial fan noise. As a test, twelve Seagate 320GB model 7200.10 hard drives were mounted in the Norco DS-1240 using an Areca ARC-1231ML PCIe RAID controller. The Areca ARC-1231ML can be configured to display SMART temperature data using it's web interface. All twelve hard drives were worked very hard duplicating a 4.2GB folder of two video files up to twenty times simultaneously for 90 minutes. At the end of 90 minutes the twelve 320GB hard drives reported temperatures of 106 to 109 degrees Fahrenheit. The next process of the cooling test was to leave the enclosure turned on with the hard drives mounted for an hour, but with no usage other than temperature monitoring. I wanted to see how well the hard drives might cool down inside the Norco DS-1240 enclosure. After resting for an hour, the twelve hard drives reported temperatures of 100 to 104 degrees Fahrenheit. The ambient air temperature was 84 degrees Fahrenheit during this test. ![]() Custom Cooling If the Norco DS-1240 will not be used in a server room, fan noise may be a consideration. Users that would like to convert this 12-bay enclosure for use in noise sensitive environments like audio/video recording, can modify the stock DS-1240 fan speeds using Zalman Fan Mate 2 controllers.The image above displays the use of three Zalman Fan Mate 2 controllers with one attached to each fan power connector. The fan power connectors are located on the right bottom side of the image above. The Zalman Fan Mate 2 controllers are installed between the power connectors and the 70mm fans. Once installed users can turn down the fan RPM speeds so that the fans operate quieter using the Fan Mate 2 speed control knob. If the user turns the Norco DS-1240 fans down to the lowest setting, typically the fan alarm will go off. The user can simply turn the fans down to just before the alarm sounds or disable the fan alarm circuits. If using the Fan Mate 2 controllers at an RPM speed fast enough to satisfy the on board fan alarm circuits is still too loud, it's time to change the fan alarm jumpers. If you open the Norco DS-1240 and look inside, facing the drive bays, you will see three, 3-pin fan connectors on the left side near the rear of the drive bays. This is where the Fan Mate 2 controllers are connected to reduce the RPM speed of the fans. Just in front of each 3-pin connector is a fan alarm jumper. As per the Norco DS-1240 manual, users can move the jumper setting from pins 1-2 to pins 2-3 to disable the fan alarm.![]() Once the fan alarm jumpers are changed to connect pins 2 and 3 the Norco DS-1240 will no longer monitor fan performance and the fan alarm will be disabled. Now turn the three Zalman Fan Mate 2, fan speed controllers all the way down. The fans will still spin, but at the lowest setting. The temperature results of this configuration are provided in the table below under the "FM2 Turned Down" columns. In this configuration, the fan noise produced by the Norco DS-1240 is minimal. Hard Drive Temperatures in Fahrenheit
Acoustics The Norco DS-1240 enclosure uses three aggressive 70mm exhaust fans with a 33.5 dBA noise rating and a quiet power supply fan. When the enclosure is running in the stock configuration it produces a significant amount of wind noise. While the powerful fans can keep the hard drives below 110 degrees Fahrenheit, the wind noise is more than many desktop users will consider acceptable. The Norco DS-1240 cooling system is great for a server room but audio and video users may want a quieter enclosure configuration. Bin Lin of Norco Technologies, Inc. has informed AMUG that a new fan controller board is being developed for the Norco DS-1240 enclosure that will reduce the fan speeds based on the temperatures of the hard drives. This should allow the Norco DS-1240 enclosure to operate much quieter. I have found that using the Zalman Fan Mate 2 controllers with the fan alarm disabled is a good solution for taming the acoustical properties of the DS-1240. This provides a quiet operating 12-bay SATA enclosure with minimal additional hard drive temperatures. In this configuration, if the enclosure is five feet away behind the Mac Pro 2.66GHz, I can hear the quiet fans of the Norco DS-1240 while the Mac Pro is running. I would classify the Norco DS-1240 as medium to quiet with this fan modification. Twelve hard drives working at full speed can generate noise as well. Without the fan modification I would classify the Norco DS-1240 as loud, based on the amount of wind noise that it produces. The EnhanceBox E8-ML is quieter than the Norco DS-1240 but only holds eight SATA hard drives not twelve. Sleep The Norco DS-1240 was tested for Macintosh sleep capability with the Areca ARC-1231ML host adapter using the mraid_macpro version 1.3.1 driver. The ARC-1231ML host adapter does not support sleep mode while a RAID is attached to the card. The Norco DS-1240 enclosure will support sleep mode with a host adapter that also supports sleep. PerformanceThe Norco DS-1240 was tested while connected to an Areca ARC-1231ML with 1GB of memory installed in slot 4 of an Apple Mac Pro running Mac OS X 10.4.10. Twelve 320GB Seagate model 7200.10 hard drives were used to create various RAID configurations while mounted inside the DS-1240. DiskTester 2.0 10GB read and write tests were performed using a run area test so that this article could display how the RAID performed. DiskTester is a Terminal application that measures the combined performance of a volume and the Mac OS X operating system. Using the command: ./disktester run-area-test --chunk-size 128M --test-size 10G --delta-percent 10 DriveName, puts DiskTester to work testing how the RAID will perform when empty, 10% full, 20% full and so on. The table below displays how the Norco DS-1240 performed when using RAID 5 and RAID 6 configurations. Norco DS-1240 - Twelve Drive RAID 5 & 6 Comparison
The performance results above indicate that the Norco DS-1240 12-bay enclosure can provide amazing RAID 5 and 6 performance when an ARC-1231ML controller is used with it inside an Apple Mac Pro. A RAID 5 setup can provide average write performance of over 700MB/sec and average read performance of 560MB/sec. Using eleven hard drives to configure the RAID 5 with a hot spare reduces the average write performance to 658MB/sec. and the average read performance to 532MB/sec. This setup reduces performance by the speed of one disk drive but provides the ability to automatically rebuild the RAID 5 if a single hard drive fails.The Norco DS-1240 RAID 6 configuration is able to provide performance averages that are very similar to a RAID 5 configuration with a hot spare. This occurs even though RAID 6 is required to calculate an additional set of parity data and distribute it across all twelve hard drives. The ARC-1231ML XOR engine is able to process a single set of parity data for RAID 5 or a dual set of parity data for RAID 6 with very little difference in performance levels. The 800MHz Intel IOP341 I/O processor mounted on the ARC-1231ML makes this possible. While RAID 6 provides twice the redundancy of RAID 5 some users may want to consider using the Norco DS-1240 RAID 6 configuration with a hot spare. This allows the RAID 6 to rebuild automatically in case of a single hard drive failure. Servers that are difficult to get to or off site may be strong candidates for hot spare usage. Hot spares setup with the ARC-1231ML controller do not spin up until they are needed. This feature helps to keep the spare from experiencing extra wear while waiting to be used. RAID 6 Redundancy Many of the SATA host adapters that AMUG has reviewed do not support RAID 5 or RAID 6. Typically, these controllers support striped RAID sets that will fail if a single hard drive failure occurs within the RAID set. This leaves the user with the need to keep timely duplicate backup copies of the data should a failure occur. Sometimes having an up to date backup copy is difficult to accomplish and data can be lost. The use of RAID 5 creates the ability for the RAID set to usually survive should a single hard drive failure occur. Adding the use of a hot spare can start the RAID rebuilding process immediately. However, if another failure occurs during the rebuilding process all data is lost on the RAID 5 volume. RAID 6 adds another layer of data protection by calculating a dual set of parity data and distributing it across all of the hard drives. Up to two hard drives can fail and the RAID 6 volume can still be recovered. While users should always have a backup of important data, the redundancy of RAID 6 provided by the ARC-1231ML controller when used with the Norco DS-1240 enclosure adds significant hard disk failure protection. Advanced RAID users will find that using the Norco DS-1240 with the ARC-1231ML adds a great deal of redundancy while providing amazing performance advantages. The added crash protection and high performance obtained while using twelve hard drives with a RAID 6 configuration provides a strong RAID platform for serious data systems. The table below displays the DiskTester performance results of an initialized RAID 6 volume and how the performance changes with a singe disk failure, a two disk failure and during the RAID 6 automatic rebuilding period. Norco DS-1240 - RAID 6 Drive Failure Performance Test
The twelve drive Areca ARC-1231ML RAID 6 configuration mounted inside the Norco DS-1240 required 5 hours and 49 minutes to initialize. Removing up to two hard drives from the RAID 6 volume did not significantly reduce its write performance. The read performance was reduced by approximately 40% when rebuilding or with up to two hard drive failures. However, the average read performance was still over 310MB/sec. The rebuilding process slows if the RAID volume is being accessed and resumes at full speed once the volume is inactive. Rebuilding the RAID 6 volume with two new hard drives required 3 hours and 33 minutes. Once the RAID 6 volume was fully rebuilt it was able to perform at full speed with no loss of data. The redundancy provided by the Norco DS-1240 / ARC-1231ML RAID 6 controller combination is far superior to striped RAID sets and even RAID 5 solutions. RAID 0 & 10 RAID users looking for maximum performance will find that RAID 0 is the speed demon. While RAID 0 provides zero crash protection, it usually provides the fastest performance. Using the same twelve Seagate 320GB model 7200.10 hard drives, the Norco DS-1240 was configured in RAID 0 and RAID 10 mode using the ARC-1231ML web interface. Performance results for these configurations are provided below. Norco DS-1240 - Twelve Drive RAID 0 & 10 Comparison
Setting up the Norco DS-1240 as a striped RAID set provides amazing speed. However, its like driving 100 MPH without a seat belt. If users have a backup of all important data and do not mind spending the time to rebuild the RAID when a hard drive fails, it is a great tool. However, once you experience the crash protection provided by RAID 6, it is hard to be satisfied with a striped RAID set when working with important data. RAID 10 also provides redundancy. A mirror of each hard disk is created with a RAID 10 configuration. As long as a single member disk drive and its mirror do not both fail, recovery is possible. The problem with RAID 10 is that it requires double the hard drives to accommodate all of the mirror drives. While RAID 10 provides a very good system for crash protection, it is not as fast and provides a smaller volume size than RAID 6. The Norco DS-1240 can be used in JBOD mode, RAID 0, 1, 3. 5. 6 or 10 when utilized with an Areca ARC-1231ML controller. When the Norco DS-1240 is combined with the ARC-1231ML the user has an amazing amount of storage configuration options. DiscussionThe Norco DS-1240 ($639.99) enclosure provides twelve hot swap SATA hard drive bays with three Infiniband multilane interfaces at a great price. The hard drive trays are easy to slide into the enclosure and easy to mount hard drives into. While the unit is designed for a rack mount, placing the DS-1240 on its side provides an excellent vertical mounting setup for hard drives. The DS-1240 is a very sturdy industrial strength enclosure. The stock fan system provides excellent cooling capability for server room installations. Desktop users looking to convert the DS-1240 into a quiet enclosure can install three Zalman Fan Mate 2 controllers as discussed earlier in this article. The Norco DS-1240 provides a cool running hard drive | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||