HighPoint RocketRAID: Array Creation
From Debian Clusters
This is the first page of a two part tutorial on HighPoint RocketRAID. For a bare bones introduction to RAID, see File System: RAID Arrays. The full RocketRAID tutorial includes:
- HighPoint RocketRAID: Installation
- HighPoint RocketRAID: Array Creation
Contents |
Creating an Array
A group of hard drives put together via RAID is called an "array". Before creating an array, you'll need to have the hard drives hooked up to your RocketRAID controller (usually a PCI card) and also connected to the power supply so they can spin up.
The management utilities installed in the HighPoint RocketRAID: Installation previous section will allow you to create arrays of hard drives. We had more lucky with the GUI than with the command line interface. With the command line interface, changes to the array just didn't seem to "stick". Your mileage may vary.
GUI Setup
However, using the GUI, you'll need to have a windowing system installed. This is pretty easy to do. Just run
apt-get install xserver-xorg xutils
As part of the install, you'll be prompted for which resolutions to keep. Use the space bar to select resolutions for the server to try to use. If you don't know, you're safer choosing lower resolutions and more typical resolution sizes.
-X at the end. For instance, I ran
kwanous@cassowary:~$ ssh eyrie -X
There's a great little utility called xlogo that's used to test if the xserver is running correct and forwarding graphics. Once you're logged in with SSH forwarding on (hence the -X), run xlogo and you should see a little X graphic popup like the one shown to the right.
Sudo Setup
Your user will need to be able to run the HighPoint RAID management utilities with root privileges without becoming root (root can't own any windowing processes). This is done through sudo privileges. First, as root,
apt-get install sudo
Then open /etc/sudoers. At the bottom, you'll see that root already has privileges to run any command with sudo. Give RAID utility privileges to another user by adding a bottom line like the one below:
# User privilege specification root ALL=(ALL) ALL kwanous ALL=NOPASSWD: /usr/bin/hptraidconf, /usr/bin/hptraid
ALL specifies that this should take affect on all hosts (meaning just this one, unless you've set up sudo to work across multiple machines). NOPASSWD means that my user, kwanous, doesn't need to enter a password in order to run the two commands listed, hptraidconf and hptraid. Notice that the full path names must be given for both of the binaries.
Whew! Now you're ready to SSH into the cluster with window forwarding (-X) as your username (if you aren't still SSH'd in with forwarding on). As the user, for the GUI interface, run
sudo hptraid
or, for the command line interface,
sudo hptraidconf
Creating an Array
Either way, you should now be ready to create an array. First, you'll need to log into the daemon on your server. The default username is RAID with a password of hpt. (The first time you log in, you'll be immediately prompted to change the password.)
Before you can use a hard drive in an array, you'll need to initialize it. Then you'll specify the array name and type of array, and add the hard drives to it one at a time. Once the array is created, it also needs to be initialized. This can take a very long time, upwards of eight to twelve hours or even over twenty-four hours, depending on the type of RAID and the size of the array.
For more information, refer to the documentation on HighPoint's website.
Once you create an array, you MUST update Debian or you'll find yourself with a system that won't boot. Continue on below...
Updating the System
Once a new RAID array has been created, Debian automatically moves that array to be the first "hard drive". For instance, if you run fdisk -l prior to creating an array, it might look something like this:
Disk /dev/sda: 81.9 GB, 81964302336 bytes 255 heads, 63 sectors/track, 9964 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Disk identifier: 0x000af9fb Device Boot Start End Blocks Id System /dev/sda1 1 1 8001 82 Linux swap / Solaris /dev/sda2 * 2 9964 80027797+ 83 Linux
Above, I have one hard drive, /dev/sda with two partitions: /dev/sda1 is my swap partition and /dev/sda2 is my file system. GRUB (the bootloader) and other programs have this hard coded into the preferences. However, once I created my RAID 5 array, the output of fdisk -l switched to look like this:
Disk /dev/sda: 5000.6 GB, 5000617000960 bytes 255 heads, 63 sectors/track, 607957 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Disk identifier: 0x8dfd657f Device Boot Start End Blocks Id System /dev/sda1 1 267349 2147480811 ee EFI GPT Disk /dev/sdb: 81.9 GB, 81964302336 bytes 255 heads, 63 sectors/track, 9964 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Disk identifier: 0x000af9fb Device Boot Start End Blocks Id System /dev/sdb1 1 1 8001 82 Linux swap / Solaris /dev/sdb2 * 2 9964 80027797+ 83 Linux
Now the hard drive I boot from is /dev/sdb instead of /dev/sda. The letter in /dev/sdX has been incremented by one. This needs to be updated in several places.
Note: If you are using an IDE array rather than a SATA array, you'll be looking to change hda's instead of sda's, in all of steps below.
GRUB
GRUB is arguably the most important place to change this information. Before the RAID array was created, it always booted from /dev/sda2 and it needs to boot from /dev/sdb2 now instead. (If you had multiple hard drives before, yours may be different letters. Run fdisk -l to check.)
Open /boot/grub/menu.lst for editing. At the bottom of the file, you should have one or more sections that look like this:
title Debian GNU/Linux, kernel 2.6.24-1-amd64 root (hd0,1) kernel /boot/vmlinuz-2.6.24-1-amd64 root=/dev/sda2 ro quiet initrd /boot/initrd.img-2.6.24-1-amd64
In all of those, increment the X in root=/dev/sdX#. For instance, I changed mine from
root=/dev/sda2
to
root=/dev/sdb2
/etc/fstab
Next, open /etc/fstab. This tells the operating system which file systems to mount and where. Yours should look something like this:
# /etc/fstab: static file system information. # # <file system> <mount point> <type> <options> <dump> <pass> proc /proc proc defaults 0 0 /dev/sdb2 / ext3 errors=remount-ro 0 1 /dev/sdb1 none swap sw 0 0 /dev/hda /media/cdrom0 udf,iso9660 user,noauto 0 0
Again, increment the letters on your /dev/sdX#'s.
uswsusp
Finally, if you're using a system that had the full windowing system installed on it, like Gnome or KDE, you need to update uswsusp. (Hopefully this won't be you, because most people shouldn't have the overhead of full windowing system installed on their head node!) Uswsusp is used to suspend (hibernate) and resume (wake up) a computer with a saved image. If you don't update it, it will be looking for a resume image in the wrong place, and you'll see errors like this at boot:
resume: could not stat the resume device file
To fix it, you'll need to run dpkg-reconfigure uswsusp. Keep the defaults, and if prompted for the location of the partition where the resume image is stored, be sure to increment the letter appropriately. This should update /etc/uswsusp.conf. However, it will finish with an error, something like this:
update-initramfs: /boot/initrd.img-2.6.24-1-amd64 has been altered. update-initramfs: Cannot update. Override with -t option.
It cannot update the ramfs image because the image is a custom one (created in the previous section). To override this, run
update-initramfs -tu
-
-uis for update -
-tis the magic for "t"aking over a custom image
Partitioning/Filesystemming the Array
You'll need to restart once in order for the above changes to take affect, and for the RAID array to be plunked down in /dev/sda (if it's your first array, or further down if it's a secondary or tertiary array).
Partitioning
Partitioning and creating a filesystem on a large array is a little different than just formatting a single hard drive. First of all, for larger filesystems (rule of thumb >2 TB), it's better to use parted rather than fdisk to create the partitions. Install gparted with
apt-get install gparted
and then run it with gparted. First, you need to select the particular "hard drive" (or array) to use. If this is your first RAID array, it will be /dev/sda for a SATA array or /dev/hda for an IDE array. Use
select <full drive path>
Once you've selected it, you can create the label for what type of array it is. (gpt is often used for larger filesystems and would be an appropriate choice.) For instance,
mklabel gpt
To create the partition itself, run mkpart. You'll be prompted for the partition number; normally a RAID array only has a single partition, so enter 1. Next it will ask for the starting block; 0 will indicate it should start at the beginning of the array. Finally you'll be asked for the ending block. -1 will tell it to continue to the end of the array.
That's it! Type quit to exit parted.
Creating the Filesystem
Similarly, the filesystem ext3 has limitations for use in a large array. Ext3 is limited (I believe) to a 16-terabyte file system. If you're using something bigger than this, or just to be fun (not mention, creating an xfs filesystem takes far less time than an ext3 one), you can install xfs (originally from SGI) on it instead. Rather than using mkfs.ext3 like normal, you'll need a new utility that comes as part of the xfsprogs package. Install it with
apt-get install xfsprogs
Then you can format your new file system with
mkfs.xfs /dev/<your drive>
where <your drive> is equal to the array that you just created (probably /dev/sda1 in most cases).
Buckle Your Seatbelt and Enjoy the RAID
At this point, your RAID controller and RAID array should be up and running, and all you need to do is pick a mount point and mount the array in /etc/fstab.
For some fun, install hdparm (apt-get install hdparm) and run a few benchmarking tests for comparisons with other single drives:
hdparm -T <drive>hdparm -t <drive>
Happy RAIDing!
