RAID fun
So, peterbilt’s RAID finally formatted out 100% good.
We’ve learned many things from this adventure:
1) PCI-X and PCIe are NOT the same connector
2) When partitioning RAID devices, aligning to a 1 megabyte sector boundery helps (use 32768s in parted)
3) parted is much easier to use than fdisk for certain things
4) msdos disklabels are no good for volumes bigger than 2T. Use gpt disklabel instead
5) onboard RAID controllers on motherboards (i.e. ‘fakeraid’) and ubuntu do not get along very well
6) The RocketRaid products are not very good either although they at least work albiet very slooowly
7) Do not ignore warnings about RAID arrays lest they become more serious warnings about RAID arrays
8) mke2fs has -T largefile for files that contain mostly huge files. Saves time, makes less inodes
9) -E stride=N – N is your RAID card’s stripe / 4. Does help. I knew this before, but was good to be reminded.
10) parted needs better error checking on msdos filesystem for sector counts that wrap
11) raid 50 is *incredibly* forgiving.
12) On thermaltake cases, DO NOT LEAVE THE SIDE PANELS OFF! They keep drives cool to the touch when the panels are on*.. or will burn you when they are off. I figure i hastened at least two drives to their graves. I feel vaugely bad about doing a RMA on them… but why doesn’t SMART *report* that they’re overheating? Good equipment shuts itself down and lives on to fight another day, IMHO..
13) Ultimately, the last ditch backup strategy that worked was to go buy a couple of 1T external disks and manually weed the directories out into <1T chunks, then pass some –exclude= lines into tar. Many many higher tech things were tried first, all failed. [But someday soon I will have one heck of a backup server. Yes, backups have now progressed in my life to where they warrent their very own server]
14) Certain unnamed people were extrordinarily helpful in retaining my sanity, and were pillars of strength..
15) I need to get some kind of psychological help – when the thought of losing a terabyte and a third of data is this traumatic.. I mean, literally, I haven’t slept well since this thing started.
In the course of a long life, a man must be willing to abandon his baggage many times. (probably misquoted) –Lazarus Long
I can’t even abandon my data..
* = in peterbilt’s configuration, which is eight SATA disks in 5.25″ removable caddies with individual fans
August 5th, 2007 at 2:36 am
Ultimately, I ended up having to use lvm – even with a gpt disklabel, linux just doesn’t like partitions bigger than 2T.
October 31st, 2007 at 12:13 pm
Haha, you’re a sick man.
I take a completely different approach because I generally hate raid…
Burn everything twice (inclkuding greppable txt catalog of course), then simply make a folder X:\MEDIA\READY-TO-DELETE or such, and put stuff in it. It’s free to be deleted any time, and you know you have a backup. You don’t have to mess with raid. If your computer has a failure, you can take out ONE harddrive with your most important stuff, and put that ONE harddrive into ANY other computer, even if it’s Windows 95 (hahahaha). (Well, I use NTFS now and not FAT32, so I guess it would really have to be Win2K?)
Obviously if my C: drive dies, I will have to re-install my OS. This is where RAID would really help me, but I was too strapped for ports in the non-SATA days. (6 sata ports still aren’t much to me… Carolyn had EIGHT drives in her last computer using THREE SATA controllers.)
Having more computers helps that problem. I dropped from 3, to 2, to 1, and just got back up to 2. Thermaltake case. And carolyn always had problems with her SATA/RAid cards. One of them was definitely RocketRaid! It’s much better to be built into the motherboard.
But my stuff is mostly video files. If they get deleted, I pull them back from the disc. If the disc is scratched, I get the 2nd copy, pull it back, and re-burn it twice again on a new disc.
My reasoning is thus:
raid means more drives means more money, and i buy 1 harddrive at a time, all different sizes, when and if i need new space. my last drive purchases (for 10 yrs): 4G, 16G, 25G, 40G, 60G, 80G, 80G#2, 80G#3, 120G, 120G#2, 120G#3, 120G#4, 120G#5, 250G, 300G, 400G, 500G, 500G#2, 750G, 750G#2, 750G#3. I’m currently at 4.5TB spread across 3 computers. All drives have the same local letter as the network letter. Sometimes this means mapping T: to point to your own C:, but it’s convenient to always think of T: as T:, even when it’s really C: on that local machine.
also: raid is a point of failure. if your raid dies it can in theory hose EVERYTHING which i find far more disturbing than losing 1 drive.
I also use Karen’s Replicator to “manual raid” — every reboot it checks against a schedule and re-copies my digital pics, mp3s, and program installers. This takes awhile, but I’m usually not around when it happens, as I made it part of my BEDTIME.BAT … (which runs an hour or so.. assuming i remember to run it when i sleep. i’m too irregular to schedule such a think).
October 31st, 2007 at 2:18 pm
This doesn’t work as well when you actually have 1.5T of media you want to archive. There are four basic problems I have:
1) I do multitrack recording – this burns 100 megs a minute of disk
2) I do video editing – fortunately, not on this volume – this burns 300+ megs a minute of disk
3) I hate commercials, so I don’t watch broadcast TV – instead, I download TV episodes
4) I’m a packrat. I hate to throw anything out, ever.
Actually, at my current growth rate, I’m going to have to move to a 4T file server soon. This is more complicated than it looks, because I will then need a 8T backup server. I’m looking at 16 port cards..
October 31st, 2007 at 9:33 pm
300 megs a minute isn’t anything a single drive can’t do. I don’t watch broadcast TV and am a packrat either. 4TB holds a small fraction of what I possess… If I move a 350M ep from 1 computer to another, it’s usually a minute tops. Just seems you could achieve much of the same without having to spend so much money. What’s the fun in working 60 hrs?
October 31st, 2007 at 11:43 pm
*shrugs* that whole experience cost me about a week’s pay and I ended up with a good desktop system as well as the backup server – I’m working 60 hours to pay off past debts, and just have to resist buying toys in order to add to them too much 😉 2TB copies to the backup server in about a day. Another factor in me playing with RAID is that I have to do these things for my day jobs and it’s nice to already know what I’m doing – we’re likely to use similar technology for a start-up that I’m doing consulting for, for example.