Since digital files are the lifeblood of the business, I just wanted to talk shop about storage methods. I'll start with ours and also give a recommendation to interested parties on building a similar system.

We have two MacGurus Port Multiplier Burly Boxes, a 4-bay and 8-bay. Each has a 500GB Seagate drive in the 8-bay (~4 TB). The 4-bay has a mix of 3x300GB and a 500GB drive (~1.4 TB). I got the drives -- a mixture of Seagate, Western Digital, and Hitachi -- from NewEgg and OtherWorldComputing.

Each box is connected to a Sonnet Tempo X4P port multiplier SATA host card in our G5 tower.

The entire array of drives are set up as individual hard drives -- not as RAID*. Most of the drives have a corresponding drive with identical data. So for example our Wedding Storage 2006 drive has a companion Wedding Storage 2006 Backup drive that exists in another slot with the same data. Generally a set of files for a shoot will live in 2-3 places in our system, between the PowerMac, 8-bay and/or 4-bay Burly boxes. Inside our PowerMac we have two 500GB hard drives.

A nightly backup program, Retrospect, kicks off automatically at 11:30 PM and backs up all of the files that were added or changed in a list of designated folders.
--

If I were building a system today I would do something very similar to our setup:

• The 8-Bay MacGurus Burly Box. ~$800
• This time I would get the CalDigit FASTA-4e SATA host card (if I had a PCI-Express compatible machine) -- $300
• 8 x either Hitachi or Seagate 500GB hard drives -- $160 x 8 = $1280.
• Total: ~ $2400.

Right now we're trying to figure out the best way to keep stuff offsite. I'm thinking of dropping a few spare drives & drive trays at the in-laws' house every other week, but we haven't implemented that plan yet!

Anyone else?

A raid array for RAW current files, Pictage for small jpegs, DVD's for RAW archiving, and a two-year file release policy to my wedding clients so my basement doesn't get cluttered.

I move files from CF to HD, then burn RAW backups to DVD.
There is an off-site backup server and pulls all new/modified files from my main computer at 3AM.
For stuff I'm currently working on, it also gets manually copied to an external 250GB HD after I'm done for the day. Then it is turned off.

I store DVD's offsite.
My backup server is also offsite.
Offsite backup storage is important. What if your studio burns down? It doesn't matter how many RAID drives you got in that case.
I use the external 250GB HD with it's own power source for a reason. This is because you can get a power surge and fry everything in your computer.... drives and all. Having one drive as external with an external power source will help isolate it from that disaster. And yes, I've had a bad power supply fry my drives before.

Some people say it's excessive, but the last thing I want is to lose someone's wedding photos.

What kind of RAID are you running? RAID 5 or RAID 10?

* After doing some research and talking to numerous people we decided to go with individual drives and basically duplicate them. The MacGurus guys really dissuaded me on RAID. The reasoning is since RAID splits data across multiple disks with a parity drive (RAID 5) it is fault-tolerant to a degree -- but you lose everything if two disks die at the same time from something like a bad power supply.

I figure if we have two independent disks in different machines, either in array 1, 2, or the PowerMac we're more insulated from failure than if we were running a software RAID. I've heard hardware RAIDs are decent, but it seems like overkill and overcomplicates something that IMO should be as simple as an electronic file cabinet.



How is your offsite backup? Is it part of your website or is it a paid service? What are the limits?



I don't think it's excessive! It's excessively cavalier to believe that a single LaCie [read: crap] external drive is a backup strategy for once-in-a-lifetime images!

Thanks Damon. This thread makes my brain hurt. Sigh. I think it's amazing how much $$$ is spent on storage. Everyone likes to say that digital will save you money over film, but I beg to differ on the storage & processing gear alone! I can't believe it will cost $2500 for a storage system (but of course I wont' even tell you guys my lab bill from last year...let's just say I could have one SWWWEEEEET system). How many images would you be able to store on this system, and how often would you have to basically purchase another one? Assuming you dump most of it, but of course not all of it, after 2 years...

Thanks!!!!!



Roger that!

I'd be really interested to hear if anyone actually used tape!

We had this discussion at the last Atlanta PUG... I can second the Burly Boxes from MacGurus! I use a 5-port and an 8-port Burly connected via eSATAII to a 4-port PCI card in my PC. The drives are fully hotswappable and work awesome.

The 5-port box has five 300GB Seagate SATAII drives with one drive on a dedicated channel as a Photoshop scatch drive and the other four on a multiplier.

The 8-port box currently only has two 400GB Seagate SATAII drives with four drives on one multiplier and the other four on another multiplier.

The idea is that I can keep adding capacity as I need it and replace older drive sets as bigger and faster drives hit the market.

I am NOT doing hardware or software RAID for the same reasons as already mentioned. I use a program called MirrorFolder that will do real-time or scheduled mirroring of data from any location or folder to another. I mostly use it in real-time mode and I keep the mirrors on different SATA channels so I don't get killed on throughput. I set up the drives in Windows XP so they mount into folders on my C drive:

C:\Data\Online\Media
C:\Data\Online\Store-01
C:\Data\Online\Store-02
C:\Data\Mirror\Media
C:\Data\Mirror\Store-01
C:\Data\Mirror\Store-02

I share the C:\Data folder and as I edit more and more on the Mac, I can connect to the entire array through a single share. As long as everything is Gigabit, I can actually keep everything in the drive array and edit with Lightroom on my Mac over the network.

I am currently backing up to DVDs (~$0.07 MB) but I am seriously considering just taking hard drives offsite (~$0.27 MB) considering the time savings and that I can recycle and recycle the hard drives.

I know it all sounds complicated but it's really not. It's also very scalable and very cheap.

We have a FireWire tape backup drive that dumps onto 80GB tapes. We need to get some tapes for 2006's stuff, but the purchasing dept (me) has been somewhat lax on that.

Time to get back to work!

You're right. It is insane how much we'll end up spending on storage and computer-related items for photography. It seems almost counter-intuitive. We just consider it insurance or the cost of doing business.

Obviously brides and other clients could care less why their pictures are lost forever on a crashed disk/unreadable DVD or CD/water damaged computer, etc.; a good data storage plan will allow part of your system to fail without having that failure destroy your good name & business.

Building a file server can save you money if you think about your long-term storage goals before you plunk down the ~$3000 it takes to get up and running.

Working backwards from the ~5 TB of storage we have, I'd say we can store about a year and a half's worth of images and backups on the array before we have to buy more drives. Notice that we don't intend to buy a whole new array -- just new hard drives -- and by next year it wouldn't surprise me to see > 1 TB individual drives!*

*For some nerd fun you should check out the Schoolhouse Rock-themed "Get Perpendicular" video from Hitachi.

MacGurus is a good place to start because you can call them up and walk through the process. If you're not terribly keen on doing the build yourself you can go through them for everything and the cost difference is < $300 I think. Some turnkey solutions exist from companies like CalDigit and FirmTek as well.

The bottom line from all of this is that a data storage plan is not optional! Making one DVD backup and calling it a day is a recipe for disaster (notice the recurring theme in my posts )

***OT***

Using film really was a lot simpler in process. I looked at your site and see you're using medium format, 35mm, and some digital (but mostly film). That's really cool.

When the wife started shooting weddings I was the backup with two film cameras and Kodak Portra VC & Fuji Velvia 35mm film. All you had to do at the end of the day was drop off the film at a good lab and then wait for your pics and scans to come back! None of that RAW stuff!

I store my files on a single drive. That drive gets synced up with a file server with a RAID 5 array. The file server, in turn, has a nightly snapshot taken onto one of two machines with even larger RAID 5 arrays.

I keep all my files on these new swanky floppy things. They're about 5.25 inches square and totally blow away all these stupid punch card things I have.. which by the way do an amazing job of getting my fireplace going.

Just avoid them new 3.5" ones all these kids are talking about.. they are terrible!

It is a much simpler process truth be told and I'm not sure why folks would ever be afraid of film! Quite the opposite! LOL.
Thanks for alll the food for thought though Damon! Great stuff in this thread.

That's a serious system.

How large is your server room and how much is your electric bill?

I agree. Film is still better than this digital stuff in a lot of ways.



Glad to help! I figure I'll contribute from my area of knowledge -- computer stuff...

No way! You lose negatives you're beat.. you lose a digital file you pull up your backup.

Touché.

I was actually going a little OT and remembering the crispness and range of film. Real negatives are a real pain to keep track of and keep safe compared to digital files...

...though it stinks how we (the paranoid/safe ones) end up managing little homemade data centers instead of, like, taking pictures!

I just shoot- my husband is the computer man. We DO work together that way! I will ask him.

The other day I read the thread about how much CF cards everyone has. I guess this is where shooting only 100 images/hr really pays off- no need for 8TB bays!

I only have 2 500GB external drives and I am completely confident in my backups. I upload to drive 1, then make a DVD backup. That backup goes offsite. I also make CD backup of the Jpegs when they are done which also goes off site as well. Drive 1 gets backed up to drive 2 every week. I run 2 battery backups through each other so power issues aren't a concern. Each year or so the drives get cycled out into retirement/storage.
I only promise clients 3 years of storage so I really don't see the need to drop $2500 on un-needed storage. The only way I could ever loose images is if my house burned down AND my off site DVD's AND jpeg CD's went bad.

Your system sounds pretty solid. Do you only shoot weddings?

I know our storage sounds kind of ridiculous, but I couldn't see having < 5 TB.

I shoot product stuff too for some wineries and bicycle companies. Nothing that takes up nearly as much space as a wedding.
The key for me is that a complete wedding is only 15GB or so with jpegs and edited jpegs. I shoot like I am using film (because that is what I am used to) so it really seems to save me time in editing and money on storage.

We come in about the same weight for a wedding (14-15 GB). We keep a lot of duplicate data online in the file servers, so that's probably why I've let it get so out of hand

I don't trust DVDs for long term storage, so that could be part of it too.

It's funny that you mention film. We both try to shoot faster, but coming from film it's just too hard to do the spray-n-pray style of shooting!

hehe, I was wondering if I was the only one that was trying to take more pics, not less!

I come from a film background and have shot weddings with film. I gotta say, I have NO problem embracing the spray style! :-)

oh... and my storage sucks. Hopefully in the near future I'll have some sort of tower with about 8 or so drives and I believe mine will be mirrored. I don't need a fast raid. I don't work off of those drives, they are simply for storage.

I also only promise my brides storage of their files for 1 year.

also, pictage has all of my jpg files slightly compressed from when I uploaded them on storage backed up for me offsite.

Any suggestions for a server/storage for 2 users in different locations. My business partner lives about 10 minutes away, we need stoarge and would like to share files. Eventually we will get an office space and be able to have a beefy system.

Thanks!

PHOTOSHELTER

Dual raid servers on each coast, backed-up nightly.
Website gallery integration
SALES

all in one place.

I select my proofs, upload them, and forget about it. I can download hi-rez files at will, and clients can get low-rez comps. It is a fantastic way to do business. Sort of like Pictage, but better, IMHO.

Feel free to PM me if you have specific questions.

You can see it in action on my website at EricHegwer.com, click on events. Each of those galleries is fully hosted and backed up with Photoshelter.

[quote name='Damon Noisette' date='March 2 2007, 02:32 PM' post='88975']
Since digital files are the lifeblood of the business, I just wanted to talk shop about storage methods. I'll start with ours and also give a recommendation to interested parties on building a similar system.

We have two MacGurus Port Multiplier Burly Boxes, a 4-bay and 8-bay. Each has a 500GB Seagate drive in the 8-bay (~4 TB). The 4-bay has a mix of 3x300GB and a 500GB drive (~1.4 TB). I got the drives -- a mixture of Seagate, Western Digital, and Hitachi -- from NewEgg and OtherWorldComputing.



We did this exact same thing. Our set up is a Burly 8 Enclosure with a Cal Digit Sata card connected to our XServe.

Last week I discovered this product and ordered one in for some of our old Lacie Drives. I wanted a way to keep all my files online via our Xserve and just needed a way to connect old ATA drives. I popped out the drives from the old enclosures and put them in the new trays and it works perfectly. It's Firewire 400, but it sure beats daisy-chaining 12 drives and having power cables everywhere.