|Tutorial: - Backup
Strategies Part 1.
Strategies Part 1.
A discussion of The CalDigit VR
By Brian Gary
This article started
off as a review of the CalDigit's
VR, but quickly evolved into the first of a series of articles
that will focus on: backup strategies, secure storage workflows,
and archiving media and data.
The whole issue of data and media
storage has become a minor obsession of mine and the need to
deal with mass amounts of data is no longer relegated to corporate
IT departments. Most of us working in post-production are dealing,
if not on a daily basis, then on a weekly basis, with terabytes
of data--most of it media content. Working with this data safely,
defining clear backup strategies and ultimately archiving it,
is quickly becoming a paramount issue for even the single editor
with the single edit suite not just the post-production houses
and studios. More and more we have to provide IT services as
part of the editorial process.
When you add file-based workflows
into the mix (meaning those projects that were not shot on tape
or film and so therefore reside only on solid-state or platter-based
media) you've added an entire level of urgency with regard to
protecting that data. I've experienced this on more than one
occasion. For example I was hired on as the editor for the independent
feature film Frankie
D when the first editor lost a day's worth of shot media.
The movie was acquired on P2 and so the production footage came
to the editor on hard drives. One of the hard drives failed and
the producers soon found out that was the only copy of that particular
day's production. A call to Drive
Savers and $8,000 in recovery costs later, only a few clips
were salvageable. Now, in the original editor's defense, P2 was
a relatively new workflow, a hard drive was provided for editorial
and no clear file-based post production process was ever fully
established. In fact no one was managing the media because the
workflow for a non-film, non-tape project wasn't planned out
and it wasn't clear that anyone in particular was responsible
for wrangling the media from the set to post production. When
I came on board I enacted both an IT protocol and post-production
strategy. Now that sounds a lot more involved than it was in
practice because in reality all that meant was I ensured that
every clip of production media existed in at least three places.
Which leads me to axiom number
one, the prime directive of backup strategies:
You do not have data until you
have data twice...preferable thrice.
In short, that means you have to
have at the very minimum a mirror copy of your data and better
yet another copy of it in another location offsite.
The original Frankie D workflow
had the editor working on the only copy of the production media
and all that precious data was housed on a LaCie Big Disk which
is basically two hard drives striped together, which is the most
precarious and unsafe of hard drive setups. So when the catastrophic
mechanical failure occurred, most the data was gone for good.
Which even if it's not your fault directly, does not bode well
for job security: especially if you're the one who has to break
the news to the producers.
As I said in the beginning I'll
be talking about many different types of strategies during the
course of these articles, but I've decided to start the discussion
with a simple, accessible and affordable system: the CalDigit
VR (MSRP $699 for the 2TB unit)
This is a two-drive hardware RAID that
can be configured as either RAID 0 or 1. The drives are hot-swappable,
but do require mounting in a tray versus some of the trayless
models from WiebeTech.
The entire VR line ships with drives pre-installed, but you can
exchange drives at a later date if, for example, you want to
increase the unit's storage capacity. The form factor and footprint
are quite nice; in fact the unit is much smaller than I expected
and as you can see from the picture it fits right in with Apple's
design of the MacPro case.
Before I get into the unit and its usage,
let's have a brief discussion of RAID (Redundant Array of Independent
Disks), specifically RAID 0 and RAID 1:
RAID 0 (also referred to as a stripe)
is taking two or more drives and running them in aggregate in
terms of both speed and capacity, meaning that all the drives
are treated as one big drive. The more drives in a RAID 0 the
faster it will run. For example, if you have two 500GB drives
and you place them in a RAID 0 set, the drive that will mount
on your desktop will be a 1-terabyte drive (formatted capacity
will be less). This drive set will run faster than a single 1-terabyte
drive because you are harnessing the mechanical elements of two
drives. The downside is that because the RAID controller is writing
data across the two (or more) disks, if one of the drives fails
you will lose ALL of the data in the RAID set. RAID 0 sets can
be managed by both hardware controllers and software like Apple's
Disk Utility or SoftRAID.
RAID 1 (also referred to as a mirror)
is taking two or more drives and duplicating the media across
all the drives in the set. For example, if you have two 500GB
drives in a RAID 1 set, a 500GB drive will appear on your desktop
(formatted capacity will be less) and all data that you write
to the RAID 1 will be duplicated on the second 500GB drive in
the RAID set. So, if you have ten 500GB drives in total in your
RAID 1, you'll still only see a single 500GB drive on your desktop,
but you'll have nine duplicate copies of the data on the other
drives in the RAID 1 set. Therefore, data loss in a RAID 1 occurs
only if every drive in the set fails simultaneously. The tradeoff
to full redundancy is loss of aggregate speed and capacity. A
RAID 1 will only have the total capacity of the smallest drive
in the set.
As a point of clarification, all of this
assumes data integrity, meaning that data corruption is different
than data loss. In both RAID 1 and RAID 0 configurations, data
can still exist, but it can be bad data because of errors during
the write process.
CalDigit's VR is a hardware RAID that
you can choose to set as either RAID 0 or RAID 1. The VR also
has the ability to operate as a JBOD (Just a Bunch Of Disks)
whereby the unit writes data to the first drive until it's full
then starts writing to the second drive until that one is full.
The VR ships with two drives pre-installed (the unit I received
had two 1TB Hitachi Brand drives) and it is configured as a RAID
0. CalDigit's website lists other capacity and format configurations
for the VR, so choose based on your storage needs.
The unit is easy to setup:
- Take it out of the box.
- Plug in the power.
- Choose your connection.
- Turn it on.
The VR has multiple hardware connection
options and it also works with Windows and Linux systems.
As you can see the VR comes standard
with eSATA, FW400 & 800 and USB 2.0 connections and it ships
with eSATA and FW 800 cables. It also comes with an "eSATA
Extender" which basically allows you to connect cables to
the latent SATA ports on the MacPro's motherboard. These cables
are then attached to a PCI slot adapter that has two eSATA connectors.
I've done this surgery before and it is not a trivial task to
install this option, so proceed with caution. The Extender PDF
that ships with the VR describes the process for installing into
a PC and that appears easier than the Mac install, but it's far
simpler to add a regular eSATA PCI card; the one I used for testing
the VR is the e2P from Sonnet.
Since RAID 0 is not a good backup or
redundant workflow strategy I won't spend much time on it, but
if your goal is raw speed here are some speed tests with the
unit empty and configured as a RAID 0 (I used AJA's System Test
Utility for benchmarking):
USB 2.0 connection
My focus for using the CalDigit VR was
not about speed (I would choose a RAID-5 setup for that), but
more so about its redundant storage capabilities. My goal is
to have an instant copy of everything I write to the unit; I
want to have the data twice.
The first order of business is to switch
the unit from a RAID 0 to a RAID 1 configuration. Now, it's important
to note that any change in RAID setup with this unit will result
in total loss of all data on the drives. That's because
the unit has to reformat the drives into the new RAID configuration.
So take great care when changing the unit's setup.
With that admonition firmly in place,
the VR's hardware RAID controller is quite easy to use. You can
either use the front panel display:
Or the bundled software:
The unit ships with the software and
PDF manual on CDROM and when I checked the application installer
using Pacifist it showed
that only the program itself installed, no other drivers etc.
CalDigit's website also has tutorial
movies explaining the use of both the front panel menus and
the bundled software. Either way you go, it's quite simple.
For example here are the steps to change
the RAID set via software:
1. Launch the CalDigit RAID Tool
2, Select the VR:
3. Choose the RAID Configuration Button:
4. Click RAID 1 and heed the warnings:
5. After a minute or so of internal processing
the VR will come back online. CalDigit recommends that you restart
your computer during that time.
6. The OS will alert you that the new
drive is unreadable and so you'll have to format it using disk
If you use the front panel to configure
the VR you'll go through a series of menus and options to chose
the new RAID configuration. There are multiple steps that prompt
you to be sure you want to erase the box and reconfigure it as
a different RAID set.
The first thing you'll notice after formatting
is that the drive is no longer 1.8TB (formatted and empty) it's
approximately 930GB formatted and empty. Everything you now copy
onto this unit will be simultaneously written to two disks, you
just don't see the second disk mounted. As I wrote earlier, you
are sacrificing both capacity and speed with a RAID 1 configuration.
Fortunately, the hardware RAID controller does a good job managing
the data flow, especially in the read department. Here are some
speed results with a FW 800 connection.
The VR was completely empty during these
tests and will inevitably slow (as all RAIDs do) as the unit
reaches full capacity.
The CalDigit RAID Tool also provides
information regarding the installed drives:
And the status of the VR's fans and heat
You can then use the drive as normal
and in the event of a hardware based drive failure, you just
have to swap out the bad drive and the VR will rebuild the Mirror.
The unit comes with two "keys" that you use to open
the back drive doors:
When removing a bad drive make sure to
replace it with a drive of the same capacity and preferably the
In conclusion, the CalDigit VR is a very
good RAID 1 (mirror) solution in both form and function. It offers
ease of use with both its hardware and software interfaces and
I really like how the front panel displays the unit's current
state and status. The price point of the VR is also quite compelling
considering it has a hardware RAID controller and ships with
two drives (price varies with capacity configuration).
With that said, is the CalDigit VR (or
any RAID 1 system for that matter) the perfect backup solution.
The answer is no, because RAID 1 technology can only partially
protect against data loss in a redundant workflow. Although unlikely,
both drives in the VR could fail and result in data loss.
There could be a flood or fire that destroys the box. The unit
could be stolen: all those possible force majeure eventualities,
however uncommon or unlikely, could possibly happen.
The only realistic safeguard against
total data loss is an offsite copy along with local redundant
storage (like CalDigit's VR) that's configured as a RAID 1 set...thereby
ensuring you have the data thrice.
editor and producer, Brian Gary heads Flying Chaucer Films
LLC in Los Angeles. Under that shingle, Gary has directed, produced
and edited commercials, independent feature films, broadcast
television, and new media content. As an expert compressionist,
he consults for the major studios, mini-majors and independent
productions. Brian is also a published author with the Apple
Certified texts on Compressor and QuickTime and he is the talent
Training's Compressor 3: Up and Running.
copyright © Brian
This article first appeared on www.kenstone.net
and is reprinted here with permission.
All screen captures and textual references are the property and
trademark of their creators/owners/publishers.