In the world
of digital video, great amounts of data must travel in a "sustained"
stream in order to avoid losing frames, audio, etc., during both
the capture and playback process. The amount of data moving between
the computer and the hard drive(s) must maintain a high enough
level that it never "dips" to a capacity that forces
the system to "drop frames" or audio as it races to
keep pace with the data being processed through the video capture
A good way to think of this would
be to picture your video capture card as throwing baseballs and
the harddrive as the catcher that must catch them. If the video
capture board in your computer starts throwing more balls than
your many-armed catcher can catch -- then balls are going to
be dropped. That's what's happenning as data is pumped into your
system and your drive(s) must have enough "hands and catcher's
mitts" to avoid dropping frames and data.
In the early days of digital video on
the desktop, this phenomenon was addressed in what were called
"AV Rated" or "Sustained Rate" drives. These
drives were capable of supporting these high bandwidths needed
in the process of desktop video. Today, you don't see those terms
much anymore. By the time you finish this article, you'll know
why and also the changes that have made drive prices drop, capacities
sky-rocket and overall performance curves to swing strongly in
favor of the user.
Over the years, on all of the lists and
forums I frequent, I have heard one new video editor after another
ask why their video is dropping frames. After all, they bought
a huge drive and "Man, it's 20gigs! So why isn't my system
working? I get a couple of seconds of good video and then it
starts dropping frames!"
They rarely go far enough in their diagnostics
to narrow it down to the drive throughput and usually they blame
their software: "Premiere sucks!" Not to pick on Premiere
here, it's simply the largest selling video editing software
out there and so it catches the most blame. But most capture,
playback and drop-frame problems are solved by resolving "throughput"
issues, not software problems. And throughput requires balancing
a computer's hard drive(s) and the controller that negotiates
between the computer and the drives.
we are dealing with basics, please be advised that using your
base "C" drive (or the "system" drive for
Mac users) for video storage and playback is asking for trouble.
You need a second drive (or more) for video. This drive is dedicated
to video and audio files. Playback will be much more reliable
if you are not trying to use your C drive for this purpose.
the world of DV, with its data-rate of just 3.6mbps to maintain,
even a single IDE drive will usually handle the job quite handily.
For more on EIDE drives in DV, read Kathlyn Lindeboom's article at the Cow.
Throughput problems are much easier to resolve in the world of
DV than in the analog world of Motion-JPEG (M-JPEG). In this
world, all the rules change. And playing by the rules of M-JPEG
has traditionally meant high-priced SCSI drives along with expensive
SCSI controller cards like those from ATTO and others.
But back in 1997, all that began to change
when drive systems started appearing in the market that were
built around low-cost off-the-shelf IDE drives -- usually in
an enclosure that used SCSI firmware to exchange the data between
the computer and the drive. By creating a drive system that used
multiple IDE drives in an array, users could get SCSI-level performance
from far-cheaper IDE drives. Some of these systems, like Medéa's
VideoRaid systems, use a built-in hardware controller card that
automatically recognizes this array of multiple IDE drives working
together as a single drive. No special software needed! The race
to change the marketplace was on.
Today, most of the SCSI drive manufacturers
are now little remembered footnotes in computer history -- casualties
of how well the idea of using IDE arrays has changed the market
In the early days of NLE systems back
in 1994-5, users had to pay approaching $4,000 or more for 9gigs
of AV-rated SCSI storage. That has dropped (at the time of this
writing) to the point where 36gigs of UW SCSI sells for about
$500. But a 40gig Ultra ATA/66 IDE drive sells for around $125
or so. Not only is there a substantial cost advantage to using
IDE drives but SCSI drives operate at heat levels that exceed
their IDE counterparts -- especially important when considering
a system that uses internal drives. With issues like cost and
cooler performance to consider, these differences are fueling
a move to IDE arrays.
HOW RAID WORKS...
RAID ARRAYS: What Are They?
In the world of personal computers, hard
drives are recognized by an individual ID such as the "C"
drive, "D" drive etc. The individual "behaviours"
of these drives are based on their speed, drive type (SCSI, IDE,
1394, etc.), etc., and you cannot break the limits of their individual
"throughput ceilings" without the system "bottle-necking"
and giving you errors like dropped frames, etc. Since the drive
can only handle as much data at any one time as it was designed
to handle, then your drive must "grab what it can and lose
the rest" if you exceed that ceiling, This is what is happening
when you "drop frames."
In a RAID system -- which stands
for: Redundant Array of Independent Disks -- you can combine
multiple single drives into a shared system wherein each drive
carries only a part of the overall load. This allows a user to
combine multiple lower throughput drives (like low-cost IDE drives)
into a system that looks like a single drive to your system and
has a total throughput high enough to handle video, etc. This
is known as an "array" of drives. Using this
arrayed scenario, your computer now sees the "array"
of drives as a single drive and will treat it that way when digitizing,
etc., from your digital video system.
For example's sake: Let's say that you
have a hard drive with a sustained rate throughput of 9mbps.
This would be high enough to handle DV footage which is less
than 3.6mbps -- but you could not work with many of the M-JPEG
analog formats that require much higher throughputs.
But by using an array of four of these
drives -- each drive pulling 9mbps -- they would now yield an
aggregate throughput of 36mbps sustained (4x9=36). This would
allow many more M-JPEG formats to be used. Uncompressed video
takes over 20mbps sustained for a single stream. And as stated
earlier, if you are using DV as your primary format, even a single
drive that can maintain speeds in excess of 3.6mbps is sufficient
for the 5:1 compressed nature of DV files.
In an IDE RAID system you can use up
to four drives in the array. In SCSI, you can have up to 15 drives
in the array.
the advantages are obvious when you consider the positive aspects
of using arrayed drives -- there is a serious downside to this
technology and it is this: If you lose one of the drives in the
arrary, you lose your project. You cannot usually recover your
project from just the remaining good drives. Companies like Medéa
have released new RAID3 systems that combine video array performnace
(known as RAID-0) with a redundant "mirroring" protocol
(RAID-3) that protects valuable data.
In computer "geekspeak," another
factor to be aware of is the Mean Time Between Failure (MTBF).
With an array of drives, the MTBF will equal the MTBF of the
lowest-rated individual drive, divided by the number of drives
in the array. If the MTBF is too low for your liking or confidence,
you can always look at the "mirrored" RAID protocols
like RAID-3 and RAID-0/1.
Here is a basic look at RAID types applicable
to video and their purpose:
||This is what
you'd use for digital audio/video use. It basically takes multiple
individual drives amd ties them into a single drive "array."
It offers no verification or "fault tolerance."
your computer as a single drive but is actually two drives, one
an exact duplicate of the other. This realtime "mirroring"
allows for data protection in mission critical operations in
fault-tolerant uses like banking, government, business, medicine,
allows for the speeds needed for digital video in the RAID-0
portion of the system and includes what is usually a near realtime
RAID-1 system for redundancy and data protection. Requires a
minimum of four drives to acheive: two as the primary data RAID
and two as the mirror drives to create the exact duplicate of
the primary drives.
||This RAID level
allows for the data drives to be striped and includes a back-up
disk for data protection. Requires at least three disks to implement.
Controller design is a complex procedure and the resource intensive
nature of the process makes it very difficult to accomplish with
SOFTWARE STRIPING OR FIRMWARE STRIPING:
Usually, RAID systems require some form
of drive "striping" software to create an array from
multiple drives. The most popular over the years have been Charismac's
Anubis and RAID and REMUS for Mac -- and today, Windows NT
and Windows 2000 have striping as a function of the OS's formatting
options and W2K tools like RaidToolBox take things even further in options
and functionality. Other varieties of Windows will require some
form of software utility or a hardware card that automatically
creates the solution in firmware. Software drive formatting utilities
like those just mentioned, do things invisibly for users that
used to have to be done with arcane and cryptic codes. Today,
tools like Anubis not only stripe the drives together but automatically
set the drives for maximum performance for digital video.
The basics of "striping" are
understood by the following points:
- Multiple drives are combined to form
a single more powerful array;
- These drives are then "interleaved"
by setting the sectoring value;
Sectors can be as small as 512k and as large as several gigs;
- Smaller sectors mean the read/write
heads will access "stripes" across drives more frequently;
- Larger stripes mean less frequent interleave
sectors and a less efficient "striping" interleave;
- Digital video and audio should therefore
use smaller sectors not larger ones.
New systems like the Medéa drives
(as mentioned earlier), need no special software as they use
a firmware system inside the drive enclosure. New cards such
as the Promise
RAID controllers also do the same and control multiple IDE drives
from a single PCI board.
You need at least two drives in order
for a RAID system to work and "No, you can't use partitions
on the same drive." (In case you were wondering!) A card
like a Promise RAID controller goes into your computer's PCI
slot and once there, your computer will recognize it as a SCSI
device. (Windows users note: This is as long as your BIOS settings
are as outlined in the support documentation.) These cards come
in two and four drive configurations and as such, double and
quadruple your drive performance.
Lastly, let's say that you put two 5,400rpm
20gig drives together with two 5,400rpm 6gig drives. Do they
then become a 52gig array??? No. They will all format at the
level of the lower capacity drive. So, if your lowest drive is
a 6gig, all your drives will format as 6gig drives even your
20gig drives. So make sure that you plan correctly or you may
find yourself wondering why with 52gigs in the capacity (as in
the previous example), you are only seeing 24gigs.
Feedback? Write Ron Lindeboom
©2001 by Ron Lindeboom
and creativecow.net. All rights are reserved.
This article first appeared on creativecow.net and is reprinted here with permission.
All screen captures and
textual references are the property and trademark of their creators/owners/publishers.