Spreading media across multiple external hard drives

I'm an assistant who just started working with a new editor. Currently, we have about 3 TB of media spread across two 2TB external hard drives that are daisy-chained. She claims that it would be faster if the media were spread across four 1TB daisy-chained drives. Is there any truth in this? In my mind, the speed does not rely on the capacity of the drive, but rather, on the speed of the drives and the method of connection (which is Firewire 800 in this case).

She also suggested that we put the incoming 1.5 TB of footage on the internal hard drive of the iMac we are using, which has a 3TB capacity. I was always under the impression that the Final Cut Pro Documents (thumbnail cache, renders, etc) should exist on a separate drive than one that also stores media.

And one more unrelated question:
Is it always necessary to merge the video footage with the external audio? I'm working on a two-camera shoot and created synced sequences, so that the editor can copy-paste both camera angles and the external audio all at once when he needs it. I would've merged it if it had been a single camera shoot, but it seemed like this was the easiest way to have all of the footage live and move together, since they are not running on the same timecode.

Thank you!

>She claims that it would be faster if the media were spread across four 1TB daisy-chained drives.

If she is accessing media on all 4 drives at the same time, eg. multicam, and you have one camera angle on one drive, it would be faster. Otherwise, no. Daisy chaining can also be problematic. Not to mention, having media spread out across more drives just makes it a media organization nightmare. Usually I have my media on one fast RAID array. I don't have them on 5 media drives.



>Is it always necessary to merge the video footage with the external audio?

Usually I create new master clips with the synced media (merged clips). This is so that I can cut dual system sound production as if it was any other type of shoot. Eg. Match frame brings up video with useable audio, reveal master clip brings you to a clip with useable audio, selecting a clip will select both the picture and the audio together, etc.. But if it's a short project it doesn't really matter.



www.strypesinpost.com

Thanks for the response! Would you recommend a particular RAID array?

Also, do you have any input about the Final Cut Pro Documents (thumbnail cache, renders, etc) existing on a separate drive than one that also stores media?

If you can afford to have 1 separate drive for renders, that would be great, but usually I render to the media drive.

There are lots of them. Caldigit makes pretty good drive arrays if you are based in the US. [www.caldigit.com]



www.strypesinpost.com

Quote

She claims that it would be faster if the media were spread across four 1TB daisy-chained drives. Is there any truth in this?

not really
maybe if you have A-cam on one drive ad B-cam on another

Quote

She also suggested that we put the incoming 1.5 TB of footage on the internal hard drive of the iMac we are using, which has a 3TB capacity.

no just get another drive.
having it on the internal will make life harder, IMO

in fact, get FOUR more drives, so you can have a backup copy of the edit drives

Quote

I was always under the impression that the Final Cut Pro Documents (thumbnail cache, renders, etc) should exist on a separate drive than one that also stores media.

not necesarily.

i keep my waveforms, thumbnails, and especially my autosave vault on my internal drive,
i render to one of my media drives.
i have often thought about having a dedicated render drive, but have gotten by without that for a long time now.

Quote

Is it always necessary to merge the video footage with the external audio?

pretty much so, yes.

Quote

I'm working on a two-camera shoot and created synced sequences, so that the editor can copy-paste both camera angles and the external audio all at once when she needs it. I would've merged it if it had been a single camera shoot, but it seemed like this was the easiest way to have all of the footage live and move together, since they are not running on the same timecode.

easiest for you, perhaps, not for the editor

problem is, what happens when the editor does a match frame?
a video clip with no audio.
that's pretty useless!

both angles should be synced/linked or merged with the audio.
that's what i would want if i was cutting it.
of course what she wants may be different, you do need to you talk with her about it.
i mean, maybe she wants multiclips, or something quite different to what is "easiest"


my biggest bug-bear with assistants is that they often just go ahead and do things with out asking me how i want them to be done.
in my case, i am fairly well on top of the tech side, so i usually know the best way to approach things.
(if i don't, i ask)
but even non-tech editors will have preferences about how they like to work,
in fact they will probably want things to be the simplest and most invisible.


nick

Thanks, Nick!

Quote
no just get another drive.
having it on the internal will make life harder, IMO

Is this because it will be significantly slower?


Also, in regards to merging clips: If I merge clips in one project file, and bring them over into a separate project file, I know that the media file path is broken, so that I can't match-frame. I wanted to use the "Create Master Clip" function in the Tools window to create new master clips to live in the main project file. Will this create any issues when on-lining?

> Is this because it will be significantly slower?

It will be slower (and in fact, depending on your media specs, some computers' internal drives aren't fast enough to handle multiple HD streams in certain codecs, frame rates and frame sizes). But in addition, dealing with video media is the most taxing operation possible on personal computers. Storing, accessing and modifying video media from a system drive will fragment the drive faster, make software corruption more likely, and affect the computer's overall performance.


www.derekmok.com

Derek, thanks for explaining this!

Getting a RAID array isn't really affordable, since I'm on a super low-budget film. It seems like we'll have about 5 TB spread across three 2 TB daisy-chained external hard drives. The capture scratch + renders will be sent to the first drive, and the thumbnail cache, autosave vault and waveforms will be set to the internal drive.

Is there any better option?

Also, the external drives have exactly the same specs and are all connected via Firewire800. Will the external drive that's connected directly to the computer have a faster transfer rate? Or will they all be relatively the same?

Thank you!

> Will the external drive that's connected directly to the computer have a faster transfer rate? Or will they all be relatively the same?

In theory the drives are supposed to function about the same. In practice, the farther a drive is away from the computer (ie. end of the chain), the worse it will function. There are also some practical considerations. For example, if the drive closest to the computer experiences trouble, the entire chain will be affected. The longer the chain, the more likely you'll have corrupt directories, power issues, etc. And the fact is, the more drives you chain, the more likely somebody will dislodge a cable by mistake and cause your entire chain to become corrupted.

Don't be short-sighted. Think through your workflow and buy the proper equipment. Too many productions are short-sighted and don't listen to editors. They end up spending two to three times more than they would have if they had listened in the first place. Not to mention the cost of missing deadlines, slower progress, and replacement equipment.


www.derekmok.com

And do the complete backup of all your media as these guys suggested. If a drive dies on you, even if you've got tapes on the shelf with the originals, it's a lot of extra work to get up and working again.

Reason I suggest a RAID is because it provides data protection in case of drive failures. A complete back up of the raw camera files goes without saying. But also during the editing process, a lot of media is created, eg. AE renders, temp working files, audio, video mixdowns, project files, XMLs, etc.. A RAID 5 array provides bang for buck both in terms of performance and cost when you compare it buying 2 sets of drives and manually syncing them at the end of every work day.

Stardom:
[www.stardom.com.tw]



www.strypesinpost.com

> Is this because it will be significantly slower?

if you are talking about putting your edit media on your system drive, that is just not a good idea, period.
Derek said it better than i could.

if it wasn't an iMac, and you were talking about an internal drive that is NOT the system drive, then no, it won't be slower, I can in fact be faster than FW800,
BUT even so, having that internal drive locks you on to working on that computer alone,
or at least makes it harder to move around, or bring in a second editor to help out.


i have been working on low-budget films, and while we work on more affordable drives,
our workflow is to have at least one backup.

this allows for redundancy, and a 2-system set-up

so on the doc i was cutting recently i personally had 10TB of storage spread across 4 drives (2x3TB + 2x2 TB), and my assistant had exact the same set-up.
when one of the drives went down, we got in another and duped it overnight off the back-up (it probably took longer)

the film before that was a narrative, with less media.
during the shoot i had 2 assistants, and we all had a full set of the media,
so during the edit i had 3 sets, and when one went down it wasn't replaced.

notice how this is not an IF arrive goes down situation,
it is a WHEN a drive goes down situation.

it is incredibly shortsighted to not have a backup.
oops, forgot to put that in caps:
INCREDIBLY SHORTSIGHTED!

if you are not comfortable about forcing this point on the producers, point them to this thread.

Quote

Also, in regards to merging clips: If I merge clips in one project file, and bring them over into a separate project file, I know that the media file path is broken, so that I can't match-frame. I wanted to use the "Create Master Clip" function in the Tools window to create new master clips to live in the main project file. Will this create any issues when on-lining?

no, that is not correct.
for the record, i don't merge in the browser,
i sync in the timeline, link, and subclip to trim excess audio.

these clips are copied from project to project regularly (from the daily rushes project into the edit project)
and they certainly maintain there linkage.,
and match framing opens the clip into the viewer with picture and linked sound.


you must be thinking of the situation that occurs when you copy / paste a bin and a timeline together.

let's say in your rushes project you have a scene bin of rushes
and the same clips laid out in a sequence.

if all was prepared correctly you can of course "Reveal in Browser" with Shift F

now if you were to copy paste BOTH of these at the same time into the edit project,
that Master Affiliate relationship can be broken.

so the trick is to copy / paste them in separate actions,
OR
just copy one and build the other from it.
so give the editor a bin, and let them put the bin clips into a timeline, (if that's how they work)
or give the editor a sequence, and let them copy the contents into a bin.
(or you do the work while they are not working on that particular project)


this same splitting of the Master Affiliate relationship in a copy/paste action can happen with Multiclips, and their single clip components.

Quote

The capture scratch + renders will be sent to the first drive

see how you go with the render scratch.
send them to the drive with the most space!



nick

Quote
so on the doc i was cutting recently i personally had 10TB of storage spread across 4 drives (2x3TB + 2x2 TB), and my assistant had exact the same set-up.

When you say exact same set-up, do you mean that you had those 4 drives daisy-chained? Fortunately, we do have clones of the drives, so that we can swap out in case of a drive failure.

I totally agree with you. The only problem is that I came in midway through the project, and the configuration had already been set up this way.

Is there a best bang-for-your-buck RAID5 array setup that you'd recommend?

Quote

When you say exact same set-up, do you mean that you had those 4 drives daisy-chained?

yes, i had 4 drives daisy-chained via FW800, my assistant had their 4 (duplicate) drives daisy-chained.

occasionally i'd turn on a 5th drive when i needed to grab some other elements from an older project.


nick