Media File Sizes

I've had a good prowl around the forum but can't find anything directly bearing on this.

I'm just getting into a project which involves 3 to 4 hundred hours of archive material on U-matic, 1", 2", BetaSP and Digibeta. It's likely that we will be digitising the lot, upscaling to 1080 through a Snell & Wilcox standards converter. Probably as Prores HQ.

Roughly half the tapes are 30 minuters, the rest 90's.

According to Digital Rebellion's marvellous new iPad app (Video Space), 30 minutes of 1080 Prores HQ will make a file of about 37 GB if digitised in one gulp. The 90's will be just over 110 GB.

My question is whether there are any issues for FCP 7 in dealing with such huge file sizes ?

Would there be any advantage in using the "Limit Capture/Export file segment size" technique Jude mentioned in this post a few days ago ?:
[www.lafcpug.org]

The media will be on 12TB e-SATA RAID 5 drives. I'm working on the basis that each drive will be capable of holding about 95 hours (12TB RAID 5 gives 9TB capacity. Allowing for 20% overhead, usable space is 7.2TB.) Any comments on that ?

As always, thanks in advance.

Mike

Nope - just finishing a feature film in 1080p24 ProRes 4444 from the Alexa and 1h48 comes in around 200GB and at much higher data rate than 29.97 ProRes HQ.

You are working well within the limits as long as your drives are running correctly and supplying the bandwidth for however many streams of video you want at any one time.

Personally I'd save space and overheads and digitise the uprezed SD to plain old ProRes 422 HD as you really won't see the difference in the quality from those sources it will be so minor.

Sure use ProRes HQ for the graphics and mastering but unless you are acquiring really high quality images from film or high resolution digital ProRes HQ will be wasting HDD space.

I'm also working on a BBC series in which we have mixed formats and I get the Graphics done at 4444 (with alpha where needed) or ProRes 422 HQ (where alpha not needed). The offline timeline is ProRes 422 but that will simply be changed to ProRes422 HQ for mastering and grading in DaVinci before output to HDCAM SR 422.

Most of the footage is XDCAM HD 422 50Mbps, ProRes 422 and up-converted archive from various SD sources and thats all done to ProRes 422.

Hope that helps.



For instant answers to more than one hundred common FCP questions, check out the LAFCPUG FAQ Wiki here : [www.lafcpug.org]

...and to add

Sme of our archive, converted and captured ProRes 422 files are several hours long and always less trouble than XDCAM at lower data rates and file sizes.

ProRes is lovely to edit with. Don't fear the format. Fear the time/cost to digitise



For instant answers to more than one hundred common FCP questions, check out the LAFCPUG FAQ Wiki here : [www.lafcpug.org]

Agree. We routinely work with large files without issues, but the quality of your source files means that upressing too far is wasted space. I'd go ProRes SD unless there was a lot of grading or FX to come.

So, no, no advantage in breaking up the files, unless they need to be moved somewhere that can't accept files over 2 or 4GB.

Of course, there may be *organising* reasons to break up the files - like several different stories on the one tape, or scenes that you'd like to break out on their own so you can find them easily during the cut.

Quote

Would there be any advantage in using the "Limit Capture/Export file segment size" technique Jude mentioned in this post a few days ago ?

I've not used this technique for a long time but you could if you felt you needed to.

One aspect is the likelihood of media corruption - if your quicktime got damaged as one huge file then its a lengthy re-digitse - however if you broke it into smaller segments then you need only deal with the section thats corrupt.

Either way I've not had any issues for several years so I don't often segment my files.

However breaking up files into smaller sections can help if you are going to transfer to MSDOS PC formatted FAT-32 drives with 4GB file size limits.

But if going to PC formatted NTFS* there is no limit so you can transfer to PC Avid or other for whatever reason if needed.

*To write enable NTFS you need to enable it [hints.macworld.com] or install an NTFS writer such as Paragon NTFS for Mac OS X [www.paragon-software.com]



For instant answers to more than one hundred common FCP questions, check out the LAFCPUG FAQ Wiki here : [www.lafcpug.org]

Crikey, Ben ! You're up late.

But - many, many thanks for all that. It's always reassuring to get opinions on these sort of things before setting off into uncharted waters.

I especially liked the bit where you answered a question I hadn't directly asked (Prores HQ or not). I had been wondering about that.

Yes - the cost to digitise all this is going to be astronomical. However, it's a unique and extremely valuable archive which has already earnt it's owner very large sums of money. It's likely to continue doing so and I think it should be future-proofed as much as possible.

cheers

Mike

Haha the Aussie and Brit dual pronged attack of the info!


The only thing I would say about upscaling with a decent hardware scaler is that it will save you a lot of rendering later and the quality is usually tangible compared to FCP upscaling.

So do you save HDD space and uprez in FCP or pay for the HD upscaling of all the footage?

Unless you think there is a lot of mileage in upscaling all the archive for future use or across many films then you could simply get all the archive in SD, there-by saving money and then once you get to online you could re-digitise the archive selects and upscale with hardware upscaler if you felt the quality was worth the expense.

My advice is always to test quality by comparison before creating a huge spend.



For instant answers to more than one hundred common FCP questions, check out the LAFCPUG FAQ Wiki here : [www.lafcpug.org]

Ah I posted before you answered! Yep up late doing a bit of CSS3 and HTML5 reworking of an old client site to make way for updates later this week!

Well if its for future archive purposes then I'd suggest getting ProRes 422 HD for all but any film stock via a decent Hardware upscaler as you mentioned.

This can then be re-purposed for whatever delivery you require.

The only other issue - to crop or to pillar box 4:3 material?

My preference is to get centre cut pillarbox so you have the 4:3 footage centred on the 16:9 HD image with black bars left and right.

You can rescale (there-by cropping top and/or bottom) and re-rack the image for a full-frame 16:9 look in the NLE.



For instant answers to more than one hundred common FCP questions, check out the LAFCPUG FAQ Wiki here : [www.lafcpug.org]

Ben King Wrote:
-------------------------------------------------------
> Haha the Aussie and Brit dual pronged attack of
> the info!

Tell me about it ! We've all been posting on top of each other. Great, though.

As you both know, I've been banging on about this for several months now. I appreciate the patience.

Jude, I think the decision has been made that we will upscale from the original tapes to pillarboxed 16:9 HD.

Yes - we could stay in SD and re-digitise in online. The reason I'm reluctant to do that is simply that these tapes are all very old and, I suspect, becoming very delicate. We may only get one chance to play them again.

Also, given the sheer number of man/machine-hours it's going to take to do the digitising, the cost of the extra drives is minimal.

>The only other issue - to crop or to pillar box 4:3 material?
>
>My preference is to get centre cut pillarbox so you have the 4:3 footage centred on the 16:9 HD >image with black bars left and right.

This question still hasn't been decided for the final deliverables. I've tested some of the very old stuff blown up and cropped to 16:9 and it becomes almost impossible to decipher the image at all. Be great for a 1970's experimental film, but...

Plus the footage is all about yachts - objects which fit nicely into a 4:3 frame.

Also - here's a curious thing that's got me scratching my head. I've had an upscaling test done through the S&W and also digitised the same tape myself, upscaling with a Matrox box. Lined up alongside each other, the Matrox version actually looks better. Way less noise. How can that be ? As the owner of the S&W says: "How can a $1,000 box do a better job than one that costs $70 or 80,000 ?"

In the meantime, thanks again for answering my initial question.

Mike

Mike Hardcastle Wrote:
> The media will be on 12TB e-SATA RAID 5 drives.
> I'm working on the basis that each drive will be
> capable of holding about 95 hours (12TB RAID 5
> gives 9TB capacity. Allowing for 20% overhead,
> usable space is 7.2TB.) Any comments on that ?

Just re-checked these figures and I see I made a mistake.

At 1080 Prores HQ the 7.2TB would be enough for 62 hours.

At 1080 Prores 422 the 7.2TB would be 100 hours.

My preference for archival footage would be to not lose any part of the frame, since once it's gone, it's gone. It may be in the future someone wants to examine the top of a mast (or something, you get my drift) and if you've cut it off trying to conform it to a 16:9 frame, and there's no going back. And also, like you say, the loss of quality is palpable. You don't even really notice pillar boxes after a while. I think it's the way to go - 4:3 pillar boxed inside a 16:9 frame.

Jude Cotter Wrote:
-------------------------------------------------------
> My preference for archival footage would be to not
> lose any part of the frame, since once it's gone,
> it's gone. It may be in the future someone wants
> to examine the top of a mast (or something, you
> get my drift) and if you've cut it off trying to
> conform it to a 16:9 frame, and there's no going
> back. And also, like you say, the loss of quality
> is palpable.

My thinking exactly.

Plus - an across-the-board generic central crop would be hopeless. In many cases it would result in a frame filled with nothing but sails. If we do end up going for full frame 16:9, pretty much each and every shot will have to be re-positioned vertically.

>You don't even really notice pillar
> boxes after a while. I think it's the way to go -
> 4:3 pillar boxed inside a 16:9 frame.

Yes. My preference is for black borders on either side, but we may end up going for some kind of graphics. Maybe even the old "pull-focus" effect you talked about many years ago

[www.kenstone.net]