HDV vs AVCHD vs P2 DVCPro HD

Hello, all,

Previously I have produced high-budget theatrical documentaries using the old-fashioned workflow of shooting DVCPro-HD on the Varicam, downconverting to DVCAM, and editing on a single standard def timeline @ 29.97 - in part because our productions include 40-50% archival footage that also comes in at SD 29.97. Then we do a traditional online assembly, color correct, and sound mix at a post-house to the tune of $100K.

I am starting a brand new documentary project that is TOTALLY different - very DIY and VERY low-budget, yet we still hope great content and story will ultimately land us a strong HD festival run and a cable broadcast deal.

I am looking to buy a camera and trying to look ahead to the post workflow - cause it makes a difference. We will be editing in Final Cut Pro on a dual-core 3 Ghz Intel MacPro with 5GB memory (could get more) and an internal 7 TB RAID 5 (currently 4 TB RAID 0 'cause we have 2 mirrored systems, but this project will be edited on one system and we are planning to upgrade with the new 2 TB internal drives if they prove reliable). Graphics card is ATI Radeon X1900 XT 512MB (2 x dual-link DVI). Still running Tiger but we will upgrade before we start this new project. We?ll likely do a fair amount of graphics designed in Adobe After Effects. There will be some SD archival media, but it?ll likely make up only 5-10% of the overall film. We plan to online ourselves, color grade in Color, and probably mix in house in ProTools or possibly Soundtrack.

I am not very tech-savvy... have been reading these forums until my head hurts but am still super confused about my options - HDV, AVCHD, or P2/DVCPro HD. I have concerns about each format and would very much appreciate an assessment from the experts.

This is a verité-style, 85-90% run-and-gun documentary that will be shot largely indoors, in corporate apartments, hotel rooms, meeting halls, school rooms, and other such nondescript locations, using available light only. We won?t be dealing with super fast motion ? some dancing, maybe. Interviews will be informal, off the cuff, and largely unlit. We will be mostly shooting handheld, possibly with a shoulder brace. Some shooting will be outdoors, but this will be the exception rather than the rule. The director and producer will be doing shooting/sound. We expect to shoot 200-250 hours of media and would like to shoot 24P.

Here are my concerns:

1) HDV - I hate the fact that HDV is a dying breed so any camera will be immediately obsolete, but my director likes the 'security' of a tape based workflow. Because of this, HDV seems like our best bet overall, but editors seem to hate it. I love the amount of storage space it takes up but I'm worried about render times, and I also wonder if it'll cause us lots of problems when doing graphics and color? If we edit natively in HDV, are the render times really terrible for basic editing (cuts, transitions)? Is our processor fast enough? If we do our basic cut, then uncompress the timeline for graphics work and color grading, will that solve the problems? Is HDV, when mastered to D5 or HDCAM-SR, adequate for blowup or broadcast?

2) AVCHD - Sounds as if this is a superior format to HDV, but it's not supported natively in FCP and I won't have the storage capacity to use the Pro-Res/ Apple Intermediate workaround. I hear you can use a Blackmagic card to transcode this footage to SD and edit in SD (while storing your AVCHD footage on a separate archive drive). True? If so, what is the 'onlining' process after we reach picture lock?

3) P2/DVCPro HD - All things considered I would much prefer to shoot DVCPro HD 720P/24, but even if I could afford enough P2 cards, I just don't have the storage space to accommodate 1 GB/min... I calculate it would take about 17 TB or more to store 250 hours! Even if I did buy a bunch of internal TB drives and use them like tapes - stored on the shelf in static free bags to be used only when I 'online' - what is the 'online' process? Is it possible to edit P2 media in standard def? How?

Please keep in mind that cost is a very big consideration for this particular project. Can you give me your recommendations for the best format/workflow based on all of the above? I am hugely grateful for any advice.

THANK YOU!

First off, your data calculations for P2 are WAYYY off. the 1GB/min is the TRANSFER TIME...it takes 1 min to offload 1GB of footage from the cards. so 16min for a 16GB card. But if you shoot 720p24PN (23.98), 250 hours will only take up 5.44TB of space.

Yeah, the biggest issue with P2 is the cost of the card. But you can always rent cards when you need them. Cheaply too.

Second biggest issue is the issue facing ALL tapeless formats...archiving. YOu need hard drives to store the P2 masters, and hard drives to edit from. Doesn't matter if you are working with AVCHD, P2, XDCAM EX or RED...same issue.

AVCHD is a far better codec than HDV. MPEG-4 as opposed to MPEG-2...and tapeless and the SD cards are cheap. Same archiving issue, and yes, as of now there is no native support in FCP. But I heard that Apple said that there will be in the next release of FCP...whenever that is.

HDV has issues with long conform times when you are finishing...and longer render times when you add filters (blame that on long GOP). AVCHD is also long GOP, but since you convert to ProRes...to an I-Frame format, that issue goes away. But the same 250 hours of footage, as ProRes (not HQ) is 19TB of space. And currently there is NO WAY to import tapeless formats at a lower resolution when you first import...only able to import, then convert to lower resolution.

MY suggestion is to go with the HPX-170 and rent cards. It comes with one 16GB card, but there are 32GB and now 64GB...and renting them should be pretty easy. But then as with tapeless you need a digital wrangler on set to deal with offloading the cards properly. If you want run and gun with little support, that doesn't require a lot of space, then the only real option you have is HDV.

IMHO.


www.shanerosseditor.com

Listen to THE EDIT BAY Podcast on iTunes
[itunes.apple.com]

Let me just throw one more format into the mix here... XDCAM EX

[www.dvuser.co.uk]

There's also the EX3 which you can look into.


A quick comparison of the codecs:

HDV- 25 mb/s long GOP, 4:2:0, 1440x1080, 8 bits

AVCHD- up to 24 mb/s, H.264 (temporally compressed), 1920x1080, 4:2:0, 8 bits

DvcproHD- 100 mb/s 1280x1080 (1440x1080 in PAL), 4:2:2, 8 bits
DvcproHD 720p24- approx 50mb/s, 960x720, 4:2:2, 8 bits

XDCAM EX- 35 mb/s long GOP, 1920x1080, 4:2:0, 8 bits

You are right. The issue with shooting P2, is that you need the storage. I've seen cases where guys do not understand the storage requirement needed to work with a tapeless 100mb/s format. You need to archive the P2 somewhere, AND store the QT on your framestore. So if you look at it in one way, it's double the storage requirements (more noticeable when you're comparing it to the rest of the codecs which are under 50mb/s). Otherwise, it's a joy to work with. It's the only intra-frame-only format in the list.

I'll avoid AVCHD, unless there's a viable offline/online option or native support. I don't see the point in converting a 24mb/s rushes into a 220mb/s codec unless it's post picture lock. Quality wise, it's definitely much much better than HDV, and possibly slightly better than XDCAM. They're usually tapeless, and tapeless ingest differs from tape, I'm not sure if the blackmagic card you're talking about is able to allow you to do a recapture at full resolution. Let us know.

Hope this helps. The others will chime in...



www.strypesinpost.com

dvcproHD is by far the easiest of the formats discussed to work with, and drive space is cheap.

i suggest you start here:
[www.caldigit.com]

If only there were some way to shoot DVCPRO HD to some inexpensive, durable format. Maybe something magnetic, like some kind of ? I dunno. Tape or something.

Oh, if only!

Yeah, but then you are talking about the $65k Varicam cameras...Which are great...but not cheap.

VERY rentable.

OH...and if you get the HPX-170 you have other options to P2. FIRESTORE is one very good option. Being a hard drive you have more record time (60GB hard drives), it costs as much as a 16GB P2 card (a bit more maybe), but aren't as durable. Drop it...jiggle it...extreme heat/cold...info will be lost. But it is still a great alternative.


www.shanerosseditor.com

Listen to THE EDIT BAY Podcast on iTunes
[itunes.apple.com]

Yeah, Shane, I know. I was just sarcastically bemoaning the trend toward tapeless cinematography without regard for workflow scalability. One guy, one camera, one CF card? Not a huge deal. But as you scale up, the IT issues become significant.

It's just a shame, is all.

Maybe not run'n'gun worthy but we're trying out a system of e-sata dock and naked 1TB sata drives for P2 backup.

Dock connects to MBP via FW800 (also USB, FW400, esata) Commodity prices for 500GB or 1TB drives (at the moment - Hitachi, used to be Seagate). This enables cheap but useable in a pinch offline storage leaving more money for edit capable storage (SATA array). We're trying to source a good static free storage box for the naked drives that, once transfered to edit system get removed to off site location and slower back up to even cheaper USB drives and (experimental use of) a DROBO.
First day was a cluster**** of misnamed folders and other laziness. Got that sorted and so far everything is organized coming from set.
HVX200(1 or 2 units ) has no TC in so second system audio (Sound Devices 744T, 1 or 2 units) relies on visible TC slate for sync.

Redundant offsite storage of two copies of verified P2 folders
One copy of imported media on edit system
Multiple redundant(offsite/web/flashkey) copies of FCP projects and settings
Minor finger crossing...

ak
Sleeplings, AWAKE!

i do a lot of run and gun stuff with my hpx170, 3, 16 gig p2 cards and just have a production assistant swapping cards to my old powerbook and a connected firewire800 drive all day.

easy peasy.

You guys are fantastic!

So psyched to be wrong about the storage space issue with DVCProHD. Under 6 TB for 250 hours? Sweet. Just to be safe... how much footage - shot at 720p24PN (23.98)- can a 16 GB P2 card hold? About 40 minutes?

I'm still worried about not having the necessary field support, but those new specs make P2 a lot more attractive. If it were just me I think I'd go for it, but I'm not sure my director will. Unless I can find p2 cards SUPER cheap for rental that option doesn't make sense for us, as we'll be shooting on & off, unpredictably, for several months.

If I do go HDV (at my director's urging)... will my processor be able to easily handle it? You think I'll get into trouble with the graphics/color grading? If I plan to 'upres' to uncompressed or ProRes after picture lock, will that convert the entire timeline to a valid intra-frame format that will circumvent graphics/color issues? Given the constraints of this particular project (in which gorgeous visuals are unlikely even if we were shooting on the best camera in the world), is HDV really such a terrible, terrible format?

Re: AVCHD... it's a big bummer FCP doesn't support it properly yet!

Again, THANK YOU for all your insight!
Thanks for the note about the XDCAM. Unfortunately the E3 is way beyond my budget... I've drooled over it... We're looking at cameras in the $3000-$5000 range.

im halfway considering selling my three 16 gig p2 cards if youre interested.

>how much footage - shot at 720p24PN (23.98)- can a 16 GB P2 card hold? About 40 minutes?

42min...good call!

>f I do go HDV (at my director's urging)... will my processor be able to easily handle it?

Yup

>You think I'll get into trouble with the graphics/color grading?

Render out as ProREs...that is an option in the Timeline settings under...something. Sorry, on Avid now.


www.shanerosseditor.com

Listen to THE EDIT BAY Podcast on iTunes
[itunes.apple.com]

>Render out as ProREs...that is an option in the Timeline settings under

Switch timeline to ProRes HQ and render (The Mees Method), or MM out to ProRes HQ. Think we had a fairly lengthy discussion on GOP formats a few threads ago..

If you're able to go DvcproHD, go for it. Hold up MUCH MUCH better than HDV..



www.strypesinpost.com

Just to toss out another perspective.

Shoot with Sony EX series.

XDCAM EX is 35mbps so storage is reasonable.

EX series can record to Transcend or Sandisk 16GB SDHC cards. $25-$40 each depending on where you get them. Cards hold 57 minutes each so camera is loaded with 2 hours. Offload ranges from 2x to 4x real time depending on whether you have CRC on or off in Sony ClipBrowser. Of course at those prices some would even consider the media as archival (compared to cost of XDCAM disc for example especially if you can get $25 Transcend cards). You can shoot with Sandisk 32GB for nearly 4 hours in camera but they're about $150 a card. All this is way cheaper than P2 though. Transcend and Sandisk SDHC cards are readily available at consumer electronics stores if you need to buy more. SxS cards are only needed for overcranking.

FCP supports XDCAM EX natively. Set sequence to render in AppleProRes and there's NO Long GOP Conform at all. You don't need to convert files to Pro Res. Just set Sequence render settings.

EX series has full 1920x1080 chips and records at full 1920x1080 not 1440x1080 (or 1280x1080) if you're shooting 1080.

Discovery Channel has given EX silver status so that's a reasonable "seal of approval" for broadcast doc work.

Okay, I just added the XDCAM codec to my subject line 1/2 inch sensors is a big deal and the EX1 isn't really that expensive! I'm surprised.

Just to be totally clear - when you say 2 to 4x real time, craig, you mean FASTER than real time, right?

Also, Sony's specs don't say anything about SDHC media (only SxS) so is that definitely an option with the EX1?

And, XDCAM, like HDV, is based on GOP and MPEG compression - why is it so much better than HDV? Because of the bigger sensors?

I don't really understand bitrates - will have to do some more reading up on that. So, can you hit me up with an estimate of how much storage 250 hours of XDCAM media will take up?

Many thanks...

8 bits = 1byte

[www.pairofhands.net]



www.strypesinpost.com

Just to be totally clear - when you say 2 to 4x real time, craig, you mean FASTER than real time, right?

He does .. 2x-4x faster than realtime

Also, Sony's specs don't say anything about SDHC media (only SxS) so is that definitely an option with the EX1?

It is not "explicitly" supported but that isn't stopping many many folks from doing it, manufacturers from developing supporting products etc ... see here:
[e-films.com.au]




And, XDCAM, like HDV, is based on GOP and MPEG compression - why is it so much better than HDV? Because of the bigger sensors?

Also 40% higher bitrate (variable) + full raster resolution

I don't really understand bitrates - will have to do some more reading up on that. So, can you hit me up with an estimate of how much storage 250 hours of XDCAM media will take up?

About 3.75 TB

My 2 cents.

This is the perfect storage device for Panasonic P2 workflows. Panasonic sent many people to the CalDigit booth at Macworld as they know the value of this product with that workflow.

Interview at Macworld with Jon on CalDigit VR:
[tinyurl.com]

Included with the CalDigit VR are a eSATA extender, with 2 ports, a FW800 cable & a eSATA cable.

and the product page:
[www.caldigit.com]

The 2TB version of this product is NOW SHIPPING, $699.00 Street pricing varies under that.

Tomorrow is your last day to register to WIN 1 of 10 CalDigit VR 500GB, click here to enter! [tinyurl.com]

And to confirm some points about Andy's response,

I am using the e-films MxR adaptor with Sandisk 32GB cards so first hand 2 to 4 times real time transfer (which is actually about 1/2 the speed SxS is). Transcend and Sandisk cards have been proven reliable. JVC's new HM cameras use the XDCAM EX codec and record to SDHC cards.
The bit rate for XDCAM makes the codec very hard to break unlike HDV. HDV 25mbps CBR can't come close to the robust nature of 35mbps VBR. Again Discovery Channel has given the EX Silver Status.

Hi again everybody!

After some more research and in direct contradiction to a lot of the advice I've been getting from this forum , my director for this documentary is now leaning toward the AVCHD camera. (Price, price, price.) So I'd be grateful for your thoughts on the space issue re: transcoding to ProRes and some possible workarounds.

1) If I recorded 250 hours of AVCHD, how much space would it take up native? How much when transcoded to ProRes?

2) What Mac-compatible NLEs can edit AVCHD natively? I read in here that Adobe Premiere 4.0 can - is that true? Not to be a 'preditor' but could a reasonable workaround be to pull selects in Adobe and then export just the selects to FCP, to be transcoded for further editing? Has anyone actually exported OMFs from Adobe to FCP, and does it work seamlessly?

3)If FCP 7 were to come out mid-production...and if FCP 7 includes native AVCHD editing... would it be possible to make the clips offline, blow away the transcoded ProRes AVCHD files, and reconnect to the original AVCHD files? In other words, do you think it would be possible to convert the project back to native AVCHD?

4) Will my processing system be able to handle AVCHD natively? Again, it's a dual-core 3 Ghz Intel MacPro with 5GB memory (could get more) and an internal 8 TB RAID 5 (planning to upgrade with the new 2 TB internal drives at RAID 5 or 6 if they prove reliable). Graphics card is ATI Radeon X1900 XT 512MB (2 x dual-link DVI).

I realize some of these questions are asking you guys to speculate but I can't think of any people whose opinions I'd value more.

Thanks!

>1) If I recorded 250 hours of AVCHD, how much space would it take up native? How much when transcoded to ProRes?

There is no native support for AVCHD. You have to choose between AIC (yuck) or ProRes. And that will eat up space. AJA.com has a data rate calculator you should download.

2. How could spending extra money on another NLE save you time and compression when working with this format? No...get Toast 9 or 10...use that to compress to DVCPRO HD. Much lighter codec.

3. No clue...no idea what FCP 7 has in store. But I can tell you that If you import P2 DVCPRO HD using the L&T tool...edit...then blow it away and install MXF4QT and try to relink to the original MXF files...you can't.

4. Again, no native support with FCP. Premiere I think yes...but is that really a reason to buy a NLE? native support? I prefer function myself. Avid does AVCHD too I think...I'll have to test. But that system can do ProRes fine...just need LOTS OF STORAGE.


www.shanerosseditor.com

Listen to THE EDIT BAY Podcast on iTunes
[itunes.apple.com]

Firstly, I would advice you not to build your workflow around products that are still in development, especially around native AVCHD editing in FCP, because more likely than not, you'll be stuck editing either ProRes or in finishing in *yuck* Premiere.

As far as FCP editing of AVCHD, the reason why I don't like it is because it takes up around same amount of storage as working Uncompressed SD, even if it's cheaper for production.

For cross platform work, the general word is not to plan for that. Capture and edit off the same NLE. I'm not too keen on Premiere. Avid would be what I'd go for. Not sure if it handles AVCHD these days, and i'm not sure if getting an Avid suite fits into your budget either.



www.strypesinpost.com

Good advice, as usual. You guys are great.

So, say cross platform is out. That means either FCP (which we have and I know, at least a little) or Avid (which we'd need to buy and I'd have to learn). My ignorance regarding Avid is immense. No idea how to capture/import footage, don't know anything about the interface... etc.

But I'm willing to learn. My workflow ignorance with Avid is such that I don't even want to start badgering you guys with questions, especially since this is an FCP site. Do you recommend any Avid forums?

I WOULD love to know (as a start) whether Avid Media Composer software or even Avid Xpress supports AVCHD natively. Based on what I could glean from their website, they don't.

Shane, re: your advice to get Toast and compress to DVCPro HD... how does this compare to using ProRes in terms of quality? If I shot 720/24p, or even 1080/24p, did my edit at DVCProHD (which I will have sufficient storage to handle), how would I proceed in terms of final finishing... would I then transcode to ProRes or Uncompressed before sending to Color?

Also, please forgive my incessant requests for space estimates. I looked on the aja.com site and couldn't find the calculator to which you were referring... but would love to find it. I am kind of a bonehead re: data rates.

Thanks again for everything...

Avid forums:

[community.avid.com]
[forums.creativecow.net]

> re: your advice to get Toast and compress to DVCPro HD... how does this compare to using ProRes in terms of quality?

It will be more compressed than ProRes...and 8-bit vs 10-bit color space...but it will still look decent. If you have the large RAID that can handle ProRes...go for it. But if you are on a budget, then you should look at DVCPRO HD.

>If I shot 720/24p, or even 1080/24p, did my edit at DVCProHD (which I will have sufficient storage to handle), how would I proceed in terms of final finishing... would I then transcode to ProRes or Uncompressed before sending to Color?

Finishing for what? DVD? HDCAM? Either DVCPRO HD or ProRes are sendable to Color. My DVCPRO HD shows are sent to Color, and come back rendered out as DVCPRO HD. ProRes would be better...but again...do you have a large raid to handle this format?

720p 23.98 ProRes is 31GB/hour
1080p 23.98 ProRes is 61GB/hour

720p 23.98 DVCPRO HD is 22GB/hour
1080p 23.98 DVCPRO HD is 48GB/hour.


www.shanerosseditor.com

Listen to THE EDIT BAY Podcast on iTunes
[itunes.apple.com]

Quote
my director for this documentary is now leaning toward the AVCHD camera. (Price, price, price.)

Given what others are saying about AVCHD I'm not sure how you're saving money. Transcoding is time. Storage needs can be a factor too especially for a documentary. Time is money in most businesses. It sounds like AVCHD is an economic sink hole. Maybe another NLE but Panasonic's sheet on this says:
FCP to ProRes
Canopus to HQ
Premiere Pro CS3 MainConcept to DVCProHD
Pinnacle Studio Plus 11 Native
Ulead Video Studio Plus 11 Native
No mention of Avid or CS4

FTP AVCCAM NLE Compatibility

Frankly, if you're going the DvcproHD route at all, and taking both production and post costs into consideration, it may be more feasible to shoot DvcproHD. Firstly, you'll be transcoding a compressed 25 mb/s signal into a compressed 100 mb/s signal, just to get a workable intermediate codec.

Then, the transcoding, file naming/media management will be kind of a dance... How you transcode or name the files will depend on what kind of source you start from and your workflow. FCP batch captures and re-link files with timecode and reel names (file names to re-link), on tapeless formats with clip UIDs and reel names. With most transcodes, they do not keep the UIDs or timecode. So depending on the workflow, you may need a back up of the DvcproHD transcodes which you can then re-link back to as a contingency.

Avid MC3 with at least a Mojo for monitoring.. Nitris will be better, but you'll need bucks for that.



www.strypesinpost.com

Argh. This is sounding like a nightmare.

Is there any transcode that DOES preserve the UIDs and reel names?

Time is money, yes, but nobody on this job is getting paid (at least not yet)... Also, on the HMC150 I meant the best price for the best visual quality, not best price overall.

If AVCHD were easy to edit, this would be an easy decision. If I could afford P2 cards, it would be an easy decision. As it is, I am about to throw up my hands and go back to HDV - our initial impulse - whether via tape or tapeless. Leaning toward the Sony Z7 for this.

Which brings me back to my earlier questions about HDV: If we edit natively in HDV, are the render times really terrible for basic editing (cuts, transitions)? Is HDV, when mastered to D5 or HDCAM-SR, adequate for blowup or broadcast? Overall - given the constraints of this particular project (in which gorgeous visuals are unlikely even if we were shooting on the best camera in the world), is HDV really such a terrible, terrible format?

I have learned so much from this thread. Thank you.

Quote
Time is money, yes, but nobody on this job is getting paid (at least not yet)... Also, on the HMC150 I meant the best price for the best visual quality, not best price overall.

Ah, OK. So why not consider the new JVC HM100 or JVC HM700?
HM100 is 1/4" chips though but it's $3999
HM700 is 1/3" chips and it's shoulder mount. Looks like the replacement to the JVC 250.
Both HM cameras use the XDCAM EX 35mbps VBR codec. They record to .mov on SDHC cards so it's simple drag and drop in FCP.
JVC HM100 promotional video
It seems like a good budget compromise for what you're doing without going through codec hell.

This video may be different
HM100 Primer

I was looking at the JVC HM100 as a good option - love the easy access to FCP and the price - but the folks at DVinfo keep saying it's not a good choice for shooting indoors in available light and that's pretty much all we're going to be doing on this project. I'll check out the HM700 but imagine it'll be too big/bulky for what my director wants.

I feel as if I'm in a Shakespeare play... every camera has some fatal flaw! I guess there is no perfect solution at this budget level.

Its not even out yet. How would they know its no good in low light? Yeah sensor size says something but who knows what JVC has up its sleeve here.

Michael Horton
-------------------