Benefits of using Pro Res as my render codec?

>Andy, are you saying capture HDV, edit on a HDV sequence with the ProRes render codec?


Thats what works for me (although for others capture HDV and edit on a ProRes sequence is the preferred workflow).

Bur workflows don't come out of thin air Casey ... experiment for yourself. If you find that the ProRes adapted workflows don't offer any advantages with your system then don't use then just because you think you should. Not every setup will work so well with ProRes settings ... I don't know your system details, but if you are storage space or storage speed challenged, and/or if you are not working with an Intel Mac then it may well not be the best workflow for you

Hi Casey, from a quality perspective, you can't get much better than that. Also, the one thing I don't recall you mentioning is which ProRes? Since you captured from HDV and I assume your not outputting to film you don't need ProRes HQ (220Mb).

Do you have two FW800 striped together in a RAID0 or two independent FW800's for more capacity? If your editing ProRes 140Mb FW800 should be capable of sustaining one or two streams. Remember that is almost equivalent to editing uncompressed SD.

Also something else to check, and sorry in advance I'm not at a FCP workstation to check this, but I believe when you convert HDV to ProRes while capturing via FW that the image size is still 1440x1080, so be sure that your sequence setting matches the source clip and not ProRes 1920x1080. It would still work OK because of the mutl format timeline but your performance will suffer.

Don't worry your head won't explode. When you get a workflow figured out that works well for you you'll be amazed at how affordable great image quality has become and all this frustration will have been worth it.

By the way, what ever happened to FindonChrispy?

Oh yes, I had just been using the regular ProRes 1440x1080 60i

I have two FW800 drives, and I just have them looped together with a FW800 cable and one into the computer.

From a quality perspective, I can't get better than...the ProRes capture, sequence and render control?

And no, I'm not outputting to film. I'm recording on a Sony Z1U and I'm laying it back to tape on that same camera.

I have a little time today, I may do some experimenting with the ProRes render codec on an HDV sequence.

Thanks!!!

Capturing ProRes from HDV buys you nothing. It just costs you disk space, and prevents you from using any of Final Cut's logging features.

Capturing HDV via Firewire is a totally lossless operation: It's just a data transfer off the tape to your framestore. So capturing HDV as HDV and then rendering it to ProRes gives you exactly the same results as capturing as ProRes, except along the way you get to log your tapes.

The place where you're taking a massive quality hit is when you "master" back to HDV. It's like "mastering" back to VHS; what ends up on the tape does not match what you saw in your edit suite. It's visibly different. And if you do any titles or graphics, well, they're just toast, man.

Mastering to ProRes is just not a big deal. Even in the absolute worst-case scenario of 220 Mbps, a two-hour ProRes Quicktime will still be less than 200 GB.

Okay, that makes sense!! Brain pressure is reducing!!

So probably what you would recommend...especially for customers who have potential for using the footage on BluRay, for instance, would be to export my HDV captured footage off my HDV timeline with the ProRes render codec as a self contained QuickTime using ProRes and keeping that file on a hard drive for archive? Maybe I'll keep a HDV tape of the edit as backup then...I like having backup. Or I suppose maybe down the road I should look into a RAID or something.

Okay...unfortunately this leads to another issue I have, and maybe I better start another topic about this...I'm sure Strypes would have a few opinions on this. I'll call the topic "Compressor MPEG2s look blocky"

Thanks!!!

Quote

Or I suppose maybe down the road I should look into a RAID or something.

This is one of those sentences that, when tossed off so casually, gives me a cold chill all down my back.

You do have some sort of fault-tolerant or redundant storage system for your framestore, right?

My jobs ? both the project folder with all the media and stuff, and the masters ? get archived on a medium-sized storage array for permanent vaulting. The storage system at my facility is something like 75 TB, with redundancies. So I can continue to archive to it for a while.

Smaller-scale storage solutions include LTO tape and RAIDs that can be directly attached to your edit system.

Sorry...didn't mean to be so flippant about that

I do generally have two copies of my media...photos, music, etc. The project I save on one hard drive and have the auto backup on another hard drive. I don't have a backup system for the actual captured video clips or renders. I figured that absolute worst case scenario, I could recapture the clips and plug them back into my project...whether that's an automatic, quick process, or fully manual...that would be my backup.

Now I'll plug my ears while you scream "NOOOOOO!!!!!!"

I would like to get a RAID system, but that's not our most important priority...we have cameras that need repair/replacing, for instance, and having a better backup system is a bit lower on the list. After all, we are a small business and money is an issue. Sometimes just making enough money to simply stay in business is an issue.

Honestly, I don't know a whole lot about RAID systems, but from what I've heard, they're rather expensive.

Also...how is it that you don't loose quality while capturing HDV via firewire, but you do when doing a Print to Video to HDV via firewire?

No, I'm not going to scream, because that's actually very reasonable. If you're shooting to tape, and you hold on to the tapes, the tapes are your backup. As long as you log-and-capture your tapes, you can always go back and batch-recapture them if you lose or discard the actual media files. What's important is your timeline itself. That's not the only way to go ? I use a RAID-5 framestore, because I don't like interruptions, and because it was reasonably priced. But it's a reasonable way to do things.

"Rather expensive" is a fuzzy concept. My framestore is about a T and a half, and I think it cost me a couple grand. I consider that cheap. Of course, for archiving you'd want a hell of a lot more than a T and a half, so the cost would go up accordingly. A Promise 12T RAID goes for about $15,000, I think. You'd have to buy a lot of videotape to make that cost-effective.

And here's the thing you need to know about HDV: It's a ridiculously compressed format. Uncompressed 1080i60 requires something like 1.2 gigabits per second. HDV is 25 megabits per second. The fact that you get a watchable, and sometimes actually incredibly good, picture out of such insane compression is a testament to the technology behind the format. But it's still really compressed.

That compression has to happen at some point. HDCAM SR is also heavily compressed. It records at either 440 or 880 Mbps, which is much less than the 1.2 Gbps of uncompressed 1080i60. But that compression is contained entirely within the deck; an SR deck has an HD-SDI input and an HD-SDI output, and while the footage is on that wire, it's uncompressed. So when you play back an SR tape, what you get out is uncompressed, and when you record on SR, what you give the deck is uncompressed.

If you wire up two SR decks together, output-to-input, and dub a tape, you're taking compressed footage, de-compressing it, squirting it over a wire, then compressing it again. What makes SR such a great format is that the compression is so minimal that you can't see the difference, with your eye, between the original tape and the dub.

HDV works slightly differently, but it's the same in principle. With HDV, you capture over Firewire; the footage is not de-compressed. It's just squirted over the wire as compressed data. Your computer is responsible for de-compressing it on playback. If you take a single, continuous piece of HDV footage on your framestore and play that back out the Firewire port to an HDV recorder of some kind, you'll get a lossless copy. That is to say, the copy was lossless; the original recording process was incredibly lossy, but the copy is bit-for-bit identical to the original compressed tape.

But because HDV is a GOP format, once you edit it, once you splice two pieces of footage together, the finished show has to be compressed again before it can be written off to tape. Your computer does this; it's what Final Cut calls "conforming." Final Cut is taking some frames which, on the original camera tapes, were represented as I-frames and converting them to P- or B-frames, and vice versa. So what Final Cut calls "printing to video" ? which most editors I know call "laying off to tape" ? imposes another generation of heavy compression on your footage.

This is exactly the same as HDCAM SR. If you digitize from SR tape, edit, then lay back to SR tape, your footage gets hit with a second round of compression. It's just that SR was designed in such a way that multiple generations of compression hold up very well; HDV was designed with no such goal in mind. So footage starts to fall apart very quickly.

(The temporal and chroma compression in HDV means that exactly when your footage starts to fall apart depends on what you've shot. If it's a talking-head interview, the second generation might look pretty darned good. But a second generation dub of a whip-pan will look like crap, and a second generation dub of a greenscreen shot will make your compositor want to stab you in the eye with a Wacom pen.)

Ironically, this is a big step backwards from HDV's standard-def ancestor, DV. DV is not temporally compressed, which means an edited timeline of DV footage can be written to tape over Firewire without additional compression. In that sense, DV editing was completely lossless, if you don't count the loss imposed by the original recording in the first place. In that very narrow sense, DV was even superior to Digital Betacam; Digibeta suffered generational loss (albeit so little loss that it was impossible to spot for many generations of dubbing), while DV never did.

Now, here's the thing: It's important that you know how all this technology works, so you can make good decisions about your pipeline. ProRes itself is a lossy format; would you be better off converting your HDV source material to uncompressed and keeping it that way? Sure, definitely. You'd be measurably, objectively better off. But building an edit system that could handle multiple streams of uncompressed HD, and buying enough storage to keep all your uncompressed media forever, would be a significant investment. So you make an educated trade-off. I'll throw away some of my picture quality to save money on disks.

Maybe mastering back to HDV is the right choice for you. Maybe you'll think about all the details and conclude that yeah, you're willing to throw away this much picture quality in order to save this much money. But it should be an educated trade-off, y'know?

OK, we're chasing our tails here.

IF your printing back to the Zi (HDV) then working natively in HDV is the most efficient way to work. So you don't need to convert to ProRes while capturing and it is debatable whether you will see any performance gains if you have your sequence render codec set to ProRes. That largely depends on what your editing and your style of editing. I believe this answers the initial question of this thread -- again...

Regarding "mastering" to HDV, regardless of what many on this forum say, there are a lot of editors and independent producers who shoot, edit and print back to HDV tape. Probably more than any other "HD" format. So continuing to have the debate about whether this is a good idea is pointless. If people choose to do this and ask for advice we all might want to try to give them advice on the best way to accomplish this. This criticism isn't just being leveled at those who are trying to give advice but also to those asking for it. It would help a lot if you clearly state what your trying to accomplish so the questions can be answered in context of the problem.

Also, Jeff, I think everyone would agree that "mastering" back to HDV is NOT the going to provide the best quality. However, it's not as bad as you appear to make it sound and if your delivering or archiving to an HDV tape it can certainly be the most efficient.

Lastly, regarding converting HDV to ProRes, saying that "Capturing ProRes from HDV buys you nothing. It just costs you disk space, and prevents you from using any of Final Cut's logging features" is not true. again the only time this statement is accurate is if your mastering back to HDV, which on the other hand Jeff your arguing against so I can see why Casey would be more confused not less. If you have HDV source material and you want to work with After Effects, Color, or many other third party applications then there can definitely be an advantage to converting to ProRes.

As much as I don't care for it, there really are good reasons to work natively in HDV, and there are good reasons to convert HDV to ProRes. But in the last year I have worked on nearly 100 HDV and XDCAM HD projects and this thread managed to confuse the crap out of me. Casey, from what I've been able to glean from your replies I would recommend that you work natively in an HDV sequence, set your sequence render to ProRes which will only improve performance in the timeline, and then conform and print back to HDV. Unless you are sharing data with others or working a lot in AE or Color then these are other things to consider that might necessitate a change in your workflow.

Sorry, I got up on the wrong side of the bed this morning.

>Honestly, I don't know a whole lot about RAID systems, but from what I've heard,
>they're rather expensive.

Not really, as Shane Ross demonstrates in his blog. It's about how you find ways to work around issues. (Tongue-in-cheek, but you should check up with your local VAR man on the cost for the amount of storage you need).

[lfhd.blogspot.com]

[lfhd.blogspot.com]

>how is it that you don't loose quality while capturing HDV via firewire, but you do when
>doing a Print to Video to HDV via firewire?

Most acquisition and post production codecs employ spatial compression only, which means that the footage is compressed by comparing and discarding pixels from within the same frame itself.

HDV is also known as a GOP format and it is heavily compressed to fit into the same data rate as DV (25mb/s). To contain the amount of information required to float a picture at the size of HD in such a small amount of space, the footage is both spatially and temporally compressed into a GOP (which stands for "group of pictures".

A group of pictures works in this way: You have an "I" frame (intra or key frame), which contains a picture. Then B (bi-directional) and P (predictive) frames, which are in fact calculations based on moving blocks. P frames contain more data than B frames and although it isn't a full picture, it can be used as a reference frame. B-frames contains the least data of the 3 types of GOP frames, and looks forwards and backwards (which is why it's called a bi-directional frame) for information from reference frames needed to rebuild a full frame on playback.

When you capture HDV via firewire, it copies exactly what was on the source tape and rewraps it into a quicktime file. When you print to video, it has to conform everything back to a 15 GOP long format.

Eg. Here's a bunch of GOP frames. With IBP frames and a fixed GOP structure just like your HDV format, I frames contain your full picture, B frames is a calculation derived from either I or P frames, P frames are derived from I frames.

On your source tape and captured into your edit machine with the exact same GOP structure:

I-B-B-P-I-B-B-P-I-B-B-P

In Final Cut, when you make a cut, Final Cut automatically creates an I frame at the point of the cut. So assuming we make a cut at the 3rd frame, which is the B frame.

I-P-I-B-B-P-I-B-B-P

When we print back out to video, that needs to be conformed back to GOP structure. So everything needs a render. Which means the entire GOP is decompressed and rearranged. So you get:

I-B-B-P-I-B-B-P-I-P

From your original:

B-P-I-B-B-P-I-B-B-P



www.strypesinpost.com

Thanks Jeff for your thorough explanation of how this works!!! Thanks Chuck as well!!!

I now believe I can make an educated decision on why I do things and what works best for me or whoever the editor is...which I think is the point of the whole thread, or this forum for that matter.

I would like to try ProRes as the render codec to see how that works for me, but yes, I better understand the limitations of HDV and, for what we paid for the cameras, for what the customers are paying for the final product, I know not to expect XDCAM HD results, but simply want to do the best I can with what I've got, in a reasonable amount of time.

Consider this question answered, as for me!

You guys rock!!!! Thanks!!!!

>IF your printing back to the Zi (HDV) then working natively in HDV is the most efficient
>way to work.

That's my word on it too. Provided you are not grading in Color. Ultimately if your client demands a HDV master, you give it to them and find the best workflow that suits your needs. If you ARE going out to a HDV master, working in HDV is actually your best choice. Your footage will be recompressed back to HDV going back to tape, and converting to ProRes may in fact be marginally worse. (1 generation for the transcode to ProRes, 1 generation back to HDV).

I do know of clients who request for a DV master (probably because DV is affordable to them), and usually I advise them to get a digital beta copy if the house has a deck which they can then use for future projects.

Otherwise, a master is generally regarded as a high quality medium that you will use to generate copies from.



www.strypesinpost.com

This is a hell of a good thread. Some of this should go in the FAQ.

[www.lafcpug.org]

Michael Horton
-------------------