XDCAM Workflow

As this is my first post here, can I begin by saying how much I appreciate the time and effort that gets put into this forum. There's a generosity exhibited which is truly admirable.

I'm relatively new to FCP and will shortly be starting on a project which will be shot on XDCAM EX. Over recent weeks I have spent hours trawling through this forum gathering all the info I can about the best workflow for this format. I have picked up different tips from many postings and I'd just like to summarise what I've gleaned and ask for any comments.

1.) Use Sony's XDCAM Transfer to get the footage onto my media drive.

2.) Use Compressor to convert the footage to Prores 422 (but not Prores 422 HQ).

3.) Import the Prores to FCP.

My understanding is that XDCAM is a heavily compressed MPEG2 format which will likely demand heavy rendering and converting to Prores will reduce this problem.

Thanks in advance for any comments.

Mike

Thanks for the kind words. You should send Mike (not you) some cookies for that.

I would actually say that you do not have to spend the weekend (or longer) converting everything to ProRes. For the most part, you will be able to work with the footage without having to render. However, renders and exports may take a while longer, so factor that into the equation.



www.strypesinpost.com

There's nothing wrong with the workflow you've described Mike, but whether or not its advisable / best for your project can depend on some info which is not stated, for example hardware and software setup for the project, basic nature of the project eg long/short form, shoot to edit ratio, edit style (cuts or transition, treatment/effects) etc.

That said, as you've done your homework I'm betting you've arrived at the above workflow carefully, so have at it, it should work just fine

Best
Andy

The only thing I'd add to what the other guys have said is that, depending on your shooting ratio, you might prefer not to batch-convert everything to ProRes. In Final Cut 6 or above, there's no reason why you can't have XDCAM footage in your bin, and edit it into a ProRes 422 timeline. Final Cut will decode and play back the XDCAM stuff in real time at preview resolution, then when you render it'll decode it and convert it to ProRes for you. The advantage of this approach is that you only have to convert the actual frames used in your timeline, rather than everything you shot. That requires less disk space, and obviously less processing time. If you're doing finishing work on your timeline, though, you might not like having to re-render from XDCAM to ProRes every time you, say, throw on a color correction, or slip a shot by a few frames. It's kind of situational.

I think your proposed workflow is pretty much optimal, once you get into editing. If you can spare the time before you cut (and the disk space) to convert everything to ProRes, then editing will be a breeze.

+1 re Jeff's suggested workflow. Just finished editing a music video I directed that was shot with a Sony EX-1 and HVX-200 (B-Roll). I created a ProRes422 sequence, dropped in the QT wrapped footage (single clips and multi-cam), edited without a hitch and sent everything over to Color. Worked like a charm.

Yeah, let me elaborate a little bit on that, 'cause it seems to be a pretty common point of confusion, and this is as good a place as any for me to spout off.

Final Cut 6 included an "open-format timeline" feature. In versions of the software before Final Cut 6, you could drop any supported media format into any timeline, but Final Cut wouldn't play anything back until you rendered it. During the rendering process, Final Cut would take your source media and convert it to your timeline format, writing the footage out as Quicktime movies in your render cache folder. When you hit the "play" button, Final Cut would read the rendered Quicktimes and play them for you. If your source and timeline formats didn't match, Final Cut couldn't play your timeline back for you until you rendered.

But in version 6 and above, that changed. Now, instead of forcing you to render before you can play back off-format clips, Final Cut does the best it can to play them back for you without rendering. Now, precisely how this works is a little bit of a mystery. Does Final Cut read your source clip from disk, convert it to the timeline format in real time and play that back from memory? Or does it convert the source clips to some uncompressed in-memory-only format instead? It's not at all clear, but my assumption has always been the latter.

Lemme explain what I mean by that. Let's say you've got a project with one source clip, in (just picking something at random here) uncompressed 10-bit 1080p24. You've got one timeline, in NTSC DV format. You edit the source clip into your timeline and press play. Obviously Final Cut has to scale the source clip down, but that's easy for modern computer systems to do in real time. It's also got to insert 2:2:2:4 pulldown to get the 24p source footage to conform to a 60i frame rate ? that is, it plays every fourth frame twice. (Note that this is not standard 3:2 pulldown, but whatever, that's another conversation entirely.) By doing these two things, Final Cut can play back your 1080p24 source material in an NTSC timeline without rendering anything. So yay.

But what about the "DV" part of "NTSC DV?" DV is a codec, with different chroma subsampling and spatial compression and all that stuff. Is Final Cut actually piping the scaled, pulled-down footage through the Quicktime DV codec in real time, as it plays your timeline? I'm not sure. I don't think it is. If my suspicion is correct, then what you actually see when you play back your unrendered timeline is not exactly what you'll see when you output. It's an approximation. A good one, but still just an approximation.

Of course, if your timeline format is ProRes 422, this isn't a big deal one way or the other. ProRes 422 was designed to be a transparent codec ? that is, you aren't supposed to see any difference between a piece of source media and a copy of that source media converted to ProRes 422 format. It's something to be aware of, though, when you're working with a perceptually lossy timeline, like DV. Just know that what you see when you play back is not exactly, pixel-for-pixel, what you'll see after you render.

Sometimes, depending on your system configuration and your footage and timeline formats, Final Cut can't play back your timeline unrendered at full quality. That's when you get that light green "preview" render bar over your clip. That means Final Cut won't play back your timeline at full resolution; it'll give you, for example, a quarter-res image instead when you hit play. When your playhead is parked, though, you'll see the full-resolution image on your broadcast monitor.

The open-format timeline capabilities in FCP 6+ are really great, but intensely confusing if you're not used to them. The rules are all different, and you can get in trouble if you're not careful. Just to pick one example, if you're used to laying off by crash recording ? that is, by putting your deck into record and just playing your timeline, rather than using print to video or edit to video ? then you have to be extra-careful to check your render settings and do a render-all first. Of course, both print to video and edit to video take care of that for you, so it's even better practice to get into the habit of using those instead. And it goes without saying that if you export a Quicktime instead of laying off to tape, Final Cut will force a render of all frames as part of that process, so you're safe there regardless.

In short, the open-format timeline is a compromise. It lets you drop off-format footage into your timeline and more or less play it back without rendering, but it isn't magic. It has its own rules, and once you learn them, it's a pretty awesome way to work.

Many thanks. You've clarified it perfectly - particularly your last post Jeff. A perfect example of the generosity I mentioned earlier ! It seems to me that quite often different opinions are expressed in different threads in forums such as this and it can be difficult to get a summary.

I have been experimenting with the two suggested options and think I will go with the ProRes timeline. Sequence Settings like this:

Frame Size: 1920 x 1080 HDTV 1080i (16:9)
Pixel Aspect Ration: Square
Field Dominance: Upper (Odd)
Compressor: Apple ProRes 422

I should explain that while I have been using FCP for a year it has always been in a post-prod facility where these settings issues where taken care of by resident experts. After 7 or 8 years on Avid before that and 5 or 6 before that on the Fast Video Machine, I became a total convert to FCP the minute I started using it. There's a logic to it which I love, but I am still struggling a bit with the settings when I have to do them myself.

Mike

I guess I'm late to this game but I've been using EX since Dec 2007 and I think you're taking the long way around the block.

Use Clip Browser (NOT XDCAM Transfer) to get the files to the media drive.

Back up the BPAV folders

Use XDCAM Transfer to wrap to MOV and import into FCP

The files work FINE in FCP timeline (no need to re-encode to ProRes).

Use the appropriate XDCAM EX Easy Setup.

In Sequence Preferences set renders to ProRes. That SPEEDS rendering of FX in the timeline so there's NO GOP reconform.

The render may be a bit longer when it finally comes time to export the master and maybe at that point you may want your master to be ProRes but this is WAY FASTER than encoding clips to ProRes first.

Craig - thank you for your comments. I have discovered that - as you say - the files work fine in an FCP timeline and that I don't need to convert them to ProRes.

I should explain that the files are being transferred from the camera to a USB portable drive on location, so I am just using XDCAM Transfer to import them onto an FW800 iTank. The first day's shooting arrived today and all imported fine except for one particular clip which repeatedly crashed XDCAM Transfer. In the end I simply couldn't bring it in. I tried to drag and drop it to the iTank, but that failed as well. Have you ever struck this or anything similar ? I can't find any comments on this problem anywhere. I'm wondering if the on-location transfer to the USB drive mght have been corrupted.

Thanks again

Mike

Quote
I am just using XDCAM Transfer to import them onto an FW800 iTank. The first day's shooting arrived today and all imported fine except for one particular clip which repeatedly crashed XDCAM Transfer.

That's exactly why you should not use XDCAM Transfer for that task. ClipBrowser can be set to do CRC checking when copying the BPAV. It ensures the copy is good or reports if they copy is bad. Once copied to hard drive you can them make another backup of the known good copy. I like to use optical disk. I use DL-DVD but certainly Blu-ray would work too. Some will even go to XDCAM Disc (used as data disk in this case).

You then use the copied BPAV from the hard drive in XDCAM Transfer to wrap to MOV.

The problem with your method is that you may have made a bad copy (MOV) and you now don't have the original BPAV to return to.

To put it another way:
Clip Browser makes a CRC Verified copy of the BPAV which you can then backup to optical disk.
XDCAM Transfer can then use the BPAV on the hard drive to wrap to MOV.

_______
Regarding the now corrupted file, If the BPAV metadata itself was corrupted, you could import the raw MP4 into Clip Browser and it would create a new BPAV with new metadata. With a corrupted MOV file you're in a much tougher place.

On location transfer is always risky business but that risk is reduced using Clip Browser with CRC checking/verification. I've been using adaptors (MxR but MxM also is good) and SDHC cards that I test beforehand. With two 32GB SDHC cards I have a four hour load in the camera which generally can get me through a shoot day without even changing a card let alone copying a file in the field. It would be easier and less risky to get two more SDHC cards than cary a laptop, hard drive, transfer in the field.

[www.e-films.com.au]
[e-films.com.au]
or
[mxmexpress.com]

Hi Mike,

What Craig was referring to is the "Render Control" pane, the 3rd folder tab over in the Sequence Settings. It is here that you set it to do your timeline render in Apple ProRes 422.

Your USB question... I once had a card, it was an SxS card that had an error in the boot block of the card or in the directory tree. Anyway it would come up in the camera with all the clips there but if I tried to ingest it into any appropriately XDCAM ready computer it would not mount. What I did was a USB transfer from the camera to my editing station and all went well from there. After that a simple card reformat fixed the issue with the card. Haven't had the problem since.

Andrew