Converting SxS cards from source to ProRes

If you wanted to keep all your clip information then use MM and recompress (transcode) to ProRes.



For instant answers to more than one hundred common FCP questions, check out the LAFCPUG FAQ Wiki here : [www.lafcpug.org]

Well you don't need to use FCP specifically, although you can.
Sony's XDCAM Transfer app is (in my opinion) a far superior interface for handling XDCAM media than FCP's Log and Transfer interface. Although I have not used it for XDCAM EX personally, you can use XDT to rewrap and import the clips to whichever destination you choose, and thereafter you can use Compressor (or any other suitable app) to batch convert those clips to ProRes (or whatever codec you wanted) ...

However, to smooth the pain why not automate the workflow and create your own one-step solution? First, set up a Compressor droplet that converts dropped files to ProRes and saves those converted clips to your Capture Scratch, and then deletes the original dropped clip. Then attach that droplet as a folder action to the import location that you use with XDCAM Transfer. Voilà. Now as XDCAM Transfer brings the rewrapped clips into that folder then Compressor will automatically handle the transcode to ProRes as each clip comes in, saves it to the Capture Scratch and cleans up the intermediate files. Not perfect perhaps but none too shabby.

The 2 options (correct me if i'm wrong here):

1. Transcoding with Compressor. It does not preserve timecode/reel name. Also, I think it clips superwhite/sub black values.

2. Transcoding with MM (recompress). It preserves timecode/reel name, file names, etc... Except the clip ID, which is used along with reel name and timecode to batch capture from source files (at least for P2).



www.strypesinpost.com

I guess I don't visit here often enough these days.
First I'll ask the dumb question. Why do you have to go to ProRes?
I ask because sometimes it's based on some misconceptions.

My typical workflow:

Import on MacBookPro via Express Card Port using Sony ClipBrowser (Alternately you can use Sony Express Card Reader in MacPro). I turn CRC checking in ClipBrowser on to make sure the copied files are checked good.

You can them backup the files to 2nd hard drive, DVD, DVD-DL, Blu-Ray (whatever your favorite security blanket is)

Open Sony XDCAM Transfer Tool and rewrap files to MOV. You can also log, label, etc in Transfer Tool if you like. BTW these files don't have crazy names and will be numbered sequentially. Alpha Numeric Naming convention is set in camera.

I import the MOV files in FCP. I set sequence using appropriate XDCAM EX Easy Setup. I then make ONE CHANGE. I set renders to Pro Res. Thus are renders are done in ProRes. There are NO LONG GOP CONFORMS. It's a breeze to edit with this way.

Once you're done you can simply export in the native codec self contained. There's no obvious long GOP conform. Now at THIS POINT if you want to change to ProRes you can in whatever method suits you but the native codec is just fine depending on intended use.

strypes, judging from your posts you seem to understand the process well. There seems to be a lot of misinformation in the discussion about using the EX codec. Set Renders to ProRes and there's not long GOP confirm. One can always export the final sequence to ProRes if needed but one doesn't have to convert the source files unless one has some imaginable motive, time, storage space.

I guess unlike P2, EX clips names are logical, sequential as set up by the shooter. In addition I shoot Time of Day Time Code too. In camera one can certainly name clip D1_XXXX and the next day D2_XXXX so one can sort by shoot day and clip number or shoot day and Time of Day Time Code. That's certainly not very descriptive though but one can add log notes in the XDCAM Transfer Tool.

BTW I'd thought I'd add that these days I only use expensive SxS cards for overcranking. I can use Transcend 16GB SDHC cards in the EX and at under $30 each they do too much more wallet damage than XDCAM discs so you can almost afford to save these as archival or at least take them out of circulation until you've backed up the files elsewhere.

Good call Craig - could you not use the more expensive (but cheaper than SxS) Class 6 SDHC cards for overcranking?



For instant answers to more than one hundred common FCP questions, check out the LAFCPUG FAQ Wiki here : [www.lafcpug.org]

Ben King Wrote:
-------------------------------------------------------
> Good call Craig - could you not use the more
> expensive (but cheaper than SxS) Class 6 SDHC
> cards for overcranking?

The limit to overcranking seems to how Sony manages the memory on USB Express cards (SDHC in adaptor) compared to PCIe Express cards (SxS). That's why Sandisk Ultra II cards fair as well As Sandisk Extreme III and IV cards. I'm finding that the cards work to about 720p24/40. That's about 65mbps write speed. I've heard some people have gotten a bit better with Transcend Class 6 cards at about 720p24/48 (or 720p30/60, 720p25/50 . . . which are all the same record data rate) which is about 70mbps. The cards are capable of much faster writes but Sony can't manage the faster speeds. I understand there are people already working on that issue (attempts to make a PCIe Express card adaptor).

All this happened with the EX1 firmware update 1.11 which made the EX1 compatible with Sony'd EX 60GB hard drive which is also Express card USB and also doesn't handle overcranking. The EX3 where firmware compatible with SDHC from the get go.

BTW I think Sony's alliance with Sandisk (SxS is Sony by Sandisk) may be part of the reason why Sony is OK with SDHC on a "wink." Mind you Sandisk makes SDHC cards and at a price that moves SDHC from reusable storage to tape like archival for the video professional.

In addition Sony announced an alliance with JVC at IBC and the results is that JVC has announce two cameras (HM100, HM700) which use XDCAM EX code AND record to SDHC (and JVC also makes an SxS add on recording device). I'll also add that JVC is aggressively advertising their camera's Final Cut Pro compatibility.

I really think the above two alliances where meant to be a major shot at Panasonic and their use of both P2 and AVC. Sony's looking to move XDCAM EX codec to be the next BetaCam. 57 minute (16GB) SDHC at $25-$35 (and that price will drop) with Sandisk benefiting in a big way and the codec's use by other camera manufacturers to increase the codec's market share.

Quote

In addition Sony announced an alliance with JVC at IBC and the results is that JVC has announce two cameras

Yeah I did a report on the Sony SXS licensing from IBC for the Digital Production Buzz with Mike and Dan and filmed SONY at IBC too for MacVideo with Rick Young.

Its definitely good news for the consumers when we can mix and match solid-state and HDD media between systems and cameras.

I would like to see Panasonic make a removable lens, low to mid range camera that handles AVCCAM or better still AVC Intra - something around the size of the EX3.

I like AVCCAM better than XDCAM-EX which has many compression issues not just the usual MPEG LongGOP and artifacts.

I haven't yet used AVCIntra but it looks great on spec.



For instant answers to more than one hundred common FCP questions, check out the LAFCPUG FAQ Wiki here : [www.lafcpug.org]

Quote
I like AVCCAM better than XDCAM-EX which has many compression issues not just the usual MPEG LongGOP and artifacts.

I think one thing Sony learned from VHS vs BetaMax is that it isn't the best format that always wins.

Note that Sony now has XDCAM 50mbps 4:2:2 now (although still Long GOP).

My own guess is Sony is going to come out with something to replace HDCAM and possibly compete with Pansonic's Varicam. I can take guess what that might be 100mbps 4:2:2, 50mbps 4:2:2 MPEg-2 I-frame, possibly at NAB (just a hunch).

I think Sony's moves with Sandisk and JVC are just the prologue to some other big moves to create a "standard" codec infrastructure.

AVCCAM is an awkward workflow for many until NLEs support it directly and time is money. Maybe FCS3 will support it directly. Like I say, the best format doesn't always win. Of course there doesn't have to be a "winner" just a choice BUT Sony's alliance with Sandisk and JVC sure looks like Sony is making moves to limit that choice.

Quote

I think one thing Sony learned from VHS vs BetaMax is that it isn't the best format that always wins.

True, true. However I don't think its that level of battle anymore - sure BluRay/HD-DVD was - but thats because it was a mass-market consumer level format and not the more niche pro-acquisition customers - its more "horses for courses" now.

Quote

I think Sony's moves with Sandisk and JVC are just the prologue to some other big moves to create a "standard" codec infrastructure.

Of course it is! Thats what

Quote

they

want cus its big bucks

Quote

Note that Sony now has XDCAM 50mbps 4:2:2 now (although still Long GOP).

Now the 50Mbps version might be great but its not likely to be implemented on the smaller cameras any time soon - which I think is a travesty from both Panasonic and Sony by not making high quality format versions ( XDCAM HD 422 50 and AVCIntra ) on smaller chip/body cameras.

I was hoping that the EX3 might have had a 50Mbps option but no...

Quote

My own guess is Sony is going to come out with something to replace HDCAM and possibly compete with Pansonic's Varicam. I can take guess what that might be 100mbps 4:2:2, 50mbps 4:2:2 MPEg-2 I-frame, possibly at NAB (just a hunch).

Hunches are all well and good if you are Quasimodo, but what does your gut say?

Mine says that RED will pounce again like The Ninja Leopard of Crystal Glass Mountain and blow the lower-end cameras and formats into the pit of bastardized technology from yesteryear where they belong...

...and that it wants some dirty southern fried chicken



For instant answers to more than one hundred common FCP questions, check out the LAFCPUG FAQ Wiki here : [www.lafcpug.org]

Actually I think the big battle will be in the codec rather than medium domain. It's what I think Sony is gearing up for and I think Panasonic is the target. VHS and Beta involved a tape format. HD-DVD and Blu-ray involved a disc form. Although there were certainly non medium related aspects, the codecs were similar. Now the battle is going to be MPEG-2 vs AVC. You'd think AVC would win hands down but Sony apparently isn't banking on it at all and are making moves with that in mind. That's my GUT.

Quote
Now the 50Mbps version might be great but its not likely to be implemented on the smaller cameras any time soon - which I think is a travesty from both Panasonic and Sony by not making high quality format versions ( XDCAM HD 422 50 and AVCIntra ) on smaller chip/body cameras.

I think they're both dealing with differentiating their product line. Do note that some would not have expected to see XDCAM codec in small "HDV like" camera from Sony and now JVC. JVC has gotten XDCAM EX down to $3999 list. Sony got 1/2" chip camera to $6500. These two options may help XDCAM EX dominate over Panasonic and AVCHD.

I think these companies see roughly definable markets. Sony is making moves in corporate and ENG markets. Sony is probably readying codec variants to take on AVCIntra as well. That's why I think there's going to be a very very very serious codec war.

Quote
Mine says that RED will pounce again like The Ninja Leopard of Crystal Glass Mountain and blow the lower-end cameras and formats into the pit of bastardized technology from yesteryear where they belong...

I would love to see RED ONE sales vs Varicam or F900. I'm not sure RED is beyond "boutique" yet. Then there's F23 and F35. That can change but again it's not about "best" it's about dominating the market (market share). RED might be banking on producing 2K and 4K cameras across the market with a an easy workflow but they seem to be a looong way from that kind of market muscle. They'd need a full line of cameras that would have to interest corporate and ENG market and that's a serious uphill battle for them.

______
Basically with Sony, Sandisk, JVC, Sony is moving SDHC XDCAM EX as the replacement to HDV tape. 50mbps 4:2:2 now gets them out of the 4:2:0 issue. They're going to make a move to replace HDCAM with a codec and that may be their attack on AVCIntra. This is their battle against Panasonic. Codec war for sure. That's my GUT and my crystal ball.

Quote

Actually I think the big battle will be in the codec rather than medium domain

Yeah I think its on already!

However we are all still using a multitude of legacy CODECs on a variety of media and I don't see this trend slowing down, even with the SxS / EX CODEC licensing shift.

As far a RED are concerned - given the new line up AND if they have a workable minimal and affordable motion camera "module set" then I think it might be a contender (as you say) for the corporate and ENG* markets.

Certainly the Scarlet body I saw at IBC (the same as NAB 08) was way too big.



*ENG - "Electronic News-Gathering" For those reading this thread who don't live/breath acronyms



For instant answers to more than one hundred common FCP questions, check out the LAFCPUG FAQ Wiki here : [www.lafcpug.org]

>Actually I think the big battle will be in the codec rather than medium domain

This will be an issue with the NLEs (and probably data storage). Every NLE has to then support the 101 available acquisition codecs (and still more), then it goes out of fashion...

I was actually hoping for a 100 mb/s long GOP AVC acqusition codec which makes more sense to transcode to ProRes, or if they can find a way to allow us to cut that natively like XDCAM/HDV.



www.strypesinpost.com

Quote
Certainly the Scarlet body I saw at IBC (the same as NAB 08) was way too big.

I think RED is reassessing the market and redesigning. It certainly won't be ready for NAB (which they won't be at). The thing has to be portable without hanging a score of accessories and in need of an IT team to get to shoot and go through the post workflow.

They also have to compete with the fact that with Sony and JVC you can now had a client an SDHC card and that camera master can be played with the free Sony ClipBrowser or even the free VLC player.

While "movie" production workflows can often deal with a support team and high quality but slower workflow, that's not the case with much of the rest of the industry and this is what RED has to be mindful of. This is also why a proprietary codec can be a problem. Certainly RED may have plans for a new camera that will address all that but I think they'll have a tough time beating Sony (or even Panasonic) in that regard. Even the "ideal" camera will need a marketing team to get it into corporate, news and related markets. Again the best codec (nor even the best camera) always wins and while I think RED can do great stuff I think they're a loooong way from having that kind of market share.

The fact that they had to go into a redesign as a result of the Canon EOS 5D Mark II shows they weren't anticipating how quickly the market is moving.

Sorry Craig you misunderstood me - I wasn't hinting that they should design a new package - I meant to say that once all the Scarlet and Epic brains are on the market it will be nice to see a cost-competitive small form of the modular design and to see how it works in practice.

Quote

The fact that they had to go into a redesign as a result of the Canon EOS 5D Mark II

Unfortunately the Canon is pretty poor attempt both quality and usability at an HD camera.

I did a fair amount of assessment on the raw footage from it and its not great. Yes when resized to web. AND if you shot so you don't have to grade. But push it a little with a CC and it blows.

Also 30p??? Whats that about? Not to mention Auto ISO... and BAD...

...no REALLY BAD! Rolling shutter issues.

RED don't have to worry unless Canon get it even slightly right with the next versions.



For instant answers to more than one hundred common FCP questions, check out the LAFCPUG FAQ Wiki here : [www.lafcpug.org]

Quote
This will be an issue with the NLEs

Have you noticed how closely JVC markets the HM100 with FCP? Have you seen the promotional video?
JVC HM100 Promotional Video
BTW notice the SDHC card they show is Sandisk (just an allusion to Sony Sandisk alliance as well as Sony and JVC).

In FCP when working with HDV or EX one can choose to "Render to Pro Pres" (the feature actually says it's only for HDV and EX!) to avoid long GOP confirm or transcode to Pro Res (as one would do for AVC).

Granted FCP MIGHT add a similar feature for AVC come Final Cut Studio III (which I'm guessing will be timed more or less with Snow Leopard).

Quote
I was actually hoping for a 100 mb/s long GOP AVC acqusition codec which makes more sense to transcode to ProRes, or if they can find a way to allow us to cut that natively like XDCAM/HDV.

The question is where FCP falls in the Sony/Sandisk/JVC relationship. Certainly it seems JVC and FCP seem tied. Interesting that JVC HM series has the MOV option (as well as a not clearly specified generic version which I think might be BPAV/MP4).

When you see Sony/Sandisk/JVC and JVC/Sandisk/FCP it almost looks collusion like. Again maybe FCS3 will support AVC with ProRes render but the above are certainly making a push to extend their market in advance of Panasonic.

Quote
Every NLE has to then support the 101 available acquisition codecs (and still more),

But new purchases (and upgrades) drive the market. If people perceive Sony, JVC, Sandisk, FCP as the "all in one" solution it helps them and not Panasonic.

Apple's whole marketing approach is different than NLEs in the past. It's not just about "having to support codec x" it's about working with cross manufacturer and product marketing. This is all part of the codec war IMHO. The marketing alliances are quite apparent. The whole thing looks to me like the above alliance is setting up to dominate market share over Panasonic and AVC.

It's not that Panasonic and AVC is going to go away. There's many reasons to choose it just as there are reasons to choose RED.

All this is just my "business and marketing" analysis and I suspect Sony is setting up a "network" of alliances to assure the EX codec gets priority treatment and increasing sales of "EX solutions" that benefit Sony (and the partners in the alliance).

_________
One wild card in this is Canon. They have to move beyond HDV tape so I expect they'll be picking something soon between AVC and EX unless they go a third way (unlikely IMHO).

Certainly AVCHD is big in consumer HD video cameras among several manufacturers. It MIGHT be interesting to see that if as part of the Sony/JVC alliance, JVC's 19mbps HDV codec starts showing up in consumer cameras. This is tangential to the things I'm thinking about though.
________
We as producers, camera people, editors, FX folks, will still choose the best tool for the job but it looks like there's a push on EX codec.

It's not about whether RED can do a better job than Canon. I'm sure they can. It's that RED decided to put Scarlet through more development as a result of Canon (and Nikon).

Maybe I'm mistaken but I could have sworn I saw statements about RED delaying Scarlet because of a market reassessment. It may well be that RED wants to address both markets (DSLR/HD Video combo device) and hence the delay.

You might be thinking about what's "best" but the manufacturers (including RED) are thinking about what "sells" and the two aren't necessarily the same.

I certainly think Sony (and Panasonic) are quite aware of RED ONE too. That may or may not "respond" depending on what they think of RED's market.

Although RED ONE is great it's still a niche market (although probably a profitable niche). Sony and Panasonic probably look at how RED ONE impacts their F900 and Varicam sales for example. RED's ancillary gear and workflow may put in more in line with the F23 though and maybe Sony is assessing that too. Certainly RED one is AMAZING quality at an AMAZING price in that market... but is it enough of a "share" to for Sony (or Panasonic) to care to respond?

My thoughts on all this is that maybe RED not only has to think about quality product but markets and where they want to expand and compete in.

Yes this also relates to codec support (and workflow). It's not just price/quality but price/quality/workflow (ease/speed of use in many cases).

If Scarlet can take great digital stills and be a B (or A?) camera on an HD shoot, it could be a killer (if the codec is also easy to integrate in an HD workflow).

Imagine if RED's response is to have a camera that records ProRes or ProRes HQ to flash cards at 2K (and 4K). Now wouldn't that be an easy high quality workflow that might shake up the EX vs AVC battle? They would have to couple it with Apple's marketing umph but that might break RED out of the niche.

If you've tried the FCP RED workflow, then you'll know it's really just like any other data-centric format. You put the card in the card reader, log and transfer, edit. Very simple. Only that treats the RED as a video camera, rather than a RAW camera. So, choose "native" and do the same as above, only when it comes time to grade, you send to Color and get the RAW data to work from. It's really, really easy. I shot Colette's first steps on the RED, did a quick edit just as I describe above and had it on the web for my folks to see quicker than I could capture from DV tape. Yes, I'm biassed, but yes, it's very, very easy.

[www.nattress.com] - Plugins for FCP-X

Quote

You might be thinking about what's "best" but the manufacturers (including RED) are thinking about what "sells" and the two aren't necessarily the same.

No its not - but the Canon cant even do a reasonable job... or fit in with any formats - Hence my statement.

Great stills - but >add expletive here< video. Really - I could not recommend it at all.



For instant answers to more than one hundred common FCP questions, check out the LAFCPUG FAQ Wiki here : [www.lafcpug.org]