Corruption over network - Need Professional Help

We're trying to finish 16 episodes of a kids show - no SAN, no fiber channel - just some drives plugged into a Mac Pro (mini sas, esata, firewire) and mac mini running some kind mini server over ethernet (I'm just one of the guys trying to edit on this system - I am not the IT guy who set it up).

We're getting lots of corruption - especially as files are copied from one drive to another. We are also getting corruption from files that haven't been moved - you clean up a section on Tuesday - look at it on thursday - and you're playing pac-man again. Mind you this is corruption of the source footage, not just render files - you can see it quicktime.

We need help - we don't know what to fix and it appears to be getting worse. At the least we need advice - do we replace busses, drives, cables (most of the drives are new, BTW) - we don't know where it's coming from. At best, I guess - is there anybody out there with a lot of experience dealing with issues like this who feels like coming to Vegas for a consult? Email me your day rate and let me know what you think.

Thanks

You really need to give more information. "Corrupt" doesn't tell us much. Describe exactly what happens, what your routine is, what you see when it happens, how often it happens. What are the specs of your show? Codec for the captured clips? Editing codec?

If I were you, I'd first take the network out of the equation. Isolate one of the workstations and work on it. From your description, it sounds to me like you're working on just one editing station. So what are you using the network for? Is it related to the project? If not, take it offline and test it.

FCP7 media (not FCPX) is pretty stable. They are self-contained QuickTime movies, and I get a master-media corruption maybe once every 40 projects. In eight years of running my own system at home, I've had maybe two corrupted clips. But that is contingent upon you setting up the project correctly in the first place. If you're using H.264 files straight out of DSLR cameras, and using them directly in the timeline, then of course you'd get problems left and right, because H.264 is not a good intermediate format. You should have converted the files to ProRes first.

So, first culprit is in how you access this media. Are you actually sharing editing files, or are you just working off individual drives with different copies of the same media? The latter would be a lot more stable than the former.

What kind of storage drives are you using? Wrong or inferior drives would contribute to file corruption. As would human error -- misuse, for example, if your editors don't dismount the drives before unplugging.

Self-contained QuickTime clips on a stable, healthy, properly spec-ed external drive should not be getting corruptions left and right. If they do, there's something wrong with your software, your computer, the drive, or the files themselves.


www.derekmok.com

Yea. Sounds strange. Do you have any 3rd party codecs installed? Eg. Perian?



www.strypesinpost.com

Thanks for responding, guys -

We have several stations - 2 lead editors, a couple of assistants, 2 animators - the show was shot 1080i xdcam ex and we finish in PreRes 422. There are a lot of files moving around.

The idea HAS BEEN to have a fairly complete set of files available for each editor and copy items over from the main raid as they are created or needed.

The drives are g-tech - the main one is a Raid 5 (5TB after striping), and the satellites are raid 0 - 2 TB.

The copying seems to be the issue - we get a lot of okay on one drive and bad on the other - which is at least fixable - but now our backups and outputs (going back to the main raid) are getting funky during that copying process.

The corruption is usually in the form of thin horizontal rectangles - which are either distorted from pixels above or below, or solid (either green or grey). Sometimes it's a splotch or smear of pixels now and then, sometimes a small red rectangle that looks like burlap and only shows up for a frame. Other times it's larger green blocks.

But, again, we see this not only in FCP 7, we reveal in finder, open in quicktime - and there it is.

nope, no perian!

Try this simple test- drop a few files into a dmg, copy it over and see if it passes verification. I somehow doubt that the corruption is caused by errors introduced during the transfers on all the hardware you pointed out. You can also put the files in read only dmgs and ship them around. There is data verification when you open a Dmg file. Another way to do it is to use Carbon Copy Cloner to do the syncing. Or Rsync.



www.strypesinpost.com

That said, are you monitoring the systems with atmonitor? Are all the machines running similar OS versions, QT, FCP?



www.strypesinpost.com

Sometimes you see the green and grey glitches because Quicktime has been corrupted. Did you try Digital Rebellion's tools for testing and fixing? Excellent products, with a free trial download.