Compressor Chained Job Error with Cluster Use.

I have a multi computer cluster consisting of 8 core mac pros and dual core G5?s on a Gigabit network. I have been working on my compression settings to improve the final result, and have come up with a 2 step chained job. (first resize and sharpen then encode).

This works great when I submit the job to my local computer and don?t use any clustering what so ever. Of course it?s very slow. When I submit this chained job to a cluster of any sorts (both quick cluster and Apple Qadministrator), the first step (resizing and sharpening) works great but it never hands the resulting file off to the second step (the chained encoding one). I get a QuickTime error every time stating ?File could not be found.?

Any help, or direction you can point me in on this issue would be hugely appreciated.

I've totally seen this happen before. I wish I could remember exactly what fixed it.

Is the destination of the first job in the chain somewhere on cluster storage? I know you said you're using Gigabit, which is fine, but on your cluster controller you specified some path to use as cluster storage for the Qmaster spool; that directory is exported automatically via NFS and mounted up as needed on the service nodes. If I remember correctly, the destination of all but the last job in the chain should be cluster storage.

Also, just for the record, are you up to date with the latest ProApps update? Apple hasn't been shy about releasing Compressor and Qmaster fixes over the past year or so.

Thanks for the response. Actually I'm not using any kind of cluster storage. I have a 3 TB internal RAID in my Mac Pro. The original ProRes(HQ) file that I'm working off of is on that drive, as are all the destinations of both the first and second step(s) in my job chain. I've tried leaving everything as "Source" and I've tried setting specific destinations. I also tried messing with the older permissions and giving read/Write access to "everyone" on all folders involved.

I am up-to-date on all the Pro-Apps Updates. Thanks for checking though. I'm running Compressor 3.0.5

Permissions shouldn't be an issue; Qmaster is set up to avoid permissions problems automatically, except when it doesn't. But it almost always does.

Have you tried taking the other Macs out of your cluster and just running it on the cluster controller? Not a "this computer" single-CPU job, but just using the eight procs in your cluster controller?

Ye I sure did. I tried it as a quick cluster (1 computer using only it's 8 processors) and got the same error. I haven't tried running it as one computer with Qadministrator. But I suspect I will get the same results since QuickCluster didn't work. I'm still going to try it and will report back shortly.

Interesting, when I leave it setup through Apple QAdministriator but take out all the other macs and just use my local mac, it actually works. As soon as I add one of the other macs back into the cluster it gives me the same error message again.

Yup. Bet you a nickel it's a cluster storage problem. I don't have time to dive into it just this second, but refer to the "Distributed Processing Setup" manual for information on configuring cluster storage.

Thanks for the response and pointing me in a direction. That's all I need. However, when you say cluster storage, you're talking about a large storage device (server) that has all the data on it. Is that right? If so then that couldn't be the case as all my data is on my local computer. Or is there another side to cluster storage?

Again, no need to get too deep into it, I can research it myself, but can you tell me if I'm missing something on the definition of cluster storage?

Thanks.

Nathan

Yeah, refer to the manual. Cluster storage is one of those fundamental Qmaster concepts that the manual covers in some detail. Every cluster has to have cluster storage, otherwise service nodes can't access the necessary media files to run their tasks.

Aha!!! You're the best! I found it in the manual, and I think this may be it! I'm going to test in a moment. Quick question, if you don't mind me asking. When clustering do you allow it to "copy source to cluster" (in compressor settings) or do you choose "never"?

OK, so slightly new curve to this issue. After I adjusted the shared storage location Qmaster and Qadministrator came crashing down on me. Long story short I just finished re-installing FCS. All is well and working again, however, the chained event still isn't working on the cluster. I did however get a new error which may or may not have more details in it. Here is the error message.

Status: Failed - 3x HOST [dn2318.local] Source file /var/spool/qmaster/713FB608-E75D94E2/shared/032909_Broadcast-Broadcast_Sizin-6EA2FF40-CB0B-43BF-8134-12C96807A536.mov not accessible on the cluster node and remote copy is disabled

BTW: dn2318 is my computer name.

It sounds an awful lot like what you said, it's a shared storage issue, but now I'm not sure how to resolve it.

Anybody ever seen this before? Is there a website I can look up error messages that anyone knows of? Thanks for all your help!