dropped frames in Cambodia

Hi there,
running FCP 7.0
Mac Pro, early 2009 with 2 monitors
2.66 GHz Quad-Core Intel Xeon
RAM: 16G
Graphics: NVIDIA GeForce GT 120 512 MB
MAC OSX 10.7.4

Editing a feature film with 75 hours of footage transcoded to 1920x1080 Apple ProRes 422(LT)
We are experiencing dropped frames after 30-60 minutes. Software restart does not help. Rebooting
the Mac does. Is the graphic card the bottleneck? If yes, any recommendations for a better Card.

Where is the footage stored? What sort of hard drive? Connected how?


www.shanerosseditor.com

Listen to THE EDIT BAY Podcast on iTunes
[itunes.apple.com]

Internal 3 tb seagate barracudas 7200rpm.

The GPU is a pretty poor one to be sure but dropped frames is usually due to

A. the disk
B. the media
and
C. unrendered effects and/or being on a timeline that isn't the same size/rate/codec as the source.

So A.

Are your media disks very full? SATA HDDs slow down as they fill up - don't fill past 50% if you want good Read/Write speeds - and no more than 80% full for safely storing data - any fuller and you run the risk of data loss.

This is the first thing to check.

Also that your render files are going to your media disk(s) and not your system HDD.

B. Is the media all the same format? Have you noticed any corrupt frames or glitches audio - surprisingly corrupt audio can cause a whole raft of issues.

C. Are you applying any effects? Are you running multi-cam? Is your timeline set to the same settings as the media?


Lastly - make sure you have RT set to "Unlimited RT"

The GPU could be an issue if you are trying to use that 120 to run 2 monitors of 1920x1080+

Recommendations for upgrade are here:

[www.ebay.co.uk]



For instant answers to more than one hundred common FCP questions, check out the LAFCPUG FAQ Wiki here : [www.lafcpug.org]

>The GPU could be an issue if you are trying to use that 120 to run 2 monitors of 1920x1080+

That's an interesting one. From what I have experienced, the computer monitor may drop frames but the playback/IO card output will be fine.



www.strypesinpost.com

I've had all manner of playback issues on older FCP systems when stressing the GPU out to the max - dropped frames is but one issue, often in collaboration with some effects that are GPU accelerated. Usually though on even older GPUs than the GT120 with 512MB VRAM or less.



For instant answers to more than one hundred common FCP questions, check out the LAFCPUG FAQ Wiki here : [www.lafcpug.org]

Quote
Ben King
...and no more than 80% full for safely storing data - any fuller and you run the risk of data loss.

I suspect that defragmentation/compaction lets you fill big HDD volumes with video data to even more than 90% with full safety. After all, it's not a system volume that's creating various caches. I use iDefrag often to defrag/compact and it has never messed up.

Dennis Couzin
Berlin, Germany

Maybe so but defragging DOES come with a risk of damaging your data in the process of moving and rewriting. If for instance you had a power failure or even just a brown-out in the middle of a rewrite. Using a UPS on your media disks and your computer is always a good idea.

Despite never having a mess up - if you defrag make sure you have a backup first!

A file by file backup actually defrags as it goes anyway!

Although if you were to do block-level backups, the fragmentation will also be copied.

Also using an HDD over 80% will mean far slower performance on the data written later.



For instant answers to more than one hundred common FCP questions, check out the LAFCPUG FAQ Wiki here : [www.lafcpug.org]

It's one big old wives' tale. Both the data loss tale and the dramatic speed loss tale (due to overly full HDDs) pertain to system disks, not to defragged/compacted scratch disks. Ben's post in a recent strand included a videoed explanation of how read speed decreases with HDD filling. See the graph at minute 1:17 of the video. Read speed decreases gradually from empty to full, progressing like so:

disk empty: top speed
disk 50% full: 85% of top speed
disk 60% full: 80% of top speed
disk 70% full: 75% of top speed
disk 80% full: 69% of top speed
disk 90% full: 62% of top speed
disk full: 55% of top speed

There is no profound change from this cause at 80% full or at 90% full. Profound speed changes, and dangers, arise in overfull system disks, not in equally full defragged/compacted scratch disks.

Dennis Couzin
Berlin, Germany

Here's a good guide with visuals of what Dennis is explaining. [macperformanceguide.com]

But to add that regardless of Dennis' personal housekeeping (which works form for him)

It is advised a "rule of thumb" that you keep at least 80% free and preferably more on all disks regardless of their use. Of course with higher capacities of HDD (2TB+) the margin could easily be pushed to 90% / 10% (even on system disks) but if you use very large video files then you might want to stick to the 80% guide.

I have also seen personally that filling HDDs past 90% up to 100% can cause data loss and I have seen it on many occasions. Old wives tale it is not!

Although if you don't mind the speed loss and if you have time to spend defragging then you might consider Dennis' idea if you are really desperate to use the space.

Avid used to recommend that you do NOT defrag your video scratch disks as it performs better fragmented than with contiguous files. So take the recommendations as you will.

However for me I cannot abide losing 30-40% of the speed of my HDDs - especially in a RAID when I rely on them to be able to finish on time -30% equates to a LOT of extra time over a day and a week if doing lots of transfers or loading and/or exporting of files. No point buying a fast disk to fill it up and have it run at ~60% speed when storage is so cheap.



For instant answers to more than one hundred common FCP questions, check out the LAFCPUG FAQ Wiki here : [www.lafcpug.org]

> I have also seen personally that filling HDDs past 90% up to 100% can cause data loss and I have seen it on many occasions. Old wives tale it is not!

Indeed it is not. I don't care what the numbers say, and I don't care what "test results" say. My field experiences see a drastic and instant slowdown of all operations the minute you hit the 90 per cent mark on storage. It's not even a gradual slowdown as the test numbers would suggest; it is a noticeable spike in Beach Balls of Death and slowdown in response. And it very definitely does not happen only with system drives.

General performance tests are all well and good, but they don't tell the whole story. I'd rather trust editors, who are using the same tools as I and doing the same tasks over weeks, than a tech performing a general test which lasts only minutes.


www.derekmok.com

Ben's and Derek's warnings about overly full hard drives should be taken very seriously. My use of "old wives' tale" was over-the-top (as well as sexist) but it is really hard to stimulate investigation in this forum.

I just did a little speed experiment. I copied a 23.92 GB folder from a RAID 0 drive over and over to a simple eSATA drive. The eSATA drive had capacity 249.72 GB, so after the 10th copy it was 95.9% full. The times required for the 10 copyings were:

1. 4:42
2. 4:49
3. 4:55
4. 5:04
5. 5:16
6. 5:31
7. 5:50
8. 6:20
9. 6:54
10. 7:52

For Ben, this may close the case. Why tolerate 7:52 or 6:54 when you can have 5:16 or so instead? Indeed the inner sectors of HDD platters are slow. To test read speeds, I copied a 2.97 GB file from the 10th copy and then the same file from the 1st copy of the folder back to the RAID 0 drive. The respective times were 0:57 and 0:35, consistent with the write speeds reported.

I dragged that file from the 10th copy into a FCP7 sequence and tried simple editing. It behaved OK. The so-so 52 MB/sec read speed was good enough for this 1920x1080 25 fps ProRes HQ file. No colored beach balls. Not a hiccup.

The troubles Ben and Derek warn about were really experienced -- I've had my share of beach balls too -- but not in this experiment. It behooves us to understand what exactly happens when scratch disks become 80% or 90% full.

Dennis Couzin
Berlin, Germany

I think Dennis has touched on a point here, in that you should actually test your own equipment to know what it is capable of. That way you won't be surprised when things take longer to read/write than you'd calculated for that client who wants to know when they can expect the master file!

It is why a lot of people come to the forums namely; their own system does not function as it should or is expected to.

Like a geek I always test my HDDs before using them in RAID arrays but it is a very lengthy process and so most people will skip this. However when they run into a bad sector or poorer performance then they act surprised but its like buying a car and never driving it over 25mph and then complaining when it rattles at 40mph! If you want your equipment to go to the limits then you must stress test it and be aware of the caveats.

And so the key rule for HDDs is don't fill them up too much if you want to retain good read/write speeds.

This does not apply to SSDs which should stay more or less consistent from 0% to 100% full.



For instant answers to more than one hundred common FCP questions, check out the LAFCPUG FAQ Wiki here : [www.lafcpug.org]