QuickTime playback framerate artifacts

When I play a 720p60 .mov file with QuickTime its playback speed bounces around between 59 and 61 according to Inspector. The sound playback speed must be steadier or this would be annoying. The less than ±2% variation in picture speed should not itself be visible, and the bouncing is fast enough that sound synch stays within one frame.

However, you have to wonder HOW the playback speed can bounce around between 59 and 61. It would be simple if the monitor refresh rate could change between 59 and 61. My monitor is a Samsung 305T with just 60Hz refresh, so something else must happen. I did a little investigation, making a clip with numbers increasing every frame. I played this with QuickTime and repeatedly photographed the monitor with a digital still camera with shutter speed set to 1/1250 sec. About half of the photos showed double images: one number upon another. (Sometimes the numbers had equal brightness but they were usually unequal.) The odds of a 1/1250 sec exposure seeing more than one 1/60 sec frame is only about 5% so there must be double images frequently on the monitor. Where exactly are these double images created? By QuickTime? By the graphics card (a Nvidia 8800 GT)?

Is there a better way to play a 720p60 .mov file which avoids the creation of images that are not in the video? Using the FCP viewer was no better.

Note added: Resolution seems not the issue, since QuickTime playing a 576p60 .mov file makes similar framerate artifacts. All the .mov files are in codec ProRes LT.

Dennis Couzin
Berlin, Germany

This sounds hardware-related. Have you played more than one 720p file?


- Loren

Today's FCP 7 keytip:
Play from Playhead to Out Mark with Shift-P !

Your Final Cut Studio KeyGuide? Power Pack.
Now available at KeyGuide Central.
www.neotrondesign.com

QuickTime Player does not provide "frame accurate" playback. It adjusts playback frame rate and visual fidelity, depending on system resources, etc. It does keep the audio playback accurate (for the most part). This is its normal behavior ( a way to dynamically adjust frame rate and visual quality over a vast spectrum of computer setups while maintaining the overall "presentation" ).

In other words, QuickTime Player is designed to try to play back many different types of media as well as possible, over a wide range of user configurations. To maintain the overall quality of presentation, the visual image is dynamically adjusted, leaving the audio alone for the most part.


-Dave

P.S. - Also, make sure the "High Quality" flag is set for the .mov file. Other than using FCP itself, MPEG Streamclip may provide better playback performance than QT Player.

Loren and Dave, yes this is about system resources. Converting the test video to codec "None" (uncompressed 8-bit R,G,B) on a RAM drive and playing with QuickTime did result in fewer departures from reported 60 fps playback (but still the occasional 59 and 61). Yet there were way too many snapshots (now with shutter speed 1/1600 sec) showing double images. Perhaps QuickTime can hold 60 fps and the graphics card presents a second bottleneck. I wonder where the frame-blending is effected.

If our software and hardware does not provide frame accurate playback but introduces framerate artifacts, do we know what 60p, 50p, etc. look like?

Dennis Couzin
Berlin, Germany

Perhaps, I wasn't clear enough.

QuickTime Player is not intended to be a frame accurate playback application. It dynamically tries to adjust quality, in order to provide the best user experience over a vast spectrum of systems.

For frame accurate playback, you need to use FCP (and maybe a few other apps). But, with FCP, if your system isn't powerful enough, you may need to adjust the RT settings. And, when you step frame by frame, you should always get a proper display (though the Viewer and Canvas still don't provide full quality display).

Quote
dcouzin
If our software and hardware does not provide frame accurate playback but introduces framerate artifacts, do we know what 60p, 50p, etc. look like?

For proper playback on a computer system, you should be using a powerful enough system and FCP, with viewing done only on a properly calibrated and set-up external broadcast monitor (not a computer display).


-Dave

Dave, MPEG Streamclip yielded the same blend frames from the 720p60 file, as did the viewer of FCP.
The computer system is an old Mac Pro with four 2.6 GHz cores, 7 GB RAM, OS 10.5.8, no extraneous processes. Should this be suspected?
FCP does not report RT failure.
The graphics card is Nvidia 8800 GT. Should this be suspected?
The monitor is a Samsung 305T. Should this be suspected? (Is it too large for the card?)
What is the problem with non-broadcast monitors in this case? Can't the graphics card and the monitor maintain sync?

Perhaps others in this forum are wondering whether they are viewing 720p60 material cleanly: without frame blending or other framerate artifacts. I've uploaded the simple ProRes LT test clip. The test method is to take still photos of the (central part of) the playback using a very high shutter speed (like 1/1250 sec). Only a tiny fraction of snapshots should be unclean.

Dennis Couzin
Berlin, Germany

Could the problem be caused by a slow decay rate inherent in most LCD monitors?



www.strypesinpost.com

Quote
dcouzin
Dave, MPEG Streamclip yielded the same blend frames from the 720p60 file, as did the viewer of FCP.
The computer system is an old Mac Pro with four 2.6 GHz cores, 7 GB RAM, OS 10.5.8, no extraneous processes. Should this be suspected?
FCP does not report RT failure.
The graphics card is Nvidia 8800 GT. Should this be suspected?
The monitor is a Samsung 305T. Should this be suspected? (Is it too large for the card?)
What is the problem with non-broadcast monitors in this case? Can't the graphics card and the monitor maintain sync?

You may want to try to play back your test clip in QT Player 7 and MPEG Streamclip, with atMonitor running. Specifically, watch the "fps" value as your system plays back the test clip. The fps value is the buffer swap rate (or context switching rate) for the OpenGL driver (or just the GPU driver). On my system, I saw a consistent 120 fps using QuickTime Player 7 and 60 fps using MPEG Streamclip. Out of your entire clip's duration, the fps varied for a total of less than a half-second for both applications.

Quote
dcouzin
Perhaps others in this forum are wondering whether they are viewing 720p60 material cleanly: without frame blending or other framerate artifacts. I've uploaded the simple ProRes LT test clip. The test method is to take still photos of the (central part of) the playback using a very high shutter speed (like 1/1250 sec). Only a tiny fraction of snapshots should be unclean.

Quote
strypes
Could the problem be caused by a slow decay rate inherent in most LCD monitors?

I agree with Strypes' (Gerard's) comment, whether or not it relates to what you're describing.

Here's a decent article that describes the steps involved with reading video from a hard drive, decoding it, sending it to the GPU for final processing, and then displaying it:

Why software genlock at 60 FPS does matter!

and an interesting wikipedia article:

Frame Rate

So, it may well be that QuickTime, the other parts of the OS, and the graphics drivers all try to present video in an "optimal" fashion (and each part possibly doing better or worse than other parts of the processing chain). Add in display quality and specifications, along with hardware performance characteristics, and you can see that things may not always work as well as expected.

Depending on your fps values (using atMonitor), you might be able to get an idea if your hardware (or SW-HW combo) isn't working well enough to maintain the proper display of your test clip.

Again, computer displays are not going to provide the best representation of a video signal playback (even beyond the fps performance and processing effects that you have mentioned).

Does any of this make sense?


-Dave

strypes, good thought. My monitor response time (GTG) is specified at 6 ms. If the black-->white and white-->black times are on that order it could explain much of what I'm seeing.

Dennis Couzin
Berlin, Germany

Dave, thanks for the many good suggestions. I'll try atMonitor and hope of course that MediaMaster Pro (auxiliary software genlock) isn't needed.
My main interest isn't so much monitor viewing as projection. There is need for a practical versatile routine for playing various video files on various video projectors. (Can the FCP viewer be full screen?) 3 DLP projectors should be best and immune to the LCD response lag.
The Wikipedia article on Frame Rate is disappointingly weak on the visual side. (Anyone remember D.H. Kelly?)

Dennis Couzin
Berlin, Germany

There is the other variable to consider- the digital still camera. I can't remember exactly what shutter settings do on a CMOs chip, or if the rolling shutter exposes each frame faster than 1/1250 of a second. You can completely eliminate slow shutter readout issues if you were on a global shutter. I've seen camera flashes occur on only a part of the frame on DSLRs, Never an issue on CCD cameras. I assume the same happens on still cameras.



www.strypesinpost.com

strypes, I was careful to make the target small in the center of the frame to avoid any such shutter problems.

(Indeed I began the experiment shooting the monitor with a CMOS camcorder, also with short shutter speed set. The camcorder being 50p and the monitor being 60Hz killed that idea.)

Dennis Couzin
Berlin, Germany

[ I wonder where the frame-blending is effected. ]

You'll also want to check PlayBack Settings in your User Prefs (or is that System Prefs?) to see if the Frame Blending option is CHECKED. You never know...

- Loren

Today's FCP keytip:
Set a motion effect keyframe instantly with Control-K!

Your Final Cut Studio KeyGuide? Power Pack.
Now available at KeyGuide Central.
www.neotrondesign.com

Frame blending only comes into effect if there is a speed change. FCP does not frame blend. The reason why I mention monitors, is because on CRTs, you ALWAYS notice flickering artifacts on high resolution edges, but a lot less on LCD monitors.



www.strypesinpost.com

>strypes, I was careful to make the target small in the center of the frame to avoid any such shutter
>problems.

Try taking horizontal pictures and see if you get a different result. I'm really not sure what the refresh rate on CMOs chips are.



www.strypesinpost.com

Thanks Loren. Sure enough FCP User Prefs > Render Settings > Frame Blending for Speed was checked. But unchecking it did not eliminate the blended frames in either the FCP viewer or canvas.

Strangely, atMonitor reported 64-76 fps while playing the 60p clip in FCP. Also strangely, atMonitor reported 54-60 fps while playing a 50p clip in QuickTime. (QuickTime reported 49-51 fps.) So I have no trustworthy report of the FCP speed playback variation, just the observed blendings in nearly half the snapshots.

I've been working under the assumption that the monitor (Samsung 305T) is 60Hz because the rare spec sheet including this spec says it is. Perhaps this should be checked. Frame blending is one step more refined than frame omission/doubling when there's speed mismatch.

How do monitors and graphics cards achieve sync? Can the graphics card make slight adjustments in the monitor refresh rate? Then frame blending would be unnecessary when displaying a 60p video on a 60Hz monitor.

Dennis Couzin
Berlin, Germany

strypes, your suggestion of the LCD's slow response explains the apparent frame blending seen in a fraction of the snapshots. The target has white numbers on a black background. Going from the '0' frame to the '1' frame all the pixels in the '0' must go from white to black and all the pixels in the '1' must go from black to white. Since these states are determined by the twist of a liquid crystal molecule, greys must be result midway twist. So a fast snapshot taken during the response time will see some greyish '0' together with some greyish '1'. (The snapshot exposure is quite a bit faster than the LCD response time.) If we could be sure that there was no real frame blending, then examining a large number of snapshots would be a good method for measuring the LCD monitor's resonse time. Unfortunately there are reasons to suspect there is some real frame blending occuring too, because QT does not hold playback speed (unless the graphics card can control the monitor to match QT's irregular speed with irregular refresh Hz).

Dennis Couzin
Berlin, Germany

>For frame accurate playback, you need to use FCP (and maybe a few other apps).

I don't think the FCP canvas was ever meant to be a display for critical monitoring. We all use broadcast monitors for that. Is there frame blending when going out via SDI? No. I check it on the tape and it is as good as what I get from FCP on the monitor. Does the output from the graphics card/qt/drive access speed cause temporal artifacts? I would think so, but they were not meant for critical monitoring. For one, I've almost never seen edge flicker on them, although I should. But really, why go to lengths to get a general purpose computer monitor to work as a broadcast monitoring set up, and is it even possible? I know there are online machines with a quadro and SDI monitoring from it. But I doubt it's supported in FCP.



www.strypesinpost.com

dcouzin,

You should really read the article I linked to in a previous post. Ignore the mentioning of the company's media player product and just focus on the technical aspects.

For 60 fps, the corresponding interval (1/60) is ~16.666 ms. Your monitor has a gray to gray response time of 6 ms. The vertical refresh rate of your monitor is 60 Hz (± 2 Hz), according to the user manual specifications. You can look at your display settings in System Preferences to see what's shown there.

The fps value given by atMonitor is the buffer refresh rate. It doesn't necessarily correspond to the playback frame rate, or the inherent frame rate of the video file. The fact that your values bounced around may indicate disk read speed issues, or any of the components of the chain involved with reading the video from disk, processing (decoding and sending to graphics card, followed by more processing), and display to the monitor. Check the article again for the general process involved.

In addition to QT Player 7 reporting 120 fps from atMonitor (double buffering) and 60 fps from MPEG Streamclip, FCP 7.0.3 on my system showed 65 fps while playing back your test clip (frames were probably being buffered).

When you take photos (aside from any issues with the camera's behavior), how are you synching the timing of the snapshots with the playback of the video? Are you sure what you think you are reporting is actually valid? This is curious.

I would suspect that the only way to really evaluate this situation would be to connect hardware test equipment to the DVI port of the computer to analyze the signal and its timing (at least to see what is coming out of the graphics card). atMonitor or, even better, OpenGL Driver Monitor (comes with the Apple developer tools) may be a good way to measure how the computer is handling the video playback... it's pretty complicated... I don't know if your testing is able to really reveal anything about the system, if it's in fact showing what you think it is. But then, you are just seeing the end of the processing chain, so it would be difficult to determine where any issues may be occurring (if at all).


-Dave



-Dave

strypes Wrote:
-------------------------------------------------------
> >For frame accurate playback, you need to use FCP
> (and maybe a few other apps).
>
> I don't think the FCP canvas was ever meant to be
> a display for critical monitoring. We all use
> broadcast monitors for that. Is there frame
> blending when going out via SDI? No. I check it on
> the tape and it is as good as what I get from FCP
> on the monitor. Does the output from the graphics
> card/qt/drive access speed cause temporal
> artifacts? I would think so, but they were not
> meant for critical monitoring. For one, I've
> almost never seen edge flicker on them, although I
> should. But really, why go to lengths to get a
> general purpose computer monitor to work as a
> broadcast monitoring set up, and is it even
> possible? I know there are online machines with a
> quadro and SDI monitoring from it. But I doubt
> it's supported in FCP.

What I meant by that was that you would see each frame at the proper time, if you were stepping through the video, frame by frame.

As far as playing back video, especially at faster frame rates, I meant to say that QuickTime was never intended to be a pro-level playback application, whereas FCP is, depending on your setup. If there are enough resources available, FCP will play back all frames, whereas QT Player may drop frames, here and there. I guess that was my (poorly-expressed) point.

And, in my second post of this thread I did say that you need a properly set-up external broadcast monitor:

Quote
D-Mac
For proper playback on a computer system, you should be using a powerful enough system and FCP, with viewing done only on a properly calibrated and set-up external broadcast monitor (not a computer display).

Sorry for any confusion from my unclear statement about "frame accurate" playback. Thanks for mentioning that, Gerard.



-Dave

I know what you are saying, Dave, I just meant that the FCP canvas/viewer may not be the place for frame accurate monitoring as what our OP is trying to do. When you monitor 50 fps through a 1/60 shutter, you encounter issues with frame accuracy, not to mention nothing is synced. The question is why bother getting a non broadcast quality monitoring system working like a broadcast monitoring system and is it even possible?



www.strypesinpost.com

strypes Wrote:
-------------------------------------------------------
> I know what you are saying, Dave, I just meant
> that the FCP canvas/viewer may not be the place
> for frame accurate monitoring as what our OP is
> trying to do. When you monitor 50 fps through a
> 1/60 shutter, you encounter issues with frame
> accuracy, not to mention nothing is synced. The
> question is why bother getting a non broadcast
> quality monitoring system working like a broadcast
> monitoring system and is it even possible?

Well said. I totally agree. That summarizes the entire thread.



-Dave

Quote
D-Mac Wrote:
You should really read the article I linked to in a previous post. Ignore the mentioning of the company's media player product and just focus on the technical aspects.

The article presumes that the monitor has a fixed refresh rate, which obviously magnifies the task of the genlocker. Is this how things really work?

Quote
D-Mac Wrote:
The vertical refresh rate of your monitor is 60 Hz (± 2 Hz), according to the user manual specifications.

How shall we interpret the ± 2 Hz? This monitor doesn't do any image processing. Then is the refresh rate adjustable ± 2 Hz and if so does the graphics card call out the adjustment? If the monitor's refresh rate can range from 58 to 62 and if QuickTime's playback ranges from 59 to 61 then there would be no need for frame blending (or dropping or doubling of frames) on this monitor.

Quote
D-Mac Wrote:
When you take photos (aside from any issues with the camera's behavior), how are you synching the
timing of the snapshots with the playback of the video? Are you sure what you think you are reporting is actually valid? This is curious.

There is no synching at all. That's why a large number of snapshots are taken. They will capture random points in the monitor's frame displays. The fraction of snapshots showing blending gives the fraction of time that there is blending on the monitor. Before strypes suggested that LCD response time was rather slow, I thought this was the same as the fraction of frames that had made with blended images. After strypes suggestion things became more complex. A snapshot might show blending because it was taken during the refresh period or it might show blending because the frame itself had a blended image. (There's also the tiny fraction of snapshots that overlap two frames.) The experiment will be repeated with a 3 DLP projector to eliminate the refresh time contribution.

Quote
D-Mac Wrote:
Your monitor has a gray to gray response time of 6 ms.

I can't determine how GTG response time is defined. It will be the White-to-Black and the Black-to-White refresh times that matter in this experiment anyhow.

Quote
D-Mac Wrote:
I would suspect that the only way to really evaluate this situation would be to connect hardware test equipment to the DVI port of the computer to analyze the signal and its timing (at least to see what is coming out of the graphics
card). atMonitor or, even better, OpenGL Driver Monitor (comes with the Apple developer tools) may be a good way to measure how the computer is handling the video playback... it's pretty complicated... I don't know if your testing is able to really reveal anything about the system, if it's in fact showing what you think it is. But then, you are just seeing the end of the processing chain, so it would be difficult to determine where any issues may be occurring (if at all).

Issues are occurring if nearly half the snapshots show double images. We just don't know if the issue is just the LCD's slow response time. I'm planning to do visual experiments about frame rates, shutter speeds, etc. and I've learned from this strand that this can't be done validly with my LCD monitor. OK, I'll use another display. The direct method for verifying that the video file is correctly displayed -- no blending, no dropped frames, no doubled frames -- is by use of a photodetector and an oscilloscope.

Dennis Couzin
Berlin, Germany

dcouzin,

Thanks for the nice post.

Apparently, your monitor's refresh rate can vary by ±2 Hz (I am not sure what that means, exactly). The response rate of 6 ms implies a complete cycle of rising and falling, which is less than the 16.7 ms of a 60 fps frame rate.

This discussion is starting to go beyond my level of knowledge.


Though, I would have to agree with Gerard (strypes) and say that this effort may not be worthwhile (unless you are really into the technical aspects of all of this stuff and have the proper means to test every part of the process).

I would be more interested if you were trying to test an actual external broadcast monitor connected to the computer via a video I/O card, since that is really the only proper way to monitor your video. Anything displayed on a computer monitor connected via DisplayPort or DVI to a computer is going to provide only a proxy image and never close to the actual thing.

I guess that you may come to the conclusion that your particular setup doesn't provide one of the better proxy images, and that a newer Mac Pro with a higher-end GPU may do a better job, but all in all none of these is a valid display setup (computer with computer display).

My brain is tired...



-Dave

Yes, so the answer is spend more money, as is often the case in broadcast. If it was cheap and easy everyone would do it.

Some kinds of film rate artifacts are easily and unambiguously tested by the snapshot method.

Make a test film image consisting of a white dot which moves a certain amount with each new frame. (For convenience, progressive numbers can replace dots.)

Snapshots shot with 1/50 second shutter will usually (80% chance) receive exposure from two consecutively displayed 60p video frames and less often (20% chance) from three. If ever a snapshot shows an extra gap between two of the dots (or numbers), this evidences a frame dropped from the display. If ever the snapshot shows just one dot (or number), this evidences a frame doubling in the display. In this way a goodly number of 1/50 second snapshots can be used to check for those two frame rate artifacts.

Possibly the display is OK for the test film, because of its simple picture, but not for the video of interest. To avoid this possibility, a test video might be made with normal picture content except the dot action in the middle.

Dennis Couzin
Berlin, Germany

Quote
D-Mac
I would be more interested if you were trying to test an actual external broadcast monitor connected to the computer via a video I/O card, since that is really the only proper way to monitor your video. Anything displayed on a computer monitor connected via DisplayPort or DVI to a computer is going to provide only a proxy image and never close to the actual thing.

This is the part I don't understand. Of what use is a video I/O card if the videos come from the camera as files, or if the videos are constructed from still images within FCP? Isn't a H.264 .mts file a video? Isn't a ProRes .mov file a video? Is there any better way to play these video files than through FCP, QuickTime, et al.? I recently had a ProRes .mov file transferred to HDCAM. I'm pretty sure the transfer lab used FCP to play the file. An I/O card and HD-SDI signal came after.

I also don't understand why a so-so LCD monitor (provided the color gamut is adequate) can't be color-calibrated well enough for critical work. Mac OS-X offers its visual calibration routine and it should also be possible to import an instrument-assisted ICC profile. (I don't know if ICC profiling can correct for all errors due to the chromaticities of pixels shifting as their intensity varies, as a poor polarizer in the LCD could cause.)

Dennis Couzin
Berlin, Germany

The capture card is used for critical monitoring in FCP, with LUTs if necessary. The graphics card through quicktime can be seen as another output device, except that it was never engineered for critical monitoring. There are some platforms that use the graphics card for critical monitoring, but not FCP.



www.strypesinpost.com

strypes, does the I/O card bypass the graphics card and send what FCP reads directly to the monitor, filling the screen with the video? Which ones do you recommend?

Dennis Couzin
Berlin, Germany

(dedicated to Anita L.)
There seem to be just three ways a display can screw up frame rate:

1. skip a frame
2. show a frame more than once
3. blend frames
   1. for the full duration of a frame
   2. for just a part of the duration of a frame

The original short shutter speed snapshot method detects artifact 3 but can't distinguish between artifacts 3a and 3b. The long (1/50 sec) shutter speed snapshot method described in a previous post detects artifacts 1 and 2. There's a modification of the long shutter speed snapshot method that also detects artifacts 3a and 3b.

The test film image consists of a white dot (or dash) which moves linearly a certain amount with each new frame. After several moves it returns to start. 1/25 sec or longer shutter speed is used for the snapshot. The camera is moved in the direction perpendicular to the dot (or dash) movement during the exposure making a time-blurred image. (I'll still call it a snapshot.)

The dots (or dashes) are photographed as smears, with the smear direction corresponding to display time. At least one of the 60p frames is photographed for its full 1/60 second. If the snapshot shows more than one dot (or dash) present throughout the 1/60 second duration of a frame, then artifact 3a is evidenced. If the smear shows a ramping up or ramping down in the time direction, then artifact 3b is evidenced (and can be measured). Artifacts 3a and 3b might both be present.

Dennis Couzin
Berlin, Germany