I think I found how to eliminate artifacts in HDV

But I'd like the experts to speak out first. I will comment if there is interest.

Oh, this thread is going to go places.

I can hardly wait.

I nominate this for "Thread of the Year".

Speak out about....?

You haven't said anything.


www.shanerosseditor.com

Listen to THE EDIT BAY Podcast on iTunes
[itunes.apple.com]

filmman,

I've noticed that in many of you posts you have often seemed to be obsessed with the idea that "experts" on this forum are not sharing their "secrets" about FCP. Yet here you hint at some new technique you've discovered, and then refrain from divulging any information at all.

Ironic, dontcha think.

Been doing this for ages with correct chroma reconstruction. The only other way is to avoid using HDV in the first place....

Graeme

[www.nattress.com] - Plugins for FCP-X

ok vic. reveal your magic potion...

I shot some scenes with my Sony Z1 in HDV 1080i 60i.

I used some old settings that I'd forgotten where on my camera. I think this is why I picked up some artifacts on handheld shots where I panned the camera a quick pan.

Normally in my usual settings I didn't see any artifacts before, so I don't know why I suddenly had artifacts. I was pretty upset. I do a lot of handholding and panning.

I went out and did some more shooting and didn't have any artifacts. I used the usual settings that worked for me before.

But I wanted to correct the scenes I'd shot originally; I didn't want to reshoot them if I didn't have to.

I tried different things and nothing worked. Finally, I made a QT movie Conversion to Uncompressed 10 bit DV NTSC, 29.97 fps, 720x486,

and the artifacts disappeared!

I made MPEG2 files for DVD SP and the resulting DVD has no artifacts!

> I tried different things and nothing worked. Finally, I made a QT movie Conversion to
> Uncompressed 10 bit DV NTSC, 29.97 fps, 720x486,
> and the artifacts disappeared!

I have doubts about this.

Are you watching on a proper broadcast monitor?


www.derekmok.com

I have dual monitors. They're Sylvanias.

But I see the artifacts clearly on the pan shots. I even have a clip to prove it.

When I play the Uncompressed movie, the artifacts are gone. I can also make a clip of the same shot. So it's not my imagination.

I'm willing to demonstrate this at the next LAFCPUG meet.

"Uncompressed 10 bit DV NTSC, 29.97 fps, 720x486"

is it Uncompresed 10 bit,
or DV?
(cant be both, surely?)


nick

Hi Vic.

Many believe that jittering they see when playing 1080i HDV is the "artifacting" that many complain about. One producer I know couldn't understand why he saw "artifacting" on his editing system, but when he played the the finished HDV tape directly through his big HD monitor, the problems went away.

Besides inherent artifacts due to MPEG-2 compression, there are two reasons you find "Jittering" when watching HDV on an editing system (1) The computer system isn't fast enough to decode the MPEG-2 info and display 1440x1080 video. (2) Incompatibile monitor refresh rates. If your HDV is getting "Fixed" after converting it to 720, remember, your are reducing the amount of data throughput by about half, and you are converting to progressive - no interlacing artifacts.

Here are some tips I've found to reduce HDV "artifacts".

(1) Don't shoot HDV - if you choose to shoot SD instead, remember that the "artifacts" you get with SD are much more apparent than HDV, the alternative is to spend about four times as much money for more robust HD formats.

(2) Follow the same general rules people who shoot film do. Keep pans, zooms, etc. slow enough to allow the compression to do its job properly.

(3) Ignore the "artifacts" - your audience will too, unless they are video professionals like Shane, or Wayne.

Travis
VoiceOver Guy and Entertainment Technology Enthusiast
[www.VOTalent.com]

**Sigh**

Vic Vic Vic

This is something I mentioned to you in a previous post concerning HDV.

[www.lafcpug.org]

Quote

A number of documentary producers lost out on some HD work because they submitted HD proposals and then proceeded to shoot HDV, in the technical specifications they had missed the part where they must be no HDV source footage. Luckily for one of them they managed to get part of the budget back for a downconverted SD version which I onlined for them to get it past the tech review. The other film I think has not been cut due to major parts of the footage having severe artifacts and compression errors on the source.

You can do this by either converting HDV to SD via Compressor and/or in FCP directly on an SD timeline.

Whilst this does hide the artifacts what you are doing is taking a 1440x1080i image and making it smaller and thus the artifacts are less noticable or with good conversion/grading dissappear entirely.

What you are NOT doing, is finding a way to remove artifacts from the HD version

If you downconvert then why are you shooting HD?

Sorry but this is not a wonder fix. In fact its not a fix at all.

Please Vic - think before you post.

This has been a total waste of time.


Ben



For instant answers to more than one hundred common FCP questions, check out the LAFCPUG FAQ Wiki here : [www.lafcpug.org]

Thanks, Travis. I suspected as much ...

I appreciate the technical analysis. I really avoided video like the plague until I realized that it was hopeless; people will shoot video no matter what ...

Now, don't get me wrong; I love FCP. FCP is going to allow me to output all my movies in DVD.

I'm in a good mood today, because I realize now that the "artifacts" might be my Sylvania monitors' inability to keep up with the HDV.

Also, I discovered that converting to 720p and de-interlacing for DVD output has helped to improve my DVD quality.

Since I don't have an HD TV or any way to view HD inhouse, I convert all my HDV to SD anyway. I don't have any need for HD as I don't do any work for broadcast at this time.

Since I've embraced DVD release and exhibition as long as DVD is in play, I don't have to worry about COMPRESSION and ARTIFACTS anymore.

The "secret" of outputting good DVDs is in good COMPRESSION practices ... which I've learned here from all of you generous experts ... and artists :-)

If there is something that I can share with you is that going to an uncompressed stage between the originating medium and the final DVD MPEG compression stage is a good practice. I don't know the technical reason for this.

It's an extra step and maybe it can be demonstrated that just going straight from HDV to MPEG2 will accomplish the same results, in practice I've not found this out to be the case. I get better results when I go through the uncompressed stage.

I have some JVC HDV I shot where the artifacts are so bad, that even taking the resolution down to postage stamp size will not hide the artifacts. As HD, they're so bad that even a smeary HD CRT monitor won't hide them.

Basically, the artifacts happen in camera as part of the HDV compression. You can hide them, or mask them by making the image smaller, crushing the blacks where more of the artifacts appear etc. etc but you can't eliminate them short of avoiding the HDV altogether.

Graeme

[www.nattress.com] - Plugins for FCP-X

I didn't see that post, Ben. We must've submitted at the same time.

Anyway, it's clear that HDV isn't a reliable format.

But not all HDV cameras are the same. There are technological differences (in the way they record) although the spec of the format may be the same.

Both you and Greame make the same point, which I believe is true:

When reduced to 720p, the 1080i 60i format becomes the same as the SD format; except LOL

I paid $5,000 for a camera that should've cost me, say, $800.

Because I'm happy with the DVCAM mode and the Zeiss lens.

And even the SD DVDs that I've output from the HDV footage I shot have been okay. The few assignments I've done, the producers were happy to get the results.

I think Sony owes me a refund :-)

Moreover, all HDV cameras and recorders should be recalled. This calls for a class action suit ...

but wait, didn't Clint Eastwood use the Sony Z1 in his movie about the Pacific War?

They must've dealt successfully with the artifacts ... without having to shrink the format; in fact they expanded it to fill the 35mm theatrical screen.

So something doesn't add up here.

Right its the down-rezzing, man. But at least its good to know that there is "better" and "worse" down-rezzing, and this gentleman has found his.

As for reasons for shooting HDV if your just going to SD anyway?

1.) future use of footage

2.) I think down-rezzed HDV footage looks like it came from a relatively expensive SD camera -- which is nice.

3.) you can blend fields on the way down and get a nice looking SD 30p that isnt too soft. (I love 30p)

- Christopher S. Johnson

Let's see, where to begin?

Vic ("filmman", if your problem is due in part to your monitor setup, the problem is not so much your monitors by themselves, it's the combination of your computer, your setup and your monitors. If you're using computer CRT monitors, your refresh rate is likely 70, 75, or 85 Hz. None of these rates is directly compatible with any of the rates used for displaying video. 60Hz kinda' works for 60Hz (29.97) video, but it's too slow for displaying computer data without strobing, unless the screen has a relatively long persistance, such as an LCD. The video card then needs to do some fancy work to get each frame or field to display, which can cause the appearance of jerkeyness or stuttering in the image. To display 24fps correctly, we really need a monitor with a refresh rate of 48 or 96 Hz, along with the video card to display at that rate. Does anybody make such a thing?

Secondly, part of your problems are due to combination of your budget constraints and your particular mindset. You saw HDV as a way to bring 35mm film quality to a process with a cost closer to those of student "films" shot on Mini-DV. I actually believe your were correct in your assessment, were it not for the fact that HD and HDV is so new. Though there are a lot of people shooting HD/HDV and doing it successfully, NOBODY is doing it without having to go through a lot of experimenting and learning.

I've been extremely happy with the HDV material I've shot and completed. Far as I'm concerned, HDV is fantastic. However, I've been extremly lucky - I don't have a client breathing down my back, or festival deadlines, so I've got time to learn and experiment. Since I use HDV primarily to promote my voiceover services to you video and film people - I get to throw away anything I'm unhappy with - and I really like to experiment with new technology - I accept that a great deal of the stuff I try just isn't going to work. - and I have time to figure out what I need to change.

Now, Ben, I must respectfully disagree with your assessment that Vic's post was "a waste of time". Believe it or not, there are a good number of people in this industry going through the same kind of frustration as Vic. They may not post often in forums like this, but they're out there - I have quite a few regular clients who approach things with very similar backgrounds and approaches to production. - Some who do work for very large companies and large budget productions who somehow (through sheer determination, I suppose) get their productions completed, their clients happy, and get repeat business. When someone like Vic has a question like this, (well in this case, an "answer" we all get to learn a bit more.

Now, back to Vic - The Z1 is an excellent instrument. It's not an $800.00 camera, but it's also not a $70,000.00 35mm Arriflex or a Varicam. It's not even an XL-2. It's a camera that will shoot excellent HD images for a remarkably low price. You can even shoot footage with the FX-1 that will fool college cinematography students into thinking they are looking at 35mm Film. However all cameras and formats (film and video) have their limitations. The best filmmakers know their medium's limitations.

Here's what you need to do.

(1) Evaluate and upgrade your viewing and editing system. First you need to determine what your system is capable of showing to you. Can it display HD at all without problems?, Can it display 720P HDV without problems? Can it display 1080i without problems? Apple says that just to look at 1080i HDV you need at least a Dual 2.0 GHz g5 or Intel Core Duo, processor, at least 512MB RAM, a 128MB minimum video card. If you're doing anything else at the same time (like editing) you'll need a faster machine. If your system can't display it, you'll have no idea whether you seeing HDV artifacts, or some other problem.

(2) Experiment to see just what the limitations of HDV are. Once your system is set up to show you HDV, take your camera out for a few "Test Drives". Shoot some still footage, Shoot some footage with lots of action. Lock the camera down and shoot some action. Then shoot some stuff with lots of camera movement. Shoot some scenes with lots of detail and strange angles, others with less detail. Learn what the system will do, what the limits of the camera and format are and what YOUR limits are. How much degradation of the original image are YOU willing to tolerate? Everybody has different ideas of just what "quality" is. Remember ALL media formats have some sort of "artifacts" or degradation from the original.

(3) Determine what your final output format will be. DVD? Film? Blu-Ray? Broadcast?. Each distribution format has its own set of problems. And, the source format will interact with it. For instance, the grain pattern of film can cause havoc with MPEG-2 encoding, used for DVD and broadcast - HDV needs to deal with recompression problems. Again, you'll need to experiment - try various compression settings, change the lighting when you shoot.... Limit the amount of action and quick cuts, if necessary.

One could argue that a true "artist" shouldn't have to deal with technicial issues such as this, but all artists, have to deal with technical matters. What would we have left today if Da Vinci didn't know so much about the characteristics of paint and plaster.

Travis
VoiceOver Guy and Entertainment Technology Enthusiast
[www.VOTalent.com]

"but wait, didn't Clint Eastwood use the Sony Z1 in his movie about the Pacific War?
They must've dealt successfully with the artifacts"

i think they wound up using about 30 seconds or so of HDV.
i also believe they had a team of people (or one hard-working individual) to hand correct each frame, to "paint out" the artefacts when they occurred.


nick

and travis. this brings us back to my whole problem with HDV. there are just TOO MANY things it doesnt do well.

sure, various camera formats have things they are good and bad at. but i have yet to see any format quite as limiting ad HDV - at least from the standpoint of a guy who shoots a wide variety of (non-feature) subject matter.

HDV - not good for high motion
HDV - not good for chromakeying
HDV - not good for heavy effect work

HDV = not good for me

vic, dont get too fished in by what you read. think of it like in the music world.

say you buy a guitar. and you want a good one. so you get the famous XYZ artist "signature model" that looks exactly like the one your favorite artist uses. 97 out of 100 times the one he actually plays is a functionally, completely different instrument. they may look identical but there is a whole lot more going on behind the scenes of his than in yours.

thats kind of the way i look at HDV. its a way to get joe average consumer a format that will play on his shiny new costco HDTV and at the same time, fool budget conscious pros into thinking they are now HD shooters.

smart move by the manufacturers to increase demand for camcorders... and their HDTVs i just cant believe so many "professionals" bought into the marketing hype

Wayne, this is an oooold and over stated position now. You were right about the limitations of HDV, but documentary and HD news gathering are benefiting from HDV in all of its limited-ness. You are incorrect about pros not benefiting from the format. Thats just wrong. I've been editing documentaries here in Los Angeles for eight years and today about half of the programs are shot with HDV cameras. A full quarter are still shot with the old SD Panasonic DVX-100-B in DV format.

Many of us were hoping the HVX would keep us away from HDV, but it didnt happen.

We're not in love with HDV, its just a useful tool for professional documentary right now.

Lets see how big and cheap the cards are going to be for the new inexpensive Sony XDCAM-EX and perhaps then we of the limited budget can leave HDV behind for good.

- Christopher S. Johnson

ok, i'll shut up. dont know why i bother anyway.

What doesn't add up are some misquotes. I talked about the JVC which actually shoots 720p native and could, at that native resolution, be severely provoked into nasty unnacceptable artifacts. See here: [www.nattress.com]

Also, you've got to be very careful with shooting HDV and letting the camera downconvert to DV rez as it's jut not good enough. Respected ex-BBC Engineer Alan Roberts tests these cameras and found that practically all didn't achieve a correct broadcast standard downconversion. His advice was to do it properly in hardware or decently in your NLE.

Next, for movies shot on HDV - they're shot by people who know what they're doing. They shoot the HDV as if it were a film camera and limit all movement of the camera and as we know, it's movement that kills HDV. Anyone can do this with care, but you must take great care and know what you're doing. Even then, you may have to do careful patch-up work.

Graeme

[www.nattress.com] - Plugins for FCP-X

Yep, fast movement on darkish red foliage -- the worst case. I EXPECT bad-ness there in HDV.

So I ask, if any of you had a choice: you just received the go ahead from funders for a low budget doc series for television. The station may or may not launch an HD simulcast station later this year, and they do not have the same hard specs as Discovery Channel. And you were going to choose between a Panasonic DVX-100B SD camera and a JVC 200 HDV, which would you choose???

Look, I cant wait to toss the format either, but Ive seen some downright beautiful sit down interviews and B-roll from the JVC that made for a doc that has some distribution life in it.

- Christopher

I'd shoot the JVC in 24p mode, very, very carefully, knowing what to avoid. Talking heads and other stuff can look great. It's all about knowing your enemy.

Graeme

[www.nattress.com] - Plugins for FCP-X

Wow - this HDV thing gets super-heated everytime doesn't it?

Can someone post a clip of this artifacting for those of us who haven't seen it 'in the flesh?"

im reasonably sure i deleted all my original test footage. but i'll see if i can find some somewhere...

there is a gallery here:
[dvinfo.net]

dont know whats all there, but there seems to be a lot of posts - big files. dont care to invest time to download myself.

Well, Graeme already did, but here, I will as well. Keep in mind that this is a worse case scenario with DARK RED MOTION going on. There were some actually nice shots from this same doc. This still is from an FX-1, color corrected and rendered on a DVCPRO-HD Sequence, a Field Blender filter was also used, then saved as a jpeg highest setting.

So even a less compressed format wouldnt look great here, but the big squares are the MPEG-2 under very high stress.

http://homepage.mac.com/cjohnsonla/PhotoAlbum19.html

- Christopher

I think I have a reasonably balanced view of this format for documentary. I am aware of its pitfalls and also know that it can be superior to a SD production. Here is an example of a penny-wise and pound foolish production decision for a doc series (bit-rate-wise):

"This American Life" on Showtime has great photographic compositions and clever editing. Undoubtedly they were worried about HDV artifacts when making their decision about shooting format. They chose a Panasonic HVX-200. Sounds smart but then they go and shoot exclusively in SD on it. I dont know why but it might have to do with wanting to shoot on tape? Then for broadcast on Showtime-HD channel they use an embarrassingly weak up-rezzing solution. Result: it looks really BAD. It actually gives me a headache sometimes.

If they had gone with the JVC HDV to begin with, it would actually look better on Showtime-HD. 100 times better.

But yeah, shooting rock n' roll drummers hand held under a dim red light is gonna suck on HDV.

- Christopher S. Johnson

thats funny! why would anyone drop all that money on a fixed lens dvcprohd camera, only to shoot SD with it. thats like driving a dragster to church?!?!?