HDV, HD, DVCPRO HD, PROGRESSIVE, field-blending - Misconceptions, self-imposed limitations - just what can we do with FCP5 as far as output is concerned?

I'm completely confused. I see on the web videos of excellent quality that open up quickly and play with sharp picture, great color -- what are we doing wrong to put up fuzzy, tiny, and dull video clips on the web? (I'm speaking for myself and the majority of video posters...)

And as for output through DVD SP, shouldn't we be able to output a great looking DVD if we're shooting with fancy cameras that cost thousands of dollars and the newest technology? Wasn't FCP developed to be an excellent tool for creative artists?

For example, many people don't like HDV, but I like the look of HDV when converted to DVCPRO HD. It looks great. I've read here that Shane Ross works with this workflow. I like to know more about the ways of maintaining quality and outputting quality regardless of what's the originating camera.

FCP5 has a lot to offer; it takes time to learn all the best workflows, however. Then there are many after-market peripherals, plugins, cards, filters -- what's absolutely essential and what are the luxury items (that make sense for the professional editor to offer his/her customers)? Personally, I'm only interested in what I can do with FCP5, but I do want to know what are my limitations, so I don't end up wasting weeks of my time pining after some effect or result that I can't have with my FCP5 Studio and dual processor G5.

Where can we get such plugins as field-blending, which are indispensable for web-streaming? I tried www.mattias.nu but couldn't find field-blending. I tried his de-interlace tool but it didn't produce a good video. I must've missed something in the way I used it or possibly it's not the right filter or plugin I used.

Would you all experts be willing to share some of your excellent knowledge on this subject in this forum? Thanks in advance for your spirit of generosity.

.

When life gives you dilemmas...make dilemmanade.

Interesting... the email version of grafixjoe's post was 9 dots with no sig, but the web one was only 1 dot with the sig.

ak

ak
Sleeplings, AWAKE!

Hey, Andrew, sorry to have to correct you, but Joey did provide his Master Yoda quote :-)

His answer was: do not. You see, for Joey there is no try.

So that single dot was very weighty. It took all the discipline of a Jedi to fix that dot on the page and not waver for a second in pressing submit. LOL

Far be it from me to question a star wars fan's use of movie quotes, I merely meant to call attention to the different format of the posting, the one that was emailed to me and the one that I could read on the tubeweb.

ak

ak
Sleeplings, AWAKE!

Filman

Keep the subject lines short, please. No reason to have so many words in the subject field

Michael Horton
-------------------

Since no one has tried to seriously answer the question - here goes!

Garbage IN = Garbage Out - It really is that simple! While FCP 5 handles all sort of formats, that don't make them all GREAT! Primarily it's about color space. MPEG-2 is the same as HDV which is 4:2:0. Same stuff you get off a DVD is what is written into HDV.
4:2:2 is the standard def (been around forever) cause it works format.

DV is 4:1:1 and is more compressed that the DigiBeta type of Standard Def SD 4:2:2. Other than Sony's abysmial use of MPEG-2 for everything including some forms of HD, the real 4:2:2 champ right now seems to be Panasonic P2 cameras. (Not dealing here with PanaVision or Ikegami cameras).

Now we have 720p - 1080i and p and all the other imitated formats that HDV works around. 24p - 24pa - 25p - 29.97 and 30 fps. Since we left just SD at 720x486, it's gotten a lot more complicated and FCP has tried to cover every format change as the come down the pike.

Don't know what more you could ask for. Bottom Line - If you want pristine output, put that in!

> I'm completely confused. I see on the web videos of excellent quality that open up quickly
> and play with sharp picture, great color ?

Beside color, deinterlacing, gamma, read about compression markers and learn how to use them, they are like keyframes for compression which you do set by your own and by visual experience - it's hand made 2-pass , 3-pass etc. As with any keyframes there should be as few as possible, but as much as needed - it's some kind of art, not a software feature.

Regards
Andreas

Thank you, John and Andreas. Regarding the HDV I'm finding out the hard way, I'm afraid.

As for compression markers, I didn't know anything more than just clicking and adding compression markers when I'm about to export to DVD SP. I'm going to read up on them if that's going to improve my web video clips.

Just for starters, where do compression markers go, as they're placed by hand -- I'm assuming in the timeline? Do they go at the beginning of each shot? Do you put more markers in a fast changing scene?

And, Mike, sorry about the length of the subject lines. I didn't think of the effect on the format of the page. Sorry. Will keep them short and sweet in the future.

One issue I have with anyone wanting great web video, if you want it to look great hire a professional! HDV can look great on the web if done properly. Just because we have "Compressor" doesn't make us a web master. I can and have put video up for any one to see, I know if I want it to look like the "Dreamworks" stuff on the web I would have to hire someone who knows what they are doing.

My rant,

kevin

How hard can it be? Thousands of people are putting video on the web. They can't all hire professional web video specialists. It should be possible to figure out what works for a three minute clip. I hope someone will come along and give us a few pointers as to compression and file type, etc.

One of the reasons this frequently asked question is never answered to anyones satisfaction is that there is no answer. There is no answer that anyone can give in single post. It's a book of answers, and if you are truly serious about compression, then you got to do the work.
Start with this book

Compression for Great Digital Video: Power Tips, Techniques, and Common Sense

Michael Horton
-------------------

filmman Wrote:
-------------------------------------------------------
> Do they go at the beginning of each
> shot?
Maybe, maybe not. Normally that's a good start.
> Do you put more markers in a fast changing
> scene?
Mostly not since the change of picture is constant. But with something like moving cam to the right, stop, moving cam to the left there should be a compression marker for each comma.

Regards
Andreas

The book costs too much. I checked on eBay, no luck; there are two for $29 plus shipping.

Andreas, those are good tips. Do you a couple more tips like that?

:-)

From a common sense point of view, we should place compression markers on those frames that don't change much or at all, so they're like stills and compression won't bother them. They will then be compressed the most.

Where there are a lot of changes in the image frame to frame, like fog in a scene or a fast moving subject, then we should place no compression markers at all.

Am I thinking right? Is this how compression works?

> From a common sense point of view, we should place compression markers on those frames
> that don't change much or at all, so they're like stills and compression won't bother them.
> They will then be compressed the most.

I'm pretty sure you've got it backwards:

[www.kenstone.net]

As far as I know, compression markers don't designate which frames get squashed the most. They designate which frames will be granted special attention during the encoding, forcing the current frame to be an I-frame which will form the basis for the compression of the segment after it.

Frames that don't change much from the frames before don't need to be I-frames. That's why you want those markers at the beginning of each cut, because the information in the frame is changing. I'm no expert, but that's how I understand it.


www.derekmok.com

Derek is right.
Finding the right place for compression markers, is like finding the right keyframes for an animation.
With AE for example you can record the mouse movement to create a motion path, you will get an awfull lot of keayframes, even if you do a smoothing. To make the motion path smoot you have to look where there is a change of direction or speed. If you did it several times, it's easy and your hand cleaned, hand recorded path is much better than any software tool can offer.

As Derek said compression markers do mark a sudden or "remarkable" change. So if there is a dark room and somebody turns on the light, it's time for a marker. If a heater starts glowing slowly is turning the light in the room into a warm red the change is slow and constant and there is no need for a marker, if somebody is walking in front of the heater and covering the the heater there is a need for several markers.
So watch your movie over and over again and try to understand what's changing and how rapidly it is changing - finally the change of the change ;-)
Like a dissolve transition, it may be slow or fast. A fast one will need a marker at the beginning, maybe one in the middle and one at the end a slow one might need none or one in the middle.
As said earlier try to minimize compression markers, otherwise you get either to much I-Frames or keyframes, which will raise the data rate and file size.

So again there is no general receipt for that marker/keyframe handling, you either learn it or not. But working with moving pictures gives you a good start point, since you always think about those things anyway.

Regards
Andreas

>>The book costs too much. I checked on eBay, no luck; there are two for $29 plus shipping. <<

Geez - $29 is too much? That's kind of offensive, really. Like saying, 'Nah, I couldn't be bothered spending my own time and money finding out about this stuff. I'd rather you all spoonfed me.'

Not sure if you read this bit in Michael's post .. "you got to do the work."

This article is pretty old, but it's free. [www.kenstone.net]

Hey, Derek, thanks for the reference to the Ken stone article. I learned a lot today.

Quote

Michael Horton Wrote:
-------------------------------------------------------
Filman
Keep the subject lines short, please. No reason to have so many words in the subject field

Quote

One of the reasons this frequently asked question is never answered to anyones satisfaction is that there is no answer. There is no answer that anyone can give in single post. It's a book of answers, and if you are truly serious about compression, then you got to do the work.

Regarding my "." post: I actually posted something similar to what Mike has been posting & a bit more (worded much differently - Mike has been VERY politically correct & I was a bit more direct) - and I thought better of it & pulled it. The forum will not allow deletion of a post so I have to leave at least one character. No need to read into it

As nicely as I can put it: Filman...just an observation (over a large period of time)...you should do a lot more of your own work & research & stop depending so much on "one line" replies from this forum for all the answers to your filmmaking education. That is my personal opinion.

- Joey

When life gives you dilemmas...make dilemmanade.

Along the lines of what Jude and Joe are saying...

Vic, a lot of your questions can be answered or at least partially addressed if you'd only just search them in Google. I'd entered "compression marker" into Google and Ken Stone's article was at the very top. When confronted with a new problem, you gotta do some legwork on basics, and then you can fill in the holes or ask for clarification on here.


www.derekmok.com

To give you the boiled-down answer, MPEG2 and several of the other compression techniques will do excellent motion and stunning image quality--but not both at the same time.

The idea that you can pick one set of compression specifications at the beginning of a long video is pure amateur. The Big Kids have hardware compressors and a Person Who Sits In A Dark Room. The PWSIADR keeps scrubbing the shot back and forth until they get just the right look with different compression numbers and then they go on to the next shot. That's how to get a 2 or 3 hour movie into a DVD with stunning quality and neck snapping motion effects. Each shot--not each scene--has been tuned to come out right.

The best we can do down here on earth, is guess at the right settings and spray markers around where we think they'll do the most good. Since we can't do it in real time, it's very much like painting the whole house and then letting it dry to make sure we got the color right--and then painting it again if we didn't.

There are fuzzy rules that you can apply, such as the obvious ones of sacrificing image quality in high motion shots, but you do need to see it after the tool is applied to really know what you did.


"What are the best compression settings for my movie?"

That happens about once a week. There is no answer and yes, it's pretty horrifying when you find out how it all really works.


We did get a significant increase in image quality (and decrease in phone calls) when we cut the ropes on DV and went all the way to full production in uncompressed 4:2:2.

MPEG2 was designed to produce an excellent product when it's exposed to real video, not compressed and damaged clips.

Koz

That's an incredible description, Koz. Thanks for taking the time to share your expertise on the subject with me.

As for the others that may have taken offense, I apologize. As Jake Gittes said in Chinatown:

GITTES
What can I say,
when you're right, you're right
... and you're right!"

Gotta remember: Google, Google ... Google everything before you ask a question. I gotta remember that.

And, oh, yes, Andrea, I read your comments very carefully. Now I understand some of the finer points of compression. It's paintaking work and the best way to do it is: well, to quote Joey:

"Try not. DO...or DO NOT. There is no TRY"
- Master Yoda

I gotta remember that too. It's best to read Ken Stone's article and go through my compression markers and place them in the bests spots and just do it the hard way as Koz suggested.

Thanks, Derek, for responding to my questions, even though I don't do my homework. There will be always those students who like to be spoonfed.

Sorry, Jude, for upsetting you. You're right, $30 is not a lot of money in this business, but I couldn't help myself, I had to go for that Mexican dinner at Acapulco -- I had the $3.99 coupon for the second meal free and the $1.50 Margaritas :-)

I will google more, I promise ... and try ... er ... do, do, there's no try, I will buy the books.

Compression is an incredibly complex science.
Its very mathematical.

I have a degree in computer science and I have problems understanding it all.
I bought a book that had hundreds of pages of mathematical algorithms and formulas.

There are specialists and they do this all the time.
They usually have perfectly graded material to work with and then their own algorithms to customise to the footage - so yes it does look damn good.

We can get it to look OK though on the web.
Both FLV (When compressed in Flash video encoder) and .wmv produce excellent results however.

Johan Polhem
Motion Graphics
www.johanpolhem.com

Well, Johan, that's my main concern; to make my videos look good over the web. I suppose that polishing a 3-4 minute clip by using compression markers might be something that I can learn to do.

I had a good experience with my last movie clip that I just posted on the show and tell forum today. I uploaded a 58 MB mpeg 4, h.264 video clip of about 4 minutes into Google, and it came out pretty good; so far the best of all my clips, I think. I don't know what the final weight of the file is, but as I understand it Google converts everything into flash. I'd be curious to know what they did to my 58 MB clip. I did several compressions before I arrived at the last one. They were so terrible, I deleted them right away. I was doing 15 MB versions and the smoke in the nightclub was falling apart before my eyes.

I suppose I need to buy Flash and learn the program. The last time I tried to learn Flash 5 a few years ago, I gave up after a couple of days of fiddling around with titles and graphics. I saw the potential but I just don't have the time to learn it from tutorials. I don't know what I have to do to learn flash, short of blowing a few hundred dollars on instruction. Even a few hundred dollars might not do it though, and I'm not going to go that route anyway.

I think Flash will be a major obstacle for me. There must be a better way ...

What I need to do is figure out a way to produce a special kind of movie format that doesn't get bogged down in technical details.

Jude, I agree with you completely.

I resent being asked questions about topics which I have spent time and money to learn by people who seem unwilling to expend their own effort to learn. I'm happy to help, but meet me halfway; learning is a 2-way street.

The question now becomes: which is costlier, knowledge or ignorance.

Our friend filmman need to decide.

HarryD