Before you even think about encoding a movie, you need to take several preliminary steps.

The first and most important step before you encode should be determining what type of content you are dealing with. If your source material comes from DVD or broadcast/cable/satellite TV, it will be stored in one of two formats: NTSC for North America and Japan, PAL for Europe, etc. It is important to realize, however, that this is just the formatting for presentation on a television, and often does not correspond to the original format of the movie. Experience shows that NTSC material is a lot more difficult to encode, because there more elements to identify in the source. In order to produce a suitable encode, you need to know the original format. Failure to take this into account will result in various flaws in your encode, including ugly combing (interlacing) artifacts and duplicated or even lost frames. Besides being ugly, the artifacts also harm coding efficiency: You will get worse quality per unit bitrate.

It is possible to encode your movie at a wide range of qualities. With modern video encoders and a bit of pre-codec compression (downscaling and denoising), it is possible to achieve very good quality at 700 MB, for a 90-110 minute widescreen movie. Furthermore, all but the longest movies can be encoded with near-perfect quality at 1400 MB.

There are three approaches to encoding the video: constant bitrate (CBR), constant quantizer, and multipass (ABR, or average bitrate).

The complexity of the frames of a movie, and thus the number of bits required to compress them, can vary greatly from one scene to another. Modern video encoders can adjust to these needs as they go and vary the bitrate. In simple modes such as CBR, however, the encoders do not know the bitrate needs of future scenes and so cannot exceed the requested average bitrate for long stretches of time. More advanced modes, such as multipass encode, can take into account the statistics from previous passes; this fixes the problem mentioned above.

In each of these modes, the video codec (such as libavcodec) breaks the video frame into 16x16 pixel macroblocks and then applies a quantizer to each macroblock. The lower the quantizer, the better the quality and higher the bitrate. The method the movie encoder uses to determine which quantizer to use for a given macroblock varies and is highly tunable. (This is an extreme over-simplification of the actual process, but the basic concept is useful to understand.)

When you specify a constant bitrate, the video codec will encode the video, discarding detail as much as necessary and as little as possible in order to remain lower than the given bitrate. If you truly do not care about file size, you could as well use CBR and specify a bitrate of infinity. (In practice, this means a value high enough so that it poses no limit, like 10000Kbit.) With no real restriction on bitrate, the result is that the codec will use the lowest possible quantizer for each macroblock (as specified by vqmin for libavcodec, which is 2 by default). As soon as you specify a low enough bitrate that the codec is forced to use a higher quantizer, then you are almost certainly ruining the quality of your video. In order to avoid that, you should probably downscale your video, according to the method described later on in this guide. In general, you should avoid CBR altogether if you care about quality.

With constant quantizer, the codec uses the same quantizer, as specified by the vqscale option (for libavcodec), on every macroblock. If you want the highest quality rip possible, again ignoring bitrate, you can use vqscale=2. This will yield the same bitrate and PSNR (peak signal-to-noise ratio) as CBR with vbitrate=infinity and the default vqmin of 2.

The problem with constant quantizing is that it uses the given quantizer whether the macroblock needs it or not. That is, it might be possible to use a higher quantizer on a macroblock without sacrificing visual quality. Why waste the bits on an unnecessarily low quantizer? Your CPU has as many cycles as there is time, but there is only so many bits on your hard disk.

With a two pass encode, the first pass will rip the movie as though it were CBR, but it will keep a log of properties for each frame. This data is then used during the second pass in order to make intelligent decisions about which quantizers to use. During fast action or high detail scenes, higher quantizers will likely be used, and during slow moving or low detail scenes, lower quantizers will be used. Normally, the amount of motion is much more important than the amount of detail.

If you use vqscale=2, then you are wasting bits. If you use vqscale=3, then you are not getting the highest quality rip. Suppose you rip a DVD at vqscale=3, and the result is 1800Kbit. If you do a two pass encode with vbitrate=1800, the resulting video will have higher quality for the same bitrate.

Since you are now convinced that two pass is the way to go, the real question now is what bitrate to use? The answer is that there is no single answer. Ideally you want to choose a bitrate that yields the best balance between quality and file size. This is going to vary depending on the source video.

If size does not matter, a good starting point for a very high quality rip is about 2000Kbit plus or minus 200Kbit. For fast action or high detail source video, or if you just have a very critical eye, you might decide on 2400 or 2600. For some DVDs, you might not notice a difference at 1400Kbit. It is a good idea to experiment with scenes at different bitrates to get a feel.

If you aim at a certain size, you will have to somehow calculate the bitrate. But before that, you need to know how much space you should reserve for the audio track(s), so you should rip those first. You can compute the bitrate with the following equation: bitrate = (target_size_in_Mbytes - sound_size_in_Mbytes) * 1024 * 1024 / length_in_secs * 8 / 1000 For instance, to squeeze a two-hour movie onto a 702MB CD, with 60MB of audio track, the video bitrate will have to be: (702 - 60) * 1024 * 1024 / (120*60) * 8 / 1000 = 740kbps

Due to the nature of MPEG-type compression, there are various constraints you should follow for maximal quality. MPEG splits the video up into 16x16 squares called macroblocks, each composed of 4 8x8 blocks of luma (intensity) information and two half-resolution 8x8 chroma (color) blocks (one for red-cyan axis and the other for the blue-yellow axis). Even if your movie width and height are not multiples of 16, the encoder will use enough 16x16 macroblocks to cover the whole picture area, and the extra space will go to waste. So in the interests of maximizing quality at a fixed file size, it is a bad idea to use dimensions that are not multiples of 16.

Most DVDs also have some degree of black borders at the edges. Leaving these in place will hurt quality a lot in several ways.

In addition to frequency domain transforms, MPEG-type compression uses motion vectors to represent the change from one frame to the next. Motion vectors naturally work much less efficiently for new content coming in from the edges of the picture, because it is not present in the previous frame. As long as the picture extends all the way to the edge of the encoded region, motion vectors have no problem with content moving out the edges of the picture. However, in the presence of black borders, there can be trouble:

For all of these reasons, it is recommended to fully crop black borders. Further, if there is an area of noise/distortion at the edge of the picture, cropping this will improve encoding efficiency as well. Videophile purists who want to preserve the original as close as possible may object to this cropping, but unless you plan to encode at constant quantizer, the quality you gain from cropping will considerably exceed the amount of information lost at the edges.

Recall from the previous section that the final picture size you encode should be a multiple of 16 (in both width and height). This can be achieved by cropping, scaling, or a combination of both.

When cropping, there are a few guidelines that must be followed to avoid damaging your movie. The normal YUV format, 4:2:0, stores chroma (color) information subsampled, i.e. chroma is only sampled half as often in each direction as luma (intensity) information. Observe this diagram, where L indicates luma sampling points and C chroma.

As you can see, rows and columns of the image naturally come in pairs. Thus your crop offsets and dimensions must be even numbers. If they are not, the chroma will no longer line up correctly with the luma. In theory, it is possible to crop with odd offsets, but it requires resampling the chroma which is potentially a lossy operation and not supported by the crop filter.

Further, interlaced video is sampled as follows:

As you can see, the pattern does not repeat until after 4 lines. So for interlaced video, your y-offset and height for cropping must be multiples of 4.

Native DVD resolution is 720x480 for NTSC, and 720x576 for PAL, but there is an aspect flag that specifies whether it is full-screen (4:3) or wide-screen (16:9). Many (if not most) widescreen DVDs are not strictly 16:9, and will be either 1.85:1 or 2.35:1 (cinescope). This means that there will be black bands in the video that will need to be cropped out.

MPlayer provides a crop detection filter that will determine the crop rectangle (-vf cropdetect). Run MPlayer with -vf cropdetect and it will print out the crop settings to remove the borders. You should let the movie run long enough that the whole picture area is used, in order to get accurate crop values.

Then, test the values you get with MPlayer, using the command line which was printed by cropdetect, and adjust the rectangle as needed. The rectangle filter can help by allowing you to interactively position the crop rectangle over your movie. Remember to follow the above divisibility guidelines so that you do not misalign the chroma planes.

In certain cases, scaling may be undesirable. Scaling in the vertical direction is difficult with interlaced video, and if you wish to preserve the interlacing, you should usually refrain from scaling. If you will not be scaling but you still want to use multiple-of-16 dimensions, you will have to overcrop. Do not undercrop, since black borders are very bad for encoding!

Because MPEG-4 uses 16x16 macroblocks, you will want to make sure that each dimension of the video you are encoding is a multiple of 16 or else you will be degrading quality, especially at lower bitrates. You can do this by rounding the width and height of the crop rectangle down to the nearest multiple of 16. As stated earlier, when cropping, you will want to increase the Y offset by half the difference of the old and the new height so that the resulting video is taken from the center of the frame. And because of the way DVD video is sampled, make sure the offset is an even number. (In fact, as a rule, never use odd values for any parameter when you are cropping and scaling video.) If you are not comfortable throwing a few extra pixels away, you might prefer to scale the video instead. We will look at this in our example below. You can actually let the cropdetect filter do all of the above for you, as it has an optional round parameter that is equal to 16 by default.

Also, be careful about "half black" pixels at the edges. Make sure you crop these out too, or else you will be wasting bits there that are better spent elsewhere.

After all is said and done, you will probably end up with video whose pixels are not quite 1.85:1 or 2.35:1, but rather something close to that. You could calculate the new aspect ratio manually, but MEncoder offers an option for libavcodec called autoaspect that will do this for you. Absolutely do not scale this video up in order to square the pixels unless you like to waste your hard disk space. Scaling should be done on playback, and the player will use the aspect stored in the AVI to determine the correct resolution. Unfortunately, not all players enforce this auto-scaling information, therefore you may still want to rescale.

If you will not be encoding in constant quantizer mode, you need to select a bitrate. The concept of bitrate is quite simple. It is the (average) number of bits that will be consumed to store your movie, per second. Normally bitrate is measured in kilobits (1000 bits) per second. The size of your movie on disk is the bitrate times the length of the movie in time, plus a small amount of "overhead" (see the section on the AVI container for instance). Other parameters such as scaling, cropping, etc. will not alter the file size unless you change the bitrate as well!

Bitrate does not scale proportionally to resolution. That is to say, a 320x240 file at 200 kbit/sec will not be the same quality as the same movie at 640x480 and 800 kbit/sec! There are two reasons for this:

Past guides have recommended choosing a bitrate and resolution based on a "bits per pixel" approach, but this is usually not valid due to the above reasons. A better estimate seems to be that bitrates scale proportional to the square root of resolution, so that 320x240 and 400 kbit/sec would be comparable to 640x480 at 800 kbit/sec. However this has not been verified with theoretical or empirical rigor. Further, given that movies vary greatly with regard to noise, detail, degree of motion, etc., it is futile to make general recommendations for bits per length-of-diagonal (the analog of bits per pixel, using the square root).

So far we have discussed the difficulty of choosing a bitrate and resolution.