X264 - Subme - Comparison

x264 Settings Comparison Part : 2 - Sub Pixel Motion Estimation

Subme define the subpixel estimation complexity. Higher values are better the question is by how much and what is the trade off.

Levels 1-5 control the subpixel refinement strength.
Level 6 enables RDO(Rate–distortion optimization) for mode decision, with level 8 enabling RDO for motion vectors and intra prediction modes. RDO levels are significantly slower than the rest.

Values available:
0 - Full pixel Only.
1 - Quater pixel(QPel) SAD 1 iteration
2 - Quater pixel SAD 2 iteration
3 - Half pixel(Hpel) on MB then QPel
4 - QPel Always
5 - Multi QPel + bi - directional motion estimation
6 - Rate Distortion(RD) on I/P
7 - RD on all frames(Default, Medium)
8 - RDO on I/P frames
9 - RDO on all frames
10 - QP-RD(Requires trellis=2, aq-mode(Adaptive Quantization Mode )>0 )*
11 - Full RD(No early termination)*

*Update found out that Subme10,11 requires other settings, as noted in post previously, will update post with new data.

Futurama SD

 Ok so lets take a look at the performance of sub pixel motion estimation

As you can see from the above graphs subme:0-3 shows significant performance improvement, with 0 and 1 have a noticeable adverse affect on compression, whilst 2-3 having better compression. However and this is important subme lower than 4 does have visible artefacts and I would not recommend any setting lower than 5.

We can also see no difference in subme 9,10,11 these values might require other additional settings for them to take effect. Safe settings are 5-8.

So lets take a look at the screenshots where you can decide for yourself.
Still Frame   

Subme=0

Subme=1

Subme=2

Subme=3

Subme=4

Subme=5

Subme=6

Subme=7(Medium)

 

Subme=8

Subme=9

Most of the still frames have similar quality, looking closely higher subme retains more detail.

Smaller Motion Detail

Subme=7(Medium)

Subme=0

Subme=1

Subme=2

Subme=3

Subme=4

Subme=5

Subme=6

Subme=8

Subme=9

 

 

 

It is difficult to see the difference between subme settings and animation type videos, however the higher settings do introduce less artfefacts and the image quality is consistently cleaner. 

Real Steel SD

 

As you can see the trend is similar to that of the previous one, with Subme 0-3 consitently faster at a loss of compression and 9,10,11 being identical.

 

Still Frame

subme=0

subme=1

subme=2

subme=3

subme=4

subme=5

subme=6

subme=8

subme=9

 

 

 

subme=7(Medium)

 

 

 

 

 

 

 

 

Looking at non animation type videos we can clearly see a visible difference between the subme settings with subme:0,1,2 retaining significantly less finer detail than the higher settings. From subme=7 is becomes a lot harder to notice the loss of fine detail.

 

  Smaller Motion Detail

Subme=7(Medium)

Subme=0

Subme=1

Subme=2

Subme=3

Subme=4

Subme=5

Subme=6

Subme=8

Subme=9

 

Again we can notice a diffirence between the various subme values, and again from subme=7 on ward it becomes very hard to tell which is better quality.

Real Steel HD

Interesting to see that according to CRF the lower subme had marginally better compression than subme=7, however there might be noticeable visual artefacts that CRF can't account for.

Still Frame

Subme=7(Medium)

Subme=0

Subme=1

Subme=2

Subme=3

Subme=4

Subme=5

Subme=6

Subme=8

Subme=9

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Although hard to see if the still frames but during video playback subme settings of 1,2,3 add VERY noticeable artefacts, which looked a bit like film noise, whilst loosing the original film noise. And again from around subme 7 the frames are near identical.

Smaller Motion Detail

Subme=7(Medium)

Subme=0

Subme=1

Subme=2

Subme=3

Subme=4

 

 

 

Subme=5

Subme=6

 

 

Subme=8

 

Subme=9

 

 

 

 

 

 

 

With HD videos it the image quality is a lot closer to each other, but looking closely one can notice the difference, and progressively is becomes harder to tell when you stop  noticing any video improvement. Again subme 0,1,2  did have noticeable problems with playback. And given the previous tests subme=7 would be the safest point when quality is concerned and subme=5 when more speed is needed.

Conclusion

As seen animation benefits less than life action videos, and during HD video the ability to notice the difference becomes significantly harder. With no noticable impact of subme 9,10,11 again this might be because they require other settings to work.

So which settings are safe?
when I talk about safe I talk about the trade off of quality that said no setting below 5 should ever be used.
 
And what settings are recommended?
Subme 5,6  if need more speed
Subme 7 if you want to retain finer details
Subme 8+ if you're paranoid 

Navigation

1. Introduction
2. Presets
3. Subme