What program is best at JPEG compression?

In rec.photo.digital Martin Brown <|||newspam|||@nezumi.demon.co.uk> wrote:
>>
>> I scaled down a square portion of the Macbeth chart, leftmost colors,
>> but Photoshop CS2 refuses to save such a small (16x16) file as JPEG,
>> or perhaps Photoshop thinks it is a diagram.
>
> That makes sense - since even in compact "save for web" mode Photoshop
> adds a small essay to the header of every JPEG it saves. 32bit BMP
> files could prossibly be smaller for anything < 32x32. And all bets
> are off it is adds a 160x120 thumbnail in as well!

Ha ha! Funny. The problem was that even in PNG it was Indexed color
so after switching Photoshop to RGB colorspace, it could save JPEG.

Here they are, Photoshop quality 0 to 12. It's interesting that, for
this image, PS quality=0 looks better than quality=2.

http://www.cacreeks.com/psp

Also, as previously observed, Photoshop quality < 7 is fairly worthless.
Quality 6 is only a bit smaller than 7 but looks much worse. I have
already run jpegdump so I know 7-6 is where Photoshop switches from
1x1 chroma to 2x2 chroma subsampling. TIA.
 >> Stay informed about: What program is best at JPEG compression? 

On Aug 4, 3:17 am, Bill Tuthill <ccree... RemoveThis @yahoo.com> wrote:
> In rec.photo.digital Martin Brown <|||newspam...@nezumi.demon.co.uk> wrote:

> > That makes sense - since even in compact "save for web" mode Photoshop
> > adds a small essay to the header of everyJPEGit saves. 32bit BMP
> > files could prossibly be smaller for anything < 32x32. And all bets
> > are off it is adds a 160x120 thumbnail in as well!
>
> Ha ha! Funny. The problem was that even in PNG it was Indexed color

Modern PhotoShop JPEGs are more than a bit voluminous. Each one
contains 2 identical thumnails in the header. And the highest quality
image Q=12 has a main JPEG stream 542 bytes long in a file totalling
over 22500 bytes. A back of the envelope calculation says it should
take about 1140 bytes for a genuine fully encoded legal JFIF file.

> Here they are, Photoshop quality 0 to 12. It's interesting that, for
> this image, PS quality=0 looks better than quality=2.

Sometimes happens. Some line art can become exact for lucky
combinations of custom quality table.
>
> http://www.cacreeks.com/psp
>
> Also, as previously observed, Photoshop quality < 7 is fairly worthless.
> Quality 6 is only a bit smaller than 7 but looks much worse. I have
> already run jpegdump so I know 7-6 is where Photoshop switches from
> 1x1 chroma to 2x2 chroma subsampling. TIA.

Thanks very much for the files.

OK here are the results (they match the old quality levels of
PhotoShop 5 + 2). Two extra compressed for web use new levels 0 and 1
have been added. The data below are based on least squares fitting of
the IJG scale factor to the actual PhotoShop quantisation tables with
the total residual as a measure of goodness (or badness) of fit.

I have done Y and C using IJG quality (100=best) separately for
reasons that should be obvious:

Level Y resid C resid
0 35 38k 81 16k
1 38 29k 82 13k
2 47 14k 84 7k
3 60 5k 86 4k
4 67 2250 87 2187
5 74 1003 89 930
6 82 254 91 299
* changes to 1x1 sampling
7 77 708 87 3200
8 85 140 89 1070
9 92 83 90 304
10 95 64 93 32
11 97 39 96 0
12 99 5 99 49

The highest quality 10,11,12 are a pretty good match to IJG. PhotoShop
has finer high frequency quantisation than the IJG tables at the lower
quality settings with a large triangular chunk of the matrix set to
12. The IJG values there are much coarser and explain the huge
residuals for low quality settings. The differences are so huge that
least squares may not be an appropriate fitting criterion - minimum 1
norm might be better.

I reckon PhotoShop tries a bit too hard at accurate quantisation of
the colour by using very high quality settings even at its lowest
quality. The luminence quantisation doesn't catch up until level 8.

Regards,
Martin Brown
 >> Stay informed about: What program is best at JPEG compression? 

In rec.photo.digital Martin Brown <|||newspam|||@nezumi.demon.co.uk> wrote:
>
> Modern PhotoShop JPEGs are more than a bit voluminous. Each one
> contains 2 identical thumnails in the header. And the highest quality
> image Q=12 has a main JPEG stream 542 bytes long in a file totalling
> over 22500 bytes. A back of the envelope calculation says it should
> take about 1140 bytes for a genuine fully encoded legal JFIF file.

Two identical thumbnails, whaaaaat, why?
It seems one can disable thumbnails by clicking off the Preview checkbox.

This supports earlier assertions on this thread not to use Photoshop JPEG,
although now I'm interested in investigating Photoshop SaveForWeb. It
might have been improved since it first came out, but at that time, it
was not able to set 1x1 chroma, only 2x2, which seemed fairly idiotic
at quality levels above IJG 85.

> OK here are the results (they match the old quality levels of
> PhotoShop 5 + 2). Two extra compressed for web use new levels 0 and 1
> have been added. The data below are based on least squares fitting of
> the IJG scale factor to the actual PhotoShop quantisation tables with
> the total residual as a measure of goodness (or badness) of fit.

By "Photoshop 5 + 2" you mean thru version 7? I thought PS version 6
had quality 1, but not 0.

> I have done Y and C using IJG quality (100=best) separately for
> reasons that should be obvious:
> Level Y resid C resid
> 0 35 38k 81 16k
> 1 38 29k 82 13k
> 2 47 14k 84 7k
> 3 60 5k 86 4k
> 4 67 2250 87 2187
> 5 74 1003 89 930
> 6 82 254 91 299
> * changes to 1x1 sampling
> 7 77 708 87 3200
> 8 85 140 89 1070
> 9 92 83 90 304
> 10 95 64 93 32
> 11 97 39 96 0
> 12 99 5 99 49

Thanks a lot, this is a valuable reference for the future.
Presumably Y=luminance, C=chroma. What does resid(ual) indicate?
Are jpegdump's qtable0 and qtable1 related to Y and C encodings?

> The highest quality 10,11,12 are a pretty good match to IJG. PhotoShop
> has finer high frequency quantisation than the IJG tables at the lower
> quality settings with a large triangular chunk of the matrix set to
> 12. The IJG values there are much coarser and explain the huge
> residuals for low quality settings. The differences are so huge that
> least squares may not be an appropriate fitting criterion - minimum 1
> norm might be better.

Yes, the fitting gets worse as the quality levels decrease below 6.
Hmm... I infer resid(ual) indicates accuracy of least squares fitting.

> I reckon PhotoShop tries a bit too hard at accurate quantisation of
> the colour by using very high quality settings even at its lowest
> quality. The luminence quantisation doesn't catch up until level 8.

This might be why Photoshop JPEG looks better at lower quality levels
than IJG, but the lower quality levels look pretty bad anyhow, so
particularly in these days of higher bandwidth, they are irrelevant.
 >> Stay informed about: What program is best at JPEG compression? 

On Aug 6, 7:57 pm, Bill Tuthill <ccree... DeleteThis @yahoo.com> wrote:
> In rec.photo.digital Martin Brown <|||newspam...@nezumi.demon.co.uk> wrote:
>
> > Modern PhotoShop JPEGs are more than a bit voluminous. Each one
> > contains 2 identical thumnails in the header. And the highest quality
> > image Q=12 has a mainJPEGstream 542 bytes long in a file totalling
> > over 22500 bytes. A back of the envelope calculation says it should
> > take about 1140 bytes for a genuine fully encoded legal JFIF file.
>
> Two identical thumbnails, whaaaaat, why?

Who knows. If I were to make a cynical guess it is that one of their
older versions of the program crashes horribly if it doesn't find a
thumbnail labelled "Photoshop 3.0" in the header somewhere.

> It seems one can disable thumbnails by clicking off the Preview checkbox.
>
> > OK here are the results (they match the old quality levels of
> > PhotoShop 5 + 2). Two extra compressed for web use new levels 0 and 1
> > have been added. The data below are based on least squares fitting of
> > the IJG scale factor to the actual PhotoShop quantisation tables with
> > the total residual as a measure of goodness (or badness) of fit.
>
> By "Photoshop 5 + 2" you mean thru version 7? I thought PS version 6
> had quality 1, but not 0.

Dunno about PS6, but PS5 will definitely save at quality level 0 if
you type a zero in the text box. But the old Level 0 is now called 2
in the later versions of software with 12 levels.

> > I have done Y and C using IJG quality (100=best) separately for
> > reasons that should be obvious:
> > Level Y resid C resid
> > 0 35 38k 81 16k
> > 1 38 29k 82 13k
> > 2 47 14k 84 7k
> > 3 60 5k 86 4k
> > 4 67 2250 87 2187
> > 5 74 1003 89 930
> > 6 82 254 91 299
> > * changes to 1x1 sampling
> > 7 77 708 87 3200
> > 8 85 140 89 1070
> > 9 92 83 90 304
> > 10 95 64 93 32
> > 11 97 39 96 0
> > 12 99 5 99 49
>
> Thanks a lot, this is a valuable reference for the future.
> Presumably Y=luminance, C=chroma. What does resid(ual) indicate?
> Are jpegdump's qtable0 and qtable1 related to Y and C encodings?

Yes. There is a possible catch here that at the lower qualities the
quantisation matrix is so unlike the Photoshop one that the least
square solution may not be the optimum approximation. There are a
handful of optimisers about that will tweak the quantisation matrix to
match a given image - NASA holds a US patent on it. eg
http://vision.arc.nasa.gov/dctune/
>
> > The highest quality 10,11,12 are a pretty good match to IJG. PhotoShop
> > has finer high frequency quantisation than the IJG tables at the lower
> > quality settings with a large triangular chunk of the matrix set to
> > 12. The IJG values there are much coarser and explain the huge
> > residuals for low quality settings. The differences are so huge that
> > least squares may not be an appropriate fitting criterion - minimum 1
> > norm might be better.
>
> Yes, the fitting gets worse as the quality levels decrease below 6.
> Hmm... I infer resid(ual) indicates accuracy of least squares fitting.

Yes. Larger means a poor fit, zero means an exact fit. And because it
is the square of the residuals you can get a rough idea of how bad it
is by dividing that number by 64 and taking the square root to get an
average rms error. However, the errors are systematically distributed
so this is rather crude.

Residual 5 means one difference of 1 and one of 2. Residual 64 means
an average error of 1 on all 64 cells, but could be just one cell out
by 8. In reality it will be something inbetween these two extreme
cases.

The worst fit of level 0 luminence with residuals of 38000 translates
to an average error of 24 across every channel. In practice it is
actually due to some very large differences in the quantisation of the
higher spatial frequencies.

Regards,
Martin Brown
 >> Stay informed about: What program is best at JPEG compression?