[gentoo-dev] FYI: libpng16 won't be able to show some broken icons libpng15 was still able to

[gentoo-dev] FYI: libpng16 won't be able to show some broken icons libpng15 was still able to

[Samuli Suominen]

Short version:

If you see "PNG IDAT errors" like:

program: IDAT: invalid distance too far back `test1.png' @

WARNING **: Icon test1 missing: (0) Fatal error reading PNG image file: 
Decompression error in IDAT

Then you should use >=media-gfx/pngcrush-1.7.57 built with 
USE="-system-libs" to it's using libpng 1.5.15 to correct the IDAT in 
the .png files using command:

$ pngcrush -fix -force old.png new.png

These bogus images were visible with libpng15 and earlier but now that 
improved code was committed these bogus files stopped showing as a 
side-effect. If I understood correct this is not going to be fixed in 
the library itself. Caused by slightly bad combination of libpng and 
zlib and some tool.

Long version:

http://bugs.gentoo.org/466190
http://bugzilla.gnome.org/show_bug.cgi?id=698286
+ the ML links mentioned in the bugs

Figured this is useful for every maintainer to know.

Re: [gentoo-dev] FYI: libpng16 won't be able to show some broken icons libpng15 was still able to

[Ian Stakenvicius]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

On 18/04/13 02:24 PM, Samuli Suominen wrote:
> Short version:
> 
> If you see "PNG IDAT errors" like:
> 
> program: IDAT: invalid distance too far back `test1.png' @
> 
> WARNING **: Icon test1 missing: (0) Fatal error reading PNG image
> file: Decompression error in IDAT
> 
> Then you should use >=media-gfx/pngcrush-1.7.57 built with 
> USE="-system-libs" to it's using libpng 1.5.15 to correct the IDAT
> in the .png files using command:
> 
> $ pngcrush -fix -force old.png new.png
> 
> These bogus images were visible with libpng15 and earlier but now
> that improved code was committed these bogus files stopped showing
> as a side-effect. If I understood correct this is not going to be
> fixed in the library itself. Caused by slightly bad combination of
> libpng and zlib and some tool.
> 
> Long version:
> 
> http://bugs.gentoo.org/466190 
> http://bugzilla.gnome.org/show_bug.cgi?id=698286 + the ML links
> mentioned in the bugs
> 
> Figured this is useful for every maintainer to know.
> 


Is this related to the mozilla issue too?  ie, should the mozilla herd
just fix and supply a 'loading' icon until upstream updates theirs?


-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.19 (GNU/Linux)

iF4EAREIAAYFAlFwPbcACgkQ2ugaI38ACPBmfwD7BIgai/x6GWiGC9KWY7cidnLW
jqwc4BbEnYsHfq7DiAIA/Rny5+nmqiszE6GVYmpX17VP6G6RRsg7hzKRLvW7luIJ
=pr0x
-----END PGP SIGNATURE-----

[gentoo-dev] Re: FYI: libpng16 won't be able to show some broken icons libpng15 was still able to

[Nikos Chantziaras]

On 18/04/13 21:24, Samuli Suominen wrote:
> Short version:
>
> If you see "PNG IDAT errors" like:
>
> program: IDAT: invalid distance too far back `test1.png' @
>
> WARNING **: Icon test1 missing: (0) Fatal error reading PNG image file:
> Decompression error in IDAT

Many of the KDE icons won't display anymore here (for example running 
SMPlayer results in many icons missing from the default interface.)  I 
suppose there's no workaround other than downgrading libpng?

Re: [gentoo-dev] FYI: libpng16 won't be able to show some broken icons libpng15 was still able to

[Samuli Suominen]

On 18/04/13 21:38, Ian Stakenvicius wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA256
>
> On 18/04/13 02:24 PM, Samuli Suominen wrote:
>> Short version:
>>
>> If you see "PNG IDAT errors" like:
>>
>> program: IDAT: invalid distance too far back `test1.png' @
>>
>> WARNING **: Icon test1 missing: (0) Fatal error reading PNG image
>> file: Decompression error in IDAT
>>
>> Then you should use >=media-gfx/pngcrush-1.7.57 built with
>> USE="-system-libs" to it's using libpng 1.5.15 to correct the IDAT
>> in the .png files using command:
>>
>> $ pngcrush -fix -force old.png new.png
>>
>> These bogus images were visible with libpng15 and earlier but now
>> that improved code was committed these bogus files stopped showing
>> as a side-effect. If I understood correct this is not going to be
>> fixed in the library itself. Caused by slightly bad combination of
>> libpng and zlib and some tool.
>>
>> Long version:
>>
>> http://bugs.gentoo.org/466190
>> http://bugzilla.gnome.org/show_bug.cgi?id=698286 + the ML links
>> mentioned in the bugs
>>
>> Figured this is useful for every maintainer to know.
>>
>
>
> Is this related to the mozilla issue too?  ie, should the mozilla herd
> just fix and supply a 'loading' icon until upstream updates theirs?

nope, that is different problem and will be fixed (AFAICS)

Re: [gentoo-dev] Re: FYI: libpng16 won't be able to show some broken icons libpng15 was still able to

[Samuli Suominen]

On 19/04/13 09:53, Nikos Chantziaras wrote:
> On 18/04/13 21:24, Samuli Suominen wrote:
>> Short version:
>>
>> If you see "PNG IDAT errors" like:
>>
>> program: IDAT: invalid distance too far back `test1.png' @
>>
>> WARNING **: Icon test1 missing: (0) Fatal error reading PNG image file:
>> Decompression error in IDAT
>
> Many of the KDE icons won't display anymore here (for example running
> SMPlayer results in many icons missing from the default interface.)  I
> suppose there's no workaround other than downgrading libpng?
>
>

Nope, but it's not clear from your post if we are even discussing the 
same problem...

Identify the file(s), match the files to package(s), and file a bug

- Samuli

[gentoo-dev] Re: FYI: libpng16 won't be able to show some broken icons libpng15 was still able to

[Duncan]

Nikos Chantziaras posted on Fri, 19 Apr 2013 09:53:12 +0300 as excerpted:

> On 18/04/13 21:24, Samuli Suominen wrote:
>> Short version:
>>
>> If you see "PNG IDAT errors" like:
>>
>> program: IDAT: invalid distance too far back `test1.png' @
>>
>> WARNING **: Icon test1 missing: (0) Fatal error reading PNG image file:
>> Decompression error in IDAT
> 
> Many of the KDE icons won't display anymore here (for example running
> SMPlayer results in many icons missing from the default interface.)  I
> suppose there's no workaround other than downgrading libpng?

Hmm. smplayer2 works/looks fine here. (smplayer2, not smplayer, here.)

I think I've seen the error a couple times in konsole (stdout/stderr) 
output, but as anyone who often runs kde apps from konsole surely knows, 
kde apps tend to be incredibly noisy on stdout/stderr anyway, and that 
was before this whole thread, so I just though it was kde being kde.

But I certainly haven't noticed a problem on ~arch, running kde 
4.10.49.9999 (live-4.10-branch, updated a couple times a week) from the 
gentoo/kde overlay, along with libpng-1.6.1-r1, according to equery.

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman

[gentoo-dev] FYI: libpng16 break showfoto (digikam) png save

[vivo75]

Hi Samuli,
  while you're at it do you know what changed between 1.5 and 1.6 in
ICC/color profiles land?

In short digikam is now unable to save png files, I've yet do debug it
so if there is some pointers that could shorten the discovery.
The culprit seem to be "libpng error: profile 'icc': 1B0Ah: invalid
length", and digikam/kipi plugins has been compiled yesterday from git.

thanks


showfoto(7331)/digikam (core) Digikam::EditorWindow::showFileSaveDialog:
Writing file to 
KUrl("file:///home/vivo/docs/NIKON/2013/20130420-test/work/DSC_9665.png")

showfoto(7331)/digikam (core)
Digikam::EditorWindow::selectValidSavingFormat: Trying to find a saving
format with filter =  "*" , targetUrl = 
KUrl("file:///home/vivo/docs/NIKON/2013/20130420-test/work/DSC_9665.png") ,
autoFilter "*|All Files"
showfoto(7331)/digikam (core)
Digikam::EditorWindow::selectValidSavingFormat: KDE Offered types: 
("xpm", "pgm", "pbm", "rgb", "rgba", "bw", "sgi", "tga", "xv", "ppm",
"bmp", "pic", "eps", "epsi", "epsf", "jp2", "mng", "tiff", "tif", "png",
"jpeg", "jpg", "pcx", "xbm")
showfoto(7331)/digikam (core)
Digikam::EditorWindow::selectValidSavingFormat: Writable formats: 
("xpm", "pgm", "pbm", "rgb", "rgba", "bw", "sgi", "tga", "xv", "ppm",
"bmp", "pic", "eps", "epsi", "epsf", "jp2", "mng", "tiff", "tif", "png",
"jpeg", "jpg", "pcx", "xbm", "TIF", "TIFF", "JPG", "JPEG", "JPE", "J2K",
"JP2", "PGF")
showfoto(7331)/digikam (core)
Digikam::EditorWindow::selectValidSavingFormat: Possible format from
local file:  "png"
showfoto(7331)/digikam (core)
Digikam::EditorWindow::selectValidSavingFormat: Using format from target
url  "png"

showfoto(7331)/digikam (core) Digikam::EditorCore::Private::saveAs:
Saving to :
"/home/vivo/docs/NIKON/2013/20130420-test/work/EditorWindow-Ly7331.digikamtempfile.png"
( "png" )
showfoto(7331)/digikam (core) Digikam::EditorCore::Private::saveNext:
Saving file
"/home/vivo/docs/NIKON/2013/20130420-test/work/EditorWindow-Ly7331.digikamtempfile.png"
at -1

showfoto(7331)/digikam (core) Digikam::PNGLoader::save: DImg quality
level:  1
showfoto(7331)/digikam (core) Digikam::PNGLoader::save: PNG compression
level:  9
libpng error: profile 'icc': 1B0Ah: invalid length

showfoto(7331)/digikam (core) Digikam::PNGLoader::save: Internal libPNG
error during writing file. Process aborted!
showfoto(7331)/digikam (core) Digikam::EditorCore::slotImageSaved: error
saving image '
/home/vivo/docs/NIKON/2013/20130420-test/work/EditorWindow-Ly7331.digikamtempfile.png
showfoto(7331)/digikam (core) Digikam::EditorWindow::slotSavingFinished:
"/home/vivo/docs/NIKON/2013/20130420-test/work/EditorWindow-Ly7331.digikamtempfile.png"
false true

Re: [gentoo-dev] FYI: libpng16 break showfoto (digikam) png save

[Markos Chandras]

On 21 April 2013 20:49, vivo75@gmail.com <vivo75@gmail.com> wrote:
> Hi Samuli,
>   while you're at it do you know what changed between 1.5 and 1.6 in
> ICC/color profiles land?
>
> In short digikam is now unable to save png files, I've yet do debug it
> so if there is some pointers that could shorten the discovery.
> The culprit seem to be "libpng error: profile 'icc': 1B0Ah: invalid
> length", and digikam/kipi plugins has been compiled yesterday from git.
>
> thanks
>
>
> showfoto(7331)/digikam (core) Digikam::EditorWindow::showFileSaveDialog:
> Writing file to
> KUrl("file:///home/vivo/docs/NIKON/2013/20130420-test/work/DSC_9665.png")
>
> showfoto(7331)/digikam (core)
> Digikam::EditorWindow::selectValidSavingFormat: Trying to find a saving
> format with filter =  "*" , targetUrl =
> KUrl("file:///home/vivo/docs/NIKON/2013/20130420-test/work/DSC_9665.png") ,
> autoFilter "*|All Files"
> showfoto(7331)/digikam (core)
> Digikam::EditorWindow::selectValidSavingFormat: KDE Offered types:
> ("xpm", "pgm", "pbm", "rgb", "rgba", "bw", "sgi", "tga", "xv", "ppm",
> "bmp", "pic", "eps", "epsi", "epsf", "jp2", "mng", "tiff", "tif", "png",
> "jpeg", "jpg", "pcx", "xbm")
> showfoto(7331)/digikam (core)
> Digikam::EditorWindow::selectValidSavingFormat: Writable formats:
> ("xpm", "pgm", "pbm", "rgb", "rgba", "bw", "sgi", "tga", "xv", "ppm",
> "bmp", "pic", "eps", "epsi", "epsf", "jp2", "mng", "tiff", "tif", "png",
> "jpeg", "jpg", "pcx", "xbm", "TIF", "TIFF", "JPG", "JPEG", "JPE", "J2K",
> "JP2", "PGF")
> showfoto(7331)/digikam (core)
> Digikam::EditorWindow::selectValidSavingFormat: Possible format from
> local file:  "png"
> showfoto(7331)/digikam (core)
> Digikam::EditorWindow::selectValidSavingFormat: Using format from target
> url  "png"
>
> showfoto(7331)/digikam (core) Digikam::EditorCore::Private::saveAs:
> Saving to :
> "/home/vivo/docs/NIKON/2013/20130420-test/work/EditorWindow-Ly7331.digikamtempfile.png"
> ( "png" )
> showfoto(7331)/digikam (core) Digikam::EditorCore::Private::saveNext:
> Saving file
> "/home/vivo/docs/NIKON/2013/20130420-test/work/EditorWindow-Ly7331.digikamtempfile.png"
> at -1
>
> showfoto(7331)/digikam (core) Digikam::PNGLoader::save: DImg quality
> level:  1
> showfoto(7331)/digikam (core) Digikam::PNGLoader::save: PNG compression
> level:  9
> libpng error: profile 'icc': 1B0Ah: invalid length
>
> showfoto(7331)/digikam (core) Digikam::PNGLoader::save: Internal libPNG
> error during writing file. Process aborted!
> showfoto(7331)/digikam (core) Digikam::EditorCore::slotImageSaved: error
> saving image '
> /home/vivo/docs/NIKON/2013/20130420-test/work/EditorWindow-Ly7331.digikamtempfile.png
> showfoto(7331)/digikam (core) Digikam::EditorWindow::slotSavingFinished:
> "/home/vivo/docs/NIKON/2013/20130420-test/work/EditorWindow-Ly7331.digikamtempfile.png"
> false true
>
>

Hi,

I've no idea what the problem is, but what is wrong with the old
traditional way of filing bugs in https://bugs.gentoo.org ?

--
Regards,
Markos Chandras - Gentoo Linux Developer
http://dev.gentoo.org/~hwoarang

Re: [gentoo-dev] FYI: libpng16 won't be able to show some broken icons libpng15 was still able to

[Tobias Klausmann]

Hi! 

Since we probably will have to fix the files coming out of
tarballs until the various upstreams have fixed them, I propose
running a PNG fixer during or after the install phase. Since
having pngcrush as dep for everything is not exactly lightweight,
I hacked together a PNG file fixer in pure(ish, see below)
Python:

http://git.schwarzvogel.de/?p=pngfixer;a=summary

Feel free to send patches

Note that all I wrote is the mind-numbingly simple pngfixer.py
script. The rest of the Python code is excised from PIL
(http://www.pythonware.com/products/pil/index.htm). I didn't have
to change anything there.

Also note that it is not _strictly_ pure Python I'm using: the
PIL components use Python's zlib.so and therefore are subject to
bugs in _that_ code.

Still needed:

- How do we ship this?
- How do we run it for every ebuild? (probably something like
  find /var/tmp/portage/.../image/ -iname \*.png -exec {...}\;)

Regards,
Tobias

Re: [gentoo-dev] FYI: libpng16 won't be able to show some broken icons libpng15 was still able to

[vivo75]

On 04/22/13 13:03, Tobias Klausmann wrote:
> Hi! 
>
> Since we probably will have to fix the files coming out of
> tarballs until the various upstreams have fixed them, I propose
> running a PNG fixer during or after the install phase. Since
> having pngcrush as dep for everything is not exactly lightweight,
> I hacked together a PNG file fixer in pure(ish, see below)
> Python:
No please, this real should stay really far from any package manager.
There are only two sane options:
1) fix libpng to be backward compatible
2) fix the package, and re-package / add a package for broken images

But please don't push on all our user systems something like this.

>
> http://git.schwarzvogel.de/?p=pngfixer;a=summary
>
> Feel free to send patches
>
> Note that all I wrote is the mind-numbingly simple pngfixer.py
> script. The rest of the Python code is excised from PIL
> (http://www.pythonware.com/products/pil/index.htm). I didn't have
> to change anything there.
>
> Also note that it is not _strictly_ pure Python I'm using: the
> PIL components use Python's zlib.so and therefore are subject to
> bugs in _that_ code.

Thanks for taking the time to write this it could be useful for
developers, but hopefully not as initially intended ;-)

>
> Still needed:
>
> - How do we ship this?
> - How do we run it for every ebuild? (probably something like
>   find /var/tmp/portage/.../image/ -iname \*.png -exec {...}\;)
>
> Regards,
> Tobias
Regards,
Francesco

Re: [gentoo-dev] FYI: libpng16 won't be able to show some broken icons libpng15 was still able to

[Samuli Suominen]

On 23/04/13 13:24, vivo75@gmail.com wrote:
> On 04/22/13 13:03, Tobias Klausmann wrote:
>> Hi!
>>
>> Since we probably will have to fix the files coming out of
>> tarballs until the various upstreams have fixed them, I propose
>> running a PNG fixer during or after the install phase. Since
>> having pngcrush as dep for everything is not exactly lightweight,
>> I hacked together a PNG file fixer in pure(ish, see below)
>> Python:
> No please, this real should stay really far from any package manager.
> There are only two sane options:
> 1) fix libpng to be backward compatible
> 2) fix the package, and re-package / add a package for broken images
>
> But please don't push on all our user systems something like this.

I appericiate the work done by Tobias utmost too but I have to agree 
this is not something I want to see running automatically, or even
from within ebuilds.

Perhaps if upstream writes the minimalized C version with no 
dependencies (or with only libpng dependency), we could add custom 
fix_IDAT() function but even then I'd prefer we fix the packages instead 
for SRC_URI=

Trying to push time for going through some of the packages at:
https://bugs.gentoo.org/show_bug.cgi?id=466190#c12

But help is needed and I bet some maintainers don't want others messing 
with their packages in the first place.
The fix is the mentioned script OR pngcrush with USE="-system-libs" and 
`pngcrush -fix -force`

>
>>
>> http://git.schwarzvogel.de/?p=pngfixer;a=summary
>>
>> Feel free to send patches
>>
>> Note that all I wrote is the mind-numbingly simple pngfixer.py
>> script. The rest of the Python code is excised from PIL
>> (http://www.pythonware.com/products/pil/index.htm). I didn't have
>> to change anything there.
>>
>> Also note that it is not _strictly_ pure Python I'm using: the
>> PIL components use Python's zlib.so and therefore are subject to
>> bugs in _that_ code.
>
> Thanks for taking the time to write this it could be useful for
> developers, but hopefully not as initially intended ;-)

+1+1+1

Thanks everyone,

- Samuli

Re: [gentoo-dev] FYI: libpng16 won't be able to show some broken icons libpng15 was still able to

[Ulrich Mueller]

>>>>> On Tue, 23 Apr 2013, Samuli Suominen wrote:

> I appericiate the work done by Tobias utmost too but I have to agree
> this is not something I want to see running automatically, or even
> from within ebuilds.

+1

In Tobias's list, I count some 80 packages that need fixing. That's
way too few to merit a general solution. Also this number will
decrease when files are fixed upstream.

Ulrich

Re: [gentoo-dev] FYI: libpng16 won't be able to show some broken icons libpng15 was still able to

[Tobias Klausmann]

Hi! 

On Tue, 23 Apr 2013, Ulrich Mueller wrote:
> > I appericiate the work done by Tobias utmost too but I have to agree
> > this is not something I want to see running automatically, or even
> > from within ebuilds.
> 
> +1
> 
> In Tobias's list, I count some 80 packages that need fixing. That's
> way too few to merit a general solution. Also this number will
> decrease when files are fixed upstream.

I see two problems with this approach:

- What do we do with unresponsive-yet-not-dead upstreams?
- What about future packages that ship broken files? I mean not
  just existing packages that keep issuing broken PNGs but also
  future packages that we just didn't cover now?

The former has two and a half solutions:

- Wait until itmagically fixes itself or upstream comes around.
  This is only 1/2 a solution, IMO. 

- Add a separate tarball or the like that the Gentoo maintainer
  generates from the broken PNGs. Use this tarball to overwrite
  the broken results of equivalent_of("make install").

- Have a convenience function that can be used for known-bad
  packages to fix broken IDATs. Basically calling my script or
  the binary Samuli mentioned.

The second problem, however, is trickier. We can rely on people
noticing the error messages/broken packages and hope they file
bugs. The other option is to have a QA-like check for it, again
using the simplest possible binary to do so.

Regards,
Tobias

Re: [gentoo-dev] FYI: libpng16 won't be able to show some broken icons libpng15 was still able to

[Samuli Suominen]

On 23/04/13 14:19, Tobias Klausmann wrote:
> bugs. The other option is to have a QA-like check for it, again
> using the simplest possible binary to do so.

Now that you said it... "QA"

Have your script (the one you generated the current list with) to 
generate it to http://qa-reports.gentoo.org/
Like once a week now and once a month later
That'd be awesome, imho

Re: [gentoo-dev] FYI: libpng16 won't be able to show some broken icons libpng15 was still able to

[Walter Dnes]

On Tue, Apr 23, 2013 at 01:19:05PM +0200, Tobias Klausmann wrote

> The second problem, however, is trickier. We can rely on people
> noticing the error messages/broken packages and hope they file
> bugs. The other option is to have a QA-like check for it, again
> using the simplest possible binary to do so.

  Will those messages be during the ebuild?  There are currently
messages like...

> QA Notice: Package triggers severe warnings which indicate that it
>            may exhibit random runtime failures.
> ExtensionSubtables.cpp:32:31: warning: dereferencing type-punned
> pointer will break strict-aliasing rules
> 
> Please do not file a Gentoo bug and instead report the above
> QA issues directly to the upstream developers of this software.
> Homepage: http://www.icu-project.org/

  A similar message, about non-displaying icons instead of "random
runtime failures" may be in order.  At the very least, it will let end
users know who to report the problem to.

-- 
Walter Dnes <waltdnes@waltdnes.org>
I don't run "desktop environments"; I run useful applications

Re: [gentoo-dev] FYI: libpng16 won't be able to show some broken icons libpng15 was still able to

[Tobias Klausmann]

Hi!

On Tue, 23 Apr 2013, Walter Dnes wrote:
> On Tue, Apr 23, 2013 at 01:19:05PM +0200, Tobias Klausmann wrote
> > The second problem, however, is trickier. We can rely on people
> > noticing the error messages/broken packages and hope they file
> > bugs. The other option is to have a QA-like check for it, again
> > using the simplest possible binary to do so.
>
>   Will those messages be during the ebuild?  There are currently
> messages like...
>
> > QA Notice: Package triggers severe warnings which indicate that it
> >            may exhibit random runtime failures.
> > ExtensionSubtables.cpp:32:31: warning: dereferencing type-punned
> > pointer will break strict-aliasing rules
> >
> > Please do not file a Gentoo bug and instead report the above
> > QA issues directly to the upstream developers of this software.
> > Homepage: http://www.icu-project.org/
>
>   A similar message, about non-displaying icons instead of "random
> runtime failures" may be in order.  At the very least, it will let end
> users know who to report the problem to.

Exactly. I'd envision something like this:

[snip]
QA Notice: Package installs broken PNG files that may cause
	   warnings, errors or fail to display completely.
[... list of bad PNGs ...]
	   See http://to-be-written-page/ for more information
	   on how to fix these files.

Please do not file a Gentoo bug and instead report the above
QA issues directly to the upstream developers of this
software. Homepage: http://project-homepage/
[pins]

Regards,
Tobias