[gentoo-user] Some files in /usr/share/doc are not compressed

[gentoo-user] Some files in /usr/share/doc are not compressed

[Francesco Turco]

Hello.

On my system Portage uses the following two variables for compressing
files in /usr/share/doc:

> $ portageq envvar PORTAGE_COMPRESS
> xz

> $ portageq envvar PORTAGE_COMPRESS_EXCLUDE_SUFFIXES
> css gif htm[l]? jp[e]?g js pdf png

It seems anyway that some files are not compressed:

> $ find /usr/share/doc -type f -regextype posix-extended ! -regex ".*\.(css|gif|htm[l]?|jp[e]?g|js|pdf|png|xz)" | wc -l
> 79

I won't list all of them here, just some examples:

> /usr/share/doc/gnome-shell-3.4.2/AUTHORS
> /usr/share/doc/udisks-2.0.0/html/udisks2/index.sgml
> /usr/share/doc/groff-1.21-r1/pic.ps
> /usr/share/doc/automake-1.12.5/amhello-1.0.tar.gz

My goal is to have everything under /usr/share/doc compressed with xz,
or at least to understand why something is being excluded from
compression. Perhaps it is due to some additional rules I'm not aware
of, or because of some bugs.

I checked the ebuilds of the packages the previous files belong to, but
I found nothing interesting.

It would be great if anyone who is interested could check his/her own
system too.

Thank you.

Re: [gentoo-user] Some files in /usr/share/doc are not compressed

[Alan McKinnon]

On Wed, 19 Dec 2012 17:41:30 +0100
Francesco Turco <fturco@fastmail.fm> wrote:

> Hello.
> 
> On my system Portage uses the following two variables for compressing
> files in /usr/share/doc:
> 
> > $ portageq envvar PORTAGE_COMPRESS
> > xz
> 
> > $ portageq envvar PORTAGE_COMPRESS_EXCLUDE_SUFFIXES
> > css gif htm[l]? jp[e]?g js pdf png
> 
> It seems anyway that some files are not compressed:
> 
> > $ find /usr/share/doc -type f -regextype posix-extended ! -regex
> > ".*\.(css|gif|htm[l]?|jp[e]?g|js|pdf|png|xz)" | wc -l 79
> 
> I won't list all of them here, just some examples:
> 
> > /usr/share/doc/gnome-shell-3.4.2/AUTHORS
> > /usr/share/doc/udisks-2.0.0/html/udisks2/index.sgml
> > /usr/share/doc/groff-1.21-r1/pic.ps
> > /usr/share/doc/automake-1.12.5/amhello-1.0.tar.gz
> 
> My goal is to have everything under /usr/share/doc compressed with xz,
> or at least to understand why something is being excluded from
> compression. Perhaps it is due to some additional rules I'm not aware
> of, or because of some bugs.
> 
> I checked the ebuilds of the packages the previous files belong to,
> but I found nothing interesting.
> 
> It would be great if anyone who is interested could check his/her own
> system too.

That stuff is controlled by the ebuild, IIRC ebuilds should call
function dodoc so they do with docs what you want them to do. The
function name may well have changed and been superceded since last I
looked.

The reason you find nothing interesting in the ebuild is because
something that should be there isn't. ebuilds not following rules wrt
doc files is a bug and should be filed at bgo as such.


-- 
Alan McKinnon
alan.mckinnon@gmail.com

Re: [gentoo-user] Some files in /usr/share/doc are not compressed

[Marc Joliet]

[-- Attachment #1: Type: text/plain, Size: 2035 bytes --]

Am Thu, 20 Dec 2012 00:11:38 +0200
schrieb Alan McKinnon <alan.mckinnon@gmail.com>:

> On Wed, 19 Dec 2012 17:41:30 +0100
> Francesco Turco <fturco@fastmail.fm> wrote:
> 
> > Hello.
> > 
> > On my system Portage uses the following two variables for compressing
> > files in /usr/share/doc:
> > 
> > > $ portageq envvar PORTAGE_COMPRESS
> > > xz
> > 
> > > $ portageq envvar PORTAGE_COMPRESS_EXCLUDE_SUFFIXES
> > > css gif htm[l]? jp[e]?g js pdf png
> > 
> > It seems anyway that some files are not compressed:
> > 
> > > $ find /usr/share/doc -type f -regextype posix-extended ! -regex
> > > ".*\.(css|gif|htm[l]?|jp[e]?g|js|pdf|png|xz)" | wc -l 79
> > 
> > I won't list all of them here, just some examples:
> > 
> > > /usr/share/doc/gnome-shell-3.4.2/AUTHORS
> > > /usr/share/doc/udisks-2.0.0/html/udisks2/index.sgml
> > > /usr/share/doc/groff-1.21-r1/pic.ps
> > > /usr/share/doc/automake-1.12.5/amhello-1.0.tar.gz
> > 
> > My goal is to have everything under /usr/share/doc compressed with xz,
> > or at least to understand why something is being excluded from
> > compression. Perhaps it is due to some additional rules I'm not aware
> > of, or because of some bugs.
> > 
> > I checked the ebuilds of the packages the previous files belong to,
> > but I found nothing interesting.
> > 
> > It would be great if anyone who is interested could check his/her own
> > system too.
> 
> That stuff is controlled by the ebuild, IIRC ebuilds should call
> function dodoc so they do with docs what you want them to do. The
> function name may well have changed and been superceded since last I
> looked.

Indeed: starting with EAPI 4 there is a DOCS variable for this. ebuild(5) says
it can be an array or a space delimited list, but for me (using EAPI 5) only the
array version worked. However, a default list of files is used when it isn't
specified.

[...]

-- 
Marc Joliet
--
"People who think they know everything really annoy those of us who know we
don't" - Bjarne Stroustrup

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: [gentoo-user] Some files in /usr/share/doc are not compressed

[Francesco Turco]

On Wed, Dec 19, 2012, at 23:11, Alan McKinnon wrote:
> That stuff is controlled by the ebuild, IIRC ebuilds should call
> function dodoc so they do with docs what you want them to do. The
> function name may well have changed and been superceded since last I
> looked.
> 
> The reason you find nothing interesting in the ebuild is because
> something that should be there isn't. ebuilds not following rules wrt
> doc files is a bug and should be filed at bgo as such.

I looked into this problem a little more before submitting any bug.

At http://devmanual.gentoo.org/ebuild-writing/eapi/ it says that, for
EAPI=4, there is a default compression exclusion list:
/usr/share/doc/${PF}/html. udisks-2.0.0 ebuild uses EAPI=4. So this
explains why /usr/share/doc/udisks-2.0.0/html/udisks2/index.sgml and
many other such files are not being compressed. I checked, and they all
use EAPI=4 or EAPI=5. So the list of uncompressed files drops to 23
instead of 79.

Then I looked for non-zero size files only, because it makes no sense to
compress empty files. The list dropped to 9 files, belonging to 4
different ebuilds or 3 packages:

> /usr/share/doc/automake-1.12.5/amhello-1.0.tar.gz
> /usr/share/doc/libasyncns-0.8-r2/README
> /usr/share/doc/automake-1.11.6/amhello-1.0.tar.gz
> /usr/share/doc/groff-1.21-r1/meref.ps
> /usr/share/doc/groff-1.21-r1/meref.me
> /usr/share/doc/groff-1.21-r1/pic.ps
> /usr/share/doc/groff-1.21-r1/pic.ms
> /usr/share/doc/groff-1.21-r1/meintro.me
> /usr/share/doc/groff-1.21-r1/meintro.ps

Perhaps I may report these ones.

Re: [gentoo-user] Some files in /usr/share/doc are not compressed

[Francesco Turco]

On Thu, Dec 20, 2012, at 10:57, Francesco Turco wrote:
> Then I looked for non-zero size files only, because it makes no sense to
> compress empty files. The list dropped to 9 files, belonging to 4
> different ebuilds or 3 packages:
> 
> > /usr/share/doc/automake-1.12.5/amhello-1.0.tar.gz
> > /usr/share/doc/libasyncns-0.8-r2/README
> > /usr/share/doc/automake-1.11.6/amhello-1.0.tar.gz
> > /usr/share/doc/groff-1.21-r1/meref.ps
> > /usr/share/doc/groff-1.21-r1/meref.me
> > /usr/share/doc/groff-1.21-r1/pic.ps
> > /usr/share/doc/groff-1.21-r1/pic.ms
> > /usr/share/doc/groff-1.21-r1/meintro.me
> > /usr/share/doc/groff-1.21-r1/meintro.ps
> 
> Perhaps I may report these ones.

It seems that amhello-1.0.tar.gz is already present in the tarball from
the distfiles directory, so it is not Portage responsible for
compressing in gz format. But it would be a good thing if Portage could
recompress that file in xz format instead. I don't know if this is
possible. So I won't file a bug report for this, as I think it could be
closed as invalid.

groff-1.21-r1 installs postscript files (which are not compressible) and
related files. So I don't think it's a good idea to file a report for
this, too.

So the only bug I decided to report is for libasyncns-0.8-r2:
https://bugs.gentoo.org/show_bug.cgi?id=447936.

Re: [gentoo-user] Some files in /usr/share/doc are not compressed

[julian]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 12/19/2012 05:41 PM, Francesco Turco wrote:
> Hello.
> 
> On my system Portage uses the following two variables for
> compressing files in /usr/share/doc:
> 
>> $ portageq envvar PORTAGE_COMPRESS xz
> 
>> $ portageq envvar PORTAGE_COMPRESS_EXCLUDE_SUFFIXES css gif
>> htm[l]? jp[e]?g js pdf png
> 
> It seems anyway that some files are not compressed:
> 
>> $ find /usr/share/doc -type f -regextype posix-extended ! -regex
>> ".*\.(css|gif|htm[l]?|jp[e]?g|js|pdf|png|xz)" | wc -l 79
> 
> I won't list all of them here, just some examples:
> 
>> /usr/share/doc/gnome-shell-3.4.2/AUTHORS 
>> /usr/share/doc/udisks-2.0.0/html/udisks2/index.sgml 
>> /usr/share/doc/groff-1.21-r1/pic.ps 
>> /usr/share/doc/automake-1.12.5/amhello-1.0.tar.gz
> 
> My goal is to have everything under /usr/share/doc compressed with
> xz, or at least to understand why something is being excluded from 
> compression. Perhaps it is due to some additional rules I'm not
> aware of, or because of some bugs.
> 
> I checked the ebuilds of the packages the previous files belong to,
> but I found nothing interesting.
> 
> It would be great if anyone who is interested could check his/her
> own system too.
> 
> Thank you.
> 

the ebuild developer can choose whether to not compress some files, look
for stuff like "docompress -x foo" in the ebuild

it is described in PMS:
https://dev.gentoo.org/~ulm/pms/head/pms.html#x1-12800011.3.3
(search for docompress)
and in the devmanual
http://devmanual.gentoo.org/ebuild-writing/eapi/index.html


Afaik pkgcore does ignore this command.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.19 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iQEcBAEBAgAGBQJQ0vi3AAoJEFpvPKfnPDWzEiYH/jnqnIRVo82F8tK4E1eTqvsN
u55oCbIEV6ySqBHHk3oHcX+K0PbuZkOUymauYVjoyf3fV33gBOLrKsKsY7U5wQvo
0ZpqY4C89UgpmuprLlflzu+Ehjnl0/lA9Qgfya94meWMBUetbddEu4ZwfJ7TG9PR
TafiVpbQ/pu0WuhfNdvgkFHYySwG4Pk+afDFTFnBGYFyV3ud/4aeU4Nt3aZTVpAN
xSBZdGFLM4WW/yKuXbbtg4UdSV4RBtX1UV6TRd4cjZxqOxPAdQXUXM3gQuVvUcnJ
sZovWSrThcmjmgP14ikjqsY6axU/GmxuqJFqX2GY56KDw84xtg7wjG230NGoG3U=
=mzU5
-----END PGP SIGNATURE-----