[gentoo-dev] RFC: deprecate /usr/share/doc/$PF

[gentoo-dev] RFC: deprecate /usr/share/doc/$PF

[Alexandre Rostovtsev]

At the moment, Gentoo documentation is supposed to be installed in
/usr/share/doc/$PF. Given the existence of slots, this directory
scheme makes little sense; versioning documentation directories with
$PF seems nearly as silly as would be e.g. appending $PVR to the
filenames of installed man pages. Moreover, /usr/share/doc/$PF results
in a number of problems:

* Since $PF changes on every revision bump and minor version change,
one cannot bookmark a documentation file for a frequently updated
package without using custom portage hooks or scripts that create
stable symlinks to docs.
* Since $PF changes on every revision bump and minor version change,
it's unnecessarily inconvenient to see which documentation files were
added or removed when upgrading a package.
* The package's documentation may be designed primarily for tools and
viewers which expect to load documentation files from a different
location.

I propose the following changes, and will write them up in GLEP form
if the feedback is positive.

1. If a package's documentation is designed to be accessed by a
specific documentation viewer tool, then the package should install
the documentation in a location where that tool will look for it (e.g.
devhelp expects to find GNOME API documentation in
/usr/share/gtk-doc/html, and khelpcenter expects to find KDE handbooks
in /usr/share/doc/HTML). This already happens in practice, but some
devs had expressed opposition to this (e.g. bug #312363) because it
had not been formalized as policy.

2. In EAPI-5 and higher, other documentation should be installed under
/usr/share/doc:
   a. if SLOT = "0": in /usr/share/doc/$CATEGORY/$PN by default, xor
(at the package maintainer's discretion) in
/usr/share/doc/$CATEGORY/$PN-0.
   b. if SLOT != "0": in /usr/share/doc/$CATEGORY/$PN-$SLOT.

3. In EAPI-5 and higher, dodoc/newdoc/dohtml will install documentation in:
   a. If SLOT = "0": /usr/share/doc/$CATEGORY/$PN/<docinto path>
(/usr/share/doc/$CATEGORY/$PN/html for dohtml);
   b. If SLOT != "0": /usr/share/doc/$CATEGORY/$PN-$SLOT/<docinto
path> (/usr/share/doc/$CATEGORY/$PN-$SLOT/html for dohtml).
   Corresponding changes will also have to be made to docompress, and
to the eclasses that currently use /usr/share/doc/${PF} as docdir.

4. In EAPI-0,1,2,3,4, ebuilds may, at the maintainer's discretion,
install documentation in the old /usr/share/doc/$PF location xor in
the new EAPI-5 location.

Answers to anticipated questions:

Q1: Why let docs designed to be viewed with special viewers be
installed outside /usr/share/doc?
A1: To match upstream expectations and to minimize maintenance burden.
This burden can be significant; for example, if documentation for
package A contains explicit links to documentation pages for package
B, then if the documentation pages are not installed where upstream
expects them, those links would have to be adjusted, probably via a
fragile script with untested corner cases.

Q2: Why /usr/share/doc/$CATEGORY/$PN-$SLOT instead of /usr/share/doc/$PN-$SLOT?
A2: To prevent file collisions between packages in different
categories but with identical package and slot names.

Q3: Why $PN-$SLOT instead of $PN:$SLOT?
A3: So that the directory names are compatible with bash's tab-completion.

Q4: Why install slot 0 docs in $CATEGORY/$PN by default instead of
$CATEGORY/$PN-0?
A4: Purely for aesthetics. A vast number of packages in the main tree
are only slot 0 and are unlikely to ever become slotted in the future,
so installing their docs in /usr/share/doc/$CATEGORY/$PN is logical,
pleasing to the eye, and results in shorter directory names.

Q5: Then why allow package maintainers to alternatively use $CATEGORY/$PN-0?
A5: Why not? It will not hurt anything, will not cause file
collisions, and some maintainers of a multislotted package, one of
which is 0, might prefer to install that slot's docs in
$CATEGORY/$PN-0 to prevent a potential impression that docs in
$CATEGORY/$PN apply to all of that package's slots.

Q6: Why can't the dodoc/dohtml path be changed before EAPI-5?
A6: Because the path where dodoc and dohtml install files is part of
the PMS. Portage can't just change it on its own. A possible
workaround for current EAPIs is adding new-style dodoc/dohtml
analogues to an eclass.

Q7: Why allow package maintainers to start using the new naming scheme
in earlier EAPIs?
A7: Because /usr/share/doc/$PF really is quite inconvenient for users
wanting to browse API documentation, and package maintainers may wish
to alleviate the users' pain before the EAPI-5 becomes finalized and
support for it is added to the eclasses that the ebuild needs.

-Alexandre.

Re: [gentoo-dev] RFC: deprecate /usr/share/doc/$PF

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 2747 bytes --]


I basically agree, it's quite a great idea. Just a few comments though.

On Sun, 18 Dec 2011 16:49:38 -0500
Alexandre Rostovtsev <tetromino@gentoo.org> wrote:

> * The package's documentation may be designed primarily for tools and
> viewers which expect to load documentation files from a different
> location.

That's why I, for instance, use gtk-doc in my libraries. It's just that
it has its standard install procedures and locations.

> 1. If a package's documentation is designed to be accessed by a
> specific documentation viewer tool, then the package should install
> the documentation in a location where that tool will look for it (e.g.
> devhelp expects to find GNOME API documentation in
> /usr/share/gtk-doc/html, and khelpcenter expects to find KDE handbooks
> in /usr/share/doc/HTML). This already happens in practice, but some
> devs had expressed opposition to this (e.g. bug #312363) because it
> had not been formalized as policy.

Agree. But that's outside of the GLEP/PMS scope; just an internal policy
should fine, I think.

> 2. In EAPI-5 and higher, other documentation should be installed under
> /usr/share/doc:
>    a. if SLOT = "0": in /usr/share/doc/$CATEGORY/$PN by default, xor
> (at the package maintainer's discretion) in
> /usr/share/doc/$CATEGORY/$PN-0.

I'd rather not see that -0 there.

>    b. if SLOT != "0": in /usr/share/doc/$CATEGORY/$PN-$SLOT.

[...]

> Q3: Why $PN-$SLOT instead of $PN:$SLOT?
> A3: So that the directory names are compatible with bash's
> tab-completion.

What if 'foo' has slot named 'bar', and there is unslotted 'foo-bar'
package? :P

> Q5: Then why allow package maintainers to alternatively use
> $CATEGORY/$PN-0? A5: Why not? It will not hurt anything, will not
> cause file collisions, and some maintainers of a multislotted
> package, one of which is 0, might prefer to install that slot's docs
> in $CATEGORY/$PN-0 to prevent a potential impression that docs in
> $CATEGORY/$PN apply to all of that package's slots.

This will make the policy less clear, and documentation locations more
enigmatic for users. While at this, I think we should somehow move
the docs for all EAPIs to avoid this, and probably move installed ones
as well.

> Q6: Why can't the dodoc/dohtml path be changed before EAPI-5?
> A6: Because the path where dodoc and dohtml install files is part of
> the PMS. Portage can't just change it on its own. A possible
> workaround for current EAPIs is adding new-style dodoc/dohtml
> analogues to an eclass.

I think some of devs agree we should be allowed to fix past mistakes
without waiting another 20 years till the tree is migrated to a new
EAPI...

-- 
Best regards,
Michał Górny

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 316 bytes --]

Re: [gentoo-dev] RFC: deprecate /usr/share/doc/$PF

[Pacho Ramos]

[-- Attachment #1: Type: text/plain, Size: 762 bytes --]

El dom, 18-12-2011 a las 23:02 +0100, Michał Górny escribió:
[...]
> > Q6: Why can't the dodoc/dohtml path be changed before EAPI-5?
> > A6: Because the path where dodoc and dohtml install files is part of
> > the PMS. Portage can't just change it on its own. A possible
> > workaround for current EAPIs is adding new-style dodoc/dohtml
> > analogues to an eclass.
> 
> I think some of devs agree we should be allowed to fix past mistakes
> without waiting another 20 years till the tree is migrated to a new
> EAPI...
> 

Maybe this situation could be improved if there was a policy forcing us
to try to use latest EAPI when possible for any package update, that way
we would move faster to latest eapi and even deprecate older eapis
easily

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: [gentoo-dev] RFC: deprecate /usr/share/doc/$PF

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 994 bytes --]

On Mon, 19 Dec 2011 00:07:45 +0100
Pacho Ramos <pacho@gentoo.org> wrote:

> El dom, 18-12-2011 a las 23:02 +0100, Michał Górny escribió:
> [...]
> > > Q6: Why can't the dodoc/dohtml path be changed before EAPI-5?
> > > A6: Because the path where dodoc and dohtml install files is part
> > > of the PMS. Portage can't just change it on its own. A possible
> > > workaround for current EAPIs is adding new-style dodoc/dohtml
> > > analogues to an eclass.
> > 
> > I think some of devs agree we should be allowed to fix past mistakes
> > without waiting another 20 years till the tree is migrated to a new
> > EAPI...
> > 
> 
> Maybe this situation could be improved if there was a policy forcing
> us to try to use latest EAPI when possible for any package update,
> that way we would move faster to latest eapi and even deprecate older
> eapis easily

Still unlikely. A bunch of old eclasses will force ebuilds to be EAPI 0
or so.

-- 
Best regards,
Michał Górny

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 316 bytes --]

Re: [gentoo-dev] RFC: deprecate /usr/share/doc/$PF

[Pacho Ramos]

[-- Attachment #1: Type: text/plain, Size: 1336 bytes --]

El lun, 19-12-2011 a las 09:31 +0100, Michał Górny escribió:
> On Mon, 19 Dec 2011 00:07:45 +0100
> Pacho Ramos <pacho@gentoo.org> wrote:
> 
> > El dom, 18-12-2011 a las 23:02 +0100, Michał Górny escribió:
> > [...]
> > > > Q6: Why can't the dodoc/dohtml path be changed before EAPI-5?
> > > > A6: Because the path where dodoc and dohtml install files is part
> > > > of the PMS. Portage can't just change it on its own. A possible
> > > > workaround for current EAPIs is adding new-style dodoc/dohtml
> > > > analogues to an eclass.
> > > 
> > > I think some of devs agree we should be allowed to fix past mistakes
> > > without waiting another 20 years till the tree is migrated to a new
> > > EAPI...
> > > 
> > 
> > Maybe this situation could be improved if there was a policy forcing
> > us to try to use latest EAPI when possible for any package update,
> > that way we would move faster to latest eapi and even deprecate older
> > eapis easily
> 
> Still unlikely. A bunch of old eclasses will force ebuilds to be EAPI 0
> or so.
> 

Well, I was meaning eapis different than 0, I know it will need to be
kept more time due backwards compatibility ;)

Regarding other eapis, how many eclasses are still requiring old eapis?
I think games.eclass was one of them, but haven't reviewed others :S

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: [gentoo-dev] RFC: deprecate /usr/share/doc/$PF

[Alexandre Rostovtsev]

On Sun, 2011-12-18 at 23:02 +0100, Michał Górny wrote:
> What if 'foo' has slot named 'bar', and there is unslotted 'foo-bar'
> package? :P

There are no such examples in the tree. The only ebuilds I could find
with non-numeric slots are various kernel sources, chromium,
google-chrome, beautifulsoup, python-dateutil, b43-firmware,
binutils-apple, gcc-apple, and rep-gtk. If cross-compiling there is also
insight, gdb, newlib, and uclibc, maybe a few more packages from
sys-devel/.

For completeness, I suppose that it wouldn't hurt to add a provision to
allow package maintainers to choose a different documentation directory
if that's required to prevent a file collision.

> > Q5: Then why allow package maintainers to alternatively use
> > $CATEGORY/$PN-0? A5: Why not? It will not hurt anything, will not
> > cause file collisions, and some maintainers of a multislotted
> > package, one of which is 0, might prefer to install that slot's docs
> > in $CATEGORY/$PN-0 to prevent a potential impression that docs in
> > $CATEGORY/$PN apply to all of that package's slots.
> 
> This will make the policy less clear, and documentation locations more
> enigmatic for users.

Fair point. OK, let's kill the "-0"; slot 0 goes in $CATEGORY/$PN only.

> While at this, I think we should somehow move
> the docs for all EAPIs to avoid this, and probably move installed ones
> as well.
> 
> [...]
> 
> I think some of devs agree we should be allowed to fix past mistakes
> without waiting another 20 years till the tree is migrated to a new
> EAPI...

Then we will also need an eclass-based solution for existing EAPIs to
replace dodoc/dohtml.

-Alexandre.

Re: [gentoo-dev] RFC: deprecate /usr/share/doc/$PF

[Jeroen Roovers]

On Sun, 18 Dec 2011 21:26:08 -0500
Alexandre Rostovtsev <tetromino@gentoo.org> wrote:

> On Sun, 2011-12-18 at 23:02 +0100, Michał Górny wrote:
> > What if 'foo' has slot named 'bar', and there is unslotted 'foo-bar'
> > package? :P
> 
> There are no such examples in the tree. The only ebuilds I could find
> with non-numeric slots are various kernel sources, chromium,
> google-chrome, beautifulsoup, python-dateutil, b43-firmware,
> binutils-apple, gcc-apple, and rep-gtk. If cross-compiling there is
> also insight, gdb, newlib, and uclibc, maybe a few more packages from
> sys-devel/.
> 
> For completeness, I suppose that it wouldn't hurt to add a provision
> to allow package maintainers to choose a different documentation
> directory if that's required to prevent a file collision.

For completeness, could you post a list of packages that would benefit
from your proposed changes? It's a little thing called scope. :)


     jer

Re: [gentoo-dev] RFC: deprecate /usr/share/doc/$PF

[Alexandre Rostovtsev]

On Mon, 2011-12-19 at 03:41 +0100, Jeroen Roovers wrote:
> For completeness, could you post a list of packages that would benefit
> from your proposed changes? It's a little thing called scope. :)

I cannot provide you the full list; for that I would have to rebuild the
full tree with USE=doc enabled, and I enable it only for a small number
of packages. But I can give you the packages that I have installed on my
system right now that install a nontrivial set of documentation files
in /usr/share/doc/$PF, files that are worth bookmarking and would
benefit from a location that doesn't change on every revision or version
bump:

app-accessibility/espeak
app-backup/dar
app-benchmarks/phoronix-test-suite
app-crypt/gnupg
app-pda/libimobiledevice
app-text/docbook-sgml-utils
app-text/ghostscript-gpl
app-text/htmltidy
app-text/hyperestraier
app-text/jpdftweak
app-text/openjade
app-text/xmlstarlet
dev-db/postgresql-docs
dev-db/qdbm
dev-db/sqlite
dev-haskell/binary
dev-haskell/dataenc
dev-haskell/deepseq
dev-haskell/haddock
dev-haskell/hashed-storage
dev-haskell/haskeline
dev-haskell/hscolour
dev-haskell/hsql
dev-haskell/hsql-mysql
dev-haskell/hsql-postgresql
dev-haskell/html
dev-haskell/http
dev-haskell/mmap
dev-haskell/mtl
dev-haskell/network
dev-haskell/parsec
dev-haskell/regex-base
dev-haskell/regex-compat
dev-haskell/regex-posix
dev-haskell/tar
dev-haskell/terminfo
dev-haskell/text
dev-haskell/utf8-string
dev-haskell/zlib
dev-java/ant-core
dev-java/icedtea
dev-java/java-sdk-docs
dev-java/sun-javamail
dev-lang/ghc
dev-lang/icon
dev-lang/lua
dev-lang/perl
dev-lang/R
dev-libs/boehm-gc
dev-libs/cyrus-sasl
dev-libs/expat
dev-libs/iniparser
dev-libs/libgamin
dev-libs/libofx
dev-libs/libpcre
dev-libs/libxslt
dev-libs/redland
dev-libs/seed
dev-python/django
dev-python/python-docs
dev-python/sqlalchemy
dev-tcltk/blt
dev-util/astyle
dev-util/valgrind
dev-vcs/git
dev-vcs/subversion
games-emulation/zsnes
games-strategy/freeciv
games-strategy/liquidwar
media-gfx/geeqie
media-gfx/gqview
media-gfx/povray
media-libs/allegro
media-libs/exiftool
media-libs/flac
media-libs/freeglut
media-libs/gd
media-libs/ladspa-sdk
media-libs/libao
media-libs/libkate
media-libs/libsamplerate
media-libs/libsdl
media-libs/libsndfile
media-libs/libtheora
media-libs/raptor
media-libs/tiff
media-sound/esound
media-sound/lame
media-sound/twolame
media-video/ogmrip
net-misc/ntp
net-proxy/polipo
sci-mathematics/maxima
sys-apps/dbus
sys-apps/dstat
sys-apps/groff
sys-apps/kbd
sys-apps/portage
sys-devel/clang
sys-devel/llvm
sys-libs/db
sys-libs/pam
sys-libs/slang
sys-process/fcron
x11-drivers/nvidia-drivers
x11-libs/qt-assistant
x11-terms/rxvt
xfce-base/exo

Re: [gentoo-dev] RFC: deprecate /usr/share/doc/$PF

[Jeroen Roovers]

On Mon, 19 Dec 2011 00:31:35 -0500
Alexandre Rostovtsev <tetromino@gentoo.org> wrote:

> On Mon, 2011-12-19 at 03:41 +0100, Jeroen Roovers wrote:
> > For completeness, could you post a list of packages that would
> > benefit from your proposed changes? It's a little thing called
> > scope. :)
> 
> I cannot provide you the full list; for that I would have to rebuild
> the full tree with USE=doc enabled, and I enable it only for a small
> number of packages. But I can give you the packages that I have
> installed on my system right now that install a nontrivial set of
> documentation files in /usr/share/doc/$PF, files that are worth
> bookmarking and would benefit from a location that doesn't change on
> every revision or version bump:

I understand you use some kind of graphical file manager that opens
files for you into some kind of "document viewer". I wouldn't think
anyone tab-completing this on a command line would ever have the need to
"bookmark" stuff otherwise.

> [...]

That's rather a long list. I expected a much shorter list of packages
that use /usr/share/doc/$PF to store data that is read at run time
(which they probably shouldn't do in the first place) so they would
benefit from this. I was not asking for a list of packages that install
documentation in /usr/share/doc/$PF. If that was the only criterium that
mattered, then I wouldn't have asked for a list. :)


     jer

Re: [gentoo-dev] RFC: deprecate /usr/share/doc/$PF

[Chí-Thanh Christopher Nguyễn]

Alexandre Rostovtsev schrieb:
> Answers to anticipated questions:
Q8: SLOT can change after the package was installed. How to handle this
case?


Best regards,
Chí-Thanh Christopher Nguyễn

Re: [gentoo-dev] RFC: deprecate /usr/share/doc/$PF

[Alexandre Rostovtsev]

On Sun, 2011-12-18 at 23:07 +0100, Chí-Thanh Christopher Nguyễn wrote:
> Alexandre Rostovtsev schrieb:
> > Answers to anticipated questions:
> Q8: SLOT can change after the package was installed. How to handle this
> case?

I think the slotmove should happen without renaming the documentation
directory; silently moving installed files after an emerge --sync
without the user's explicit request seems a bad idea.

This means that a slotmove could result in a future file collision. If
foo-1.0 and foo-2.0 were in slot "1", and >=foo-2.0 was slotmoved from
"1" to "2", then if a user had foo-2.0 installed before the slotmove,
attempting to install foo-1.0 after the slotmove would result in a file
collision in /usr/share/doc/*/foo-1. Such collisions would have to be
resolved with the usual technique of revbumps and blockers.

However, in practice slotmoves that could cause such collisions are
infrequent. Among the 94 slotmoves for 2009 and 2010, I see only five
that would have resulted in potential file collisions in /usr/share/doc
under the proposed scheme: ati-drivers, clojure, vala, libchamplain, and
clutter-gtk. Among the several hundred slotmoves for 2011 so far, I see
*zero* that would have resulted in a collision.

It is possible that I missed some, but even then, I think that less than
10% of slotmoves - probably much less - would have been affected.

-Alexandre.

Re: [gentoo-dev] RFC: deprecate /usr/share/doc/$PF

[Ulrich Mueller]

[Why are there different Reply-To: headers in -dev and in -pms MLs?
Following up to both lists.]

>>>>> On Sun, 18 Dec 2011, Alexandre Rostovtsev wrote:

> At the moment, Gentoo documentation is supposed to be installed in
> /usr/share/doc/$PF. [...]

> I propose the following changes, and will write them up in GLEP form
> if the feedback is positive.

> 1. If a package's documentation is designed to be accessed by a
> specific documentation viewer tool, then the package should install
> the documentation in a location where that tool will look for it
> (e.g. devhelp expects to find GNOME API documentation in
> /usr/share/gtk-doc/html, and khelpcenter expects to find KDE
> handbooks in /usr/share/doc/HTML). This already happens in practice,
> but some devs had expressed opposition to this (e.g. bug #312363)
> because it had not been formalized as policy.

How do you handle FEATURES="nodoc" if you spread the documentation all
over the filesystem? Should Portage learn about all the special cases?
IMHO it would make more sense to leave the documentation under
/usr/share/doc and either configure the documentation viewer to find
it there, or (if that's not possible) create symlinks.

> 2. In EAPI-5 and higher, other documentation should be installed
> under /usr/share/doc:
>    a. if SLOT = "0": in /usr/share/doc/$CATEGORY/$PN by default, xor
> (at the package maintainer's discretion) in
> /usr/share/doc/$CATEGORY/$PN-0.
>    b. if SLOT != "0": in /usr/share/doc/$CATEGORY/$PN-$SLOT.

Can we please avoid the bloat of another directory level here?
${CATEGORY}/${PN} will be even longer than ${PF} in most cases.

Or, to say it in your own words: "${CATEGORY}/${PN} seems nearly as
silly as would be e.g. prepending ${CATEGORY} to the filenames of
installed man pages." ;-)

> [...]

> 4. In EAPI-0,1,2,3,4, ebuilds may, at the maintainer's discretion,
> install documentation in the old /usr/share/doc/$PF location xor in
> the new EAPI-5 location.

If we change our policy, then we should move everything to the new
location (with a transition period of course). It would be very
inconvenient for users if they had to search two different places
for documentation.

Ulrich

Re: [gentoo-dev] RFC: deprecate /usr/share/doc/$PF

[Alexandre Rostovtsev]

On Mon, 2011-12-19 at 01:08 +0100, Ulrich Mueller wrote:
> [Why are there different Reply-To: headers in -dev and in -pms MLs?
> Following up to both lists.]

I apologize for the mess; I had intended to bring the question up before
a wider audience, but failed to think through the consequences of two
mailing lists ending up in the reply-to.

For the sake of keeping discussion in one thread, I ask that further
replies should be made to gentoo-dev, not gentoo-pms.

> How do you handle FEATURES="nodoc" if you spread the documentation all
> over the filesystem? Should Portage learn about all the special cases?
> IMHO it would make more sense to leave the documentation under
> /usr/share/doc and either configure the documentation viewer to find
> it there, or (if that's not possible) create symlinks.

It's not "all over the filesystem"; in practice, the number of locations
I believe is fairly small (/usr/share/gtk-doc and /usr/lib/monodoc for
API documentation, and /usr/share/help, /usr/share/omf,
and /usr/share/doc/HTML for end-user help files are the only ones that I
know of), and adding them to portage's nodoc list seems much easier than
editing hundreds of ebuilds that already install docs there.

Documentation in Gentoo-specific /usr/share/doc subdirectories would not
be able to link to documentation pages in other packages without
fragile, hard-to-maintain scripts - and even with the best scripts,
things would break on package renames. Symlinks could work, but (if the
nodoc situation is resolved) would give package maintainers extra work
for no real benefit.

> Can we please avoid the bloat of another directory level here?
> ${CATEGORY}/${PN} will be even longer than ${PF} in most cases.

The problem is that ($PN, $CATEGORY) pairs are not unique. Think of
x11-terms/terminal:0 and gnustep-apps/terminal:0, or app-misc/beagle:0
and sci-libs/beagle:0, or app-misc/nut:0 and sys-power/nut:0. I could
not think of any better solution than using $CATEGORY/$PN-$SLOT. Do you
have a better proposal that does not rely on $PVR?

> If we change our policy, then we should move everything to the new
> location (with a transition period of course). It would be very
> inconvenient for users if they had to search two different places
> for documentation.

If everything is to be moved to the new locations, we will need an
eclass-based solution to replace dodoc/dohtml for existing EAPIs.

-Alexandre.

Re: [gentoo-dev] RFC: deprecate /usr/share/doc/$PF

[Ulrich Mueller]

>>>>> On Sun, 18 Dec 2011, Alexandre Rostovtsev wrote:

>> Can we please avoid the bloat of another directory level here?
>> ${CATEGORY}/${PN} will be even longer than ${PF} in most cases.

> The problem is that ($PN, $CATEGORY) pairs are not unique. Think of
> x11-terms/terminal:0 and gnustep-apps/terminal:0, or
> app-misc/beagle:0 and sci-libs/beagle:0, or app-misc/nut:0 and
> sys-power/nut:0. I could not think of any better solution than using
> $CATEGORY/$PN-$SLOT.

Thinking about it a little more, I believe that ${CATEGORY} shouldn't
appear anywhere in the path of installed files, for the following
reasons:

1. Users may not know the category of a package, therefore it's not
   obvious for them where to find its documentation. (Think of it from
   the perspective of a user on a multiuser system, who didn't install
   the packages on that system.) OTOH, the name of the package (PN) is
   obvious in most cases, since it will coincide with the upstream
   name.

2. It doesn't play well with bash completion. When searching for
   documentation of a specific package (and only knowing PN), one can
   currently type the pathname up to PN and press tab which will
   complete PVR. With CATEGORY _before_ PN this would no longer work.

3. CATEGORY and SLOT are Gentoo specific, related to the way how we
   organise our packages. Neither of them should appear in the
   directory structure of installed packages. The problems related to
   package and slot moves (where CATEGORY or SLOT change) also show
   that something is wrong with the approach. (BTW, in the current
   system, PR is also Gentoo specific. It doesn't suffer from problems
   with package moves though.)

> Do you have a better proposal that does not rely on $PVR?

Leave things as they are. It's not perfect, but IMHO your approach
would create at least as many problems as it would solve.
Alternatively, a minimal solution would be to drop only ${PR}, i.e.
install documentation under /usr/share/doc/${P}.

Ulrich

Re: [gentoo-dev] RFC: deprecate /usr/share/doc/$PF

[Dale]

Ulrich Mueller wrote:
>>>>>> On Sun, 18 Dec 2011, Alexandre Rostovtsev wrote:
>>> Can we please avoid the bloat of another directory level here?
>>> ${CATEGORY}/${PN} will be even longer than ${PF} in most cases.
>> The problem is that ($PN, $CATEGORY) pairs are not unique. Think of
>> x11-terms/terminal:0 and gnustep-apps/terminal:0, or
>> app-misc/beagle:0 and sci-libs/beagle:0, or app-misc/nut:0 and
>> sys-power/nut:0. I could not think of any better solution than using
>> $CATEGORY/$PN-$SLOT.
> Thinking about it a little more, I believe that ${CATEGORY} shouldn't
> appear anywhere in the path of installed files, for the following
> reasons:
>
> 1. Users may not know the category of a package, therefore it's not
>     obvious for them where to find its documentation. (Think of it from
>     the perspective of a user on a multiuser system, who didn't install
>     the packages on that system.) OTOH, the name of the package (PN) is
>     obvious in most cases, since it will coincide with the upstream
>     name.
>
> 2. It doesn't play well with bash completion. When searching for
>     documentation of a specific package (and only knowing PN), one can
>     currently type the pathname up to PN and press tab which will
>     complete PVR. With CATEGORY _before_ PN this would no longer work.
>
> 3. CATEGORY and SLOT are Gentoo specific, related to the way how we
>     organise our packages. Neither of them should appear in the
>     directory structure of installed packages. The problems related to
>     package and slot moves (where CATEGORY or SLOT change) also show
>     that something is wrong with the approach. (BTW, in the current
>     system, PR is also Gentoo specific. It doesn't suffer from problems
>     with package moves though.)
>
>> Do you have a better proposal that does not rely on $PVR?
> Leave things as they are. It's not perfect, but IMHO your approach
> would create at least as many problems as it would solve.
> Alternatively, a minimal solution would be to drop only ${PR}, i.e.
> install documentation under /usr/share/doc/${P}.
>
> Ulrich
>
>

I like the logic in #1 as a user.  What if two packages have the same 
name but different categories tho?  It is rare but I do run into this 
from time to time.  I try to emerge something but am informed by emerge 
that I have to include the category for emerge to know exactly which 
package I want installed.

I do agree that docs are sometimes hard to find.  The only way I have 
any success is equery files <package name> then see where it put them or 
if there is none to find.

Back to my hole.

Dale

:-)  :-)

-- 
I am only responsible for what I said ... Not for what you understood or how you interpreted my words!

Miss the compile output?  Hint:
EMERGE_DEFAULT_OPTS="--quiet-build=n"

[gentoo-dev] Re: RFC: deprecate /usr/share/doc/$PF

[Duncan]

Dale posted on Sun, 18 Dec 2011 22:48:27 -0600 as excerpted:

> Ulrich Mueller wrote:
>> On Sun, 18 Dec 2011, Alexandre Rostovtsev wrote:
>>> The problem is that ($PN, $CATEGORY) pairs are not unique.

>> Thinking about it a little more, I believe that ${CATEGORY} shouldn't
>> appear anywhere in the path of installed files, for the following
>> reasons:
>>
>> 1. Users may not know the category of a package

> I like the logic in #1 as a user.  What if two packages have the same
> name but different categories tho?  It is rare but I do run into this
> from time to time.  I try to emerge something but am informed by emerge
> that I have to include the category for emerge to know exactly which
> package I want installed.

At present that's semi-accounted-for by including version-revision.  
Luckily, $PN clashes don't often have the same versions installed at the 
same time, so including it helps avoid clashes on $PN only.  The proposal 
removes version-revision, which would ordinarily be good, but in the 
context of $PN clashes causes problems.

I believe that's the rock upon which most previous similar proposals have 
foundered.

> I do agree that docs are sometimes hard to find.  The only way I have
> any success is equery files <package name> then see where it put them or
> if there is none to find.

For docs, I used to have problems too, but it's been awhile, I think 
because I've learned to work with the system, in this case, tab-
completion (the next point), not against it.  I do sometimes use equery 
files or equery belongs to find where a package put files, but seldom for 
docs in the docdir, since I know where to look and can tab-complete for 
them.  But for what manpages a package includes, certainly, and over the 
years I've found a number of commands I didn't know about, that way. =:^)

>> 2. It doesn't play well with bash completion. When searching for
>>     documentation of a specific package (and only knowing PN), one can
>>     currently type the pathname up to PN and press tab which will
>>     complete PVR. With CATEGORY _before_ PN this would no longer work.

I often find myself doing equery list, simply to find the category, as 
when reporting bugs or checking to see which binpkg versions I still have 
around, etc.  But the various package trees (ebuild, binpkg, installed-
db) are on filesystems that aren't normally mounted unless I'm working on 
them, on mds that likewise aren't normally assembled, and having to 
assemble an md and mount a filesystem just to be able to do an equery 
list, so I can lookup a doc of a package that's under some category name 
I don't remember, just because someone broke the previously working bash-
completion lookup method...

Very much ungood!

>> 3. CATEGORY and SLOT are Gentoo specific [and n]either of them should
>>     appear in the directory structure of installed packages.

>> Leave things as they are. It's not perfect, but IMHO your approach
>> would create at least as many problems as it would solve.

Indeed.

If a method allowing permanent bookmarking is desired, scripts that 
create and update a tree of symlinks and/or uses existing portage phase 
hooks has already been mentioned.  Someone can package such a script and 
as with any other package, people that want/need it can emerge and run it 
as necessary, without getting into the quagmire of confusing category 
names, broken tab-completion and potential doc-file collisions that this 
proposal is opening up.

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman

Re: [gentoo-dev] RFC: deprecate /usr/share/doc/$PF

[Alexandre Rostovtsev]

On Mon, 2011-12-19 at 05:23 +0100, Ulrich Mueller wrote:
> Thinking about it a little more, I believe that ${CATEGORY} shouldn't
> appear anywhere in the path of installed files, for the following
> reasons:
> 
> 1. Users may not know the category of a package, therefore it's not
>    obvious for them where to find its documentation. (Think of it from
>    the perspective of a user on a multiuser system, who didn't install
>    the packages on that system.) OTOH, the name of the package (PN) is
>    obvious in most cases, since it will coincide with the upstream
>    name.
> 
> 2. It doesn't play well with bash completion. When searching for
>    documentation of a specific package (and only knowing PN), one can
>    currently type the pathname up to PN and press tab which will
>    complete PVR. With CATEGORY _before_ PN this would no longer work.

I am forced to agree with your points #1 and #2. $CATEGORY/$PN-$SLOT is
optimized for the "bookmark a document and go back to it a week later"
use case, but you are correct that it would work poorly for the "quickly
look up a doc that you had not cared about until now" use case.

Using /usr/share/doc/$PN-$SLOT with exceptions for packages that have
the same ($PN, $SLOT) but different categories would not scale: it turns
out there are >100 of them in the main tree.

Using /usr/share/doc/$P would be worse than the current situation: doc
locations would still shift on minor version bumps, and packages with
multiple slots in the same $P (e.g. webkit-gtk) would collide.

And using $CATEGORY after $PN is simply too ugly to contemplate.

Fortunately, there *is* a neat solution.

Symlinks.

In other words: no change to dodoc/newdoc/dohtml, they will continue to
install documentation in /usr/share/doc/$PF like they do today.

BUT the package manager will automatically create a symlink
from /usr/share/doc/$CATEGORY/$PN (for slot 0) or
from /usr/share/doc/$CATEGORY/$PN-$SLOT (for slot != 0)
to /usr/share/doc/$PF. This would have to be done after the end of
src_install() so that the symlink goes in the VDB.

I think that this change, similarly to the automatic .la file fixing,
could be implemented by portage without waiting for a new EAPI.

> 3. CATEGORY and SLOT are Gentoo specific, related to the way how we
>    organise our packages. Neither of them should appear in the
>    directory structure of installed packages.

Slot 0 really is Gentoo-specific; that is why I propose to omit it.
Non-zero slots, however, are usually based on some obvious upstream
property, for example on the major version of the package or on the name
of the pkgconfig file.

You are correct that categories are Gentoo-specific and are therefore
not ideal for installed paths. However, for generating a stable path to
documentation files, one that does not shift on version bumps and
revbumps, there doesn't seem to be any alternative.

-Alexandre.

Re: [gentoo-dev] RFC: deprecate /usr/share/doc/$PF

[Ulrich Mueller]

>>>>> On Mon, 19 Dec 2011, Alexandre Rostovtsev wrote:

> I am forced to agree with your points #1 and #2.

Good. :-)

> $CATEGORY/$PN-$SLOT is optimized for the "bookmark a document and go
> back to it a week later" use case, but you are correct that it would
> work poorly for the "quickly look up a doc that you had not cared
> about until now" use case.

> Using /usr/share/doc/$PN-$SLOT with exceptions for packages that
> have the same ($PN, $SLOT) but different categories would not scale:
> it turns out there are >100 of them in the main tree.

Probably most of them in app-emacs and app-xemacs.

> Using /usr/share/doc/$P would be worse than the current situation:
> doc locations would still shift on minor version bumps, and packages
> with multiple slots in the same $P (e.g. webkit-gtk) would collide.

> And using $CATEGORY after $PN is simply too ugly to contemplate.

I agree.

> Fortunately, there *is* a neat solution.

> Symlinks.

> In other words: no change to dodoc/newdoc/dohtml, they will continue
> to install documentation in /usr/share/doc/$PF like they do today.

> BUT the package manager will automatically create a symlink
> from /usr/share/doc/$CATEGORY/$PN (for slot 0) or
> from /usr/share/doc/$CATEGORY/$PN-$SLOT (for slot != 0)
> to /usr/share/doc/$PF. This would have to be done after the end of
> src_install() so that the symlink goes in the VDB.

Maybe the symlinks should live in a different directory? Otherwise it
could be confusing for packages like gnustep-base or net-dns whose
name is equal to a category name.

Also I think that the slot should better be separated by a character
that cannot occur in a package name, i.e. something other than
hyphen (-), plus sign (+), or underscore (_). What was the problem
with the colon (:) again?

> I think that this change, similarly to the automatic .la file
> fixing, could be implemented by portage without waiting for a new
> EAPI.

Not sure about this one. It's not Portage only, but affects all
package managers.

Ulrich

[gentoo-dev] Re: RFC: deprecate /usr/share/doc/$PF

[Duncan]

Ulrich Mueller posted on Mon, 19 Dec 2011 09:23:59 +0100 as excerpted:

>> I think that this change, similarly to the automatic .la file fixing,
>> could be implemented by portage without waiting for a new EAPI.
> 
> Not sure about this one. It's not Portage only, but affects all package
> managers.

If it's a portage thing, please have it be a FEATURE that can be toggled 
off.  Symlink trees are ugly, as someone else mentioned, and not everyone 
will want them.

Alternatively, as I suggested in a different subthread, make it a 
separate package.  Then people can install it if they want/need.  This 
package could even include a script that hooks into the existing portage 
phase-hook structure, and presumably similar scripts for the other PMs, 
thus making the whole thing automatic.

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman

Re: [gentoo-dev] RFC: deprecate /usr/share/doc/$PF

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 2091 bytes --]

On Mon, 19 Dec 2011 01:41:00 -0500
Alexandre Rostovtsev <tetromino@gentoo.org> wrote:

> Using /usr/share/doc/$PN-$SLOT with exceptions for packages that have
> the same ($PN, $SLOT) but different categories would not scale: it
> turns out there are >100 of them in the main tree.

And I hope there will be more. We should seriously start splitting
packages rather than using old Gentoo bloat like IUSE='perl python
foobar foobaz'.

> Fortunately, there *is* a neat solution.
> 
> Symlinks.

Symlinks are never neat. In this particular case, they just mean
someone has failed horribly and now is hoping to fix it taking
the path of least resistance.

> > 3. CATEGORY and SLOT are Gentoo specific, related to the way how we
> >    organise our packages. Neither of them should appear in the
> >    directory structure of installed packages.
> 
> You are correct that categories are Gentoo-specific and are therefore
> not ideal for installed paths. However, for generating a stable path
> to documentation files, one that does not shift on version bumps and
> revbumps, there doesn't seem to be any alternative.

To be honest, the whole ${PF} is Gentoo-specific. Package names not
necessarily follow upstream ones; sometimes we need to change versions
as well to match ${PV} semantics or logic. Perl modules are quite
a large case here.

Sometimes packages in different categories collide. Right now, devs
have to be aware not to install colliding docs -- usually through
renaming files. Using category will at least partially fix this.

Shifting is unavoidable. SLOTs can change, categories can change,
package names can change. Of course, all the mentioned cases are much
rarer than PF changing but -- as pointed out before -- these could
change without reinstalling packages.

If we decided to use such names, the most correct approach would be to
have PM handle docmoves as well. But -- on the other hand -- there will
be always some hardwired paths which will be updated only on real
package rebuild...

-- 
Best regards,
Michał Górny

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 316 bytes --]

Re: [gentoo-dev] RFC: deprecate /usr/share/doc/$PF

[Ciaran McCreesh]

[-- Attachment #1: Type: text/plain, Size: 456 bytes --]

On Mon, 19 Dec 2011 10:02:07 +0100
Michał Górny <mgorny@gentoo.org> wrote:
> Shifting is unavoidable. SLOTs can change, categories can change,
> package names can change.

We should really just abolish updates and handle changes by reinstalls
plus blockers. Updates are a huge pain, they cause all kinds of problems
and anyone who objects to recompiling a few packages a few times a year
is using the wrong distribution.

-- 
Ciaran McCreesh

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: [gentoo-dev] RFC: deprecate /usr/share/doc/$PF

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 537 bytes --]

On Mon, 19 Dec 2011 05:23:08 +0100
Ulrich Mueller <ulm@gentoo.org> wrote:

> 2. It doesn't play well with bash completion. When searching for
>    documentation of a specific package (and only knowing PN), one can
>    currently type the pathname up to PN and press tab which will
>    complete PVR. With CATEGORY _before_ PN this would no longer work.

You could type /usr/share/doc/*/foo and use tab-completion then. Bash
will suggest all categories or even fill it if only one matches.

-- 
Best regards,
Michał Górny

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 316 bytes --]

Re: [gentoo-dev] RFC: deprecate /usr/share/doc/$PF

[Stelian Ionescu]

[-- Attachment #1: Type: text/plain, Size: 1791 bytes --]

On Mon, 2011-12-19 at 05:23 +0100, Ulrich Mueller wrote:
> >>>>> On Sun, 18 Dec 2011, Alexandre Rostovtsev wrote:
> 
> >> Can we please avoid the bloat of another directory level here?
> >> ${CATEGORY}/${PN} will be even longer than ${PF} in most cases.
> 
> > The problem is that ($PN, $CATEGORY) pairs are not unique. Think of
> > x11-terms/terminal:0 and gnustep-apps/terminal:0, or
> > app-misc/beagle:0 and sci-libs/beagle:0, or app-misc/nut:0 and
> > sys-power/nut:0. I could not think of any better solution than using
> > $CATEGORY/$PN-$SLOT.
> 
> Thinking about it a little more, I believe that ${CATEGORY} shouldn't
> appear anywhere in the path of installed files, for the following
> reasons:
> 
> 1. Users may not know the category of a package, therefore it's not
>    obvious for them where to find its documentation. (Think of it from
>    the perspective of a user on a multiuser system, who didn't install
>    the packages on that system.) OTOH, the name of the package (PN) is
>    obvious in most cases, since it will coincide with the upstream
>    name.

Every other distro includes the category in the package name, for
example Debian puts mod_fastcgi documentation
in /usr/share/doc/libapache2-mod-fastcgi and in practice it's really not
that big of a problem

> 2. It doesn't play well with bash completion. When searching for
>    documentation of a specific package (and only knowing PN), one can
>    currently type the pathname up to PN and press tab which will
>    complete PVR. With CATEGORY _before_ PN this would no longer work.

It's just an ls -d /usr/share/doc/*/$PN, not worth worrying about


-- 
Stelian Ionescu a.k.a. fe[nl]ix
Quidquid latine dictum sit, altum videtur.
http://common-lisp.net/project/iolib


[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: [gentoo-dev] RFC: deprecate /usr/share/doc/$PF

[Maciej Mrozowski]

[-- Attachment #1: Type: Text/Plain, Size: 2110 bytes --]

On Monday 19 of December 2011 02:52:54 Alexandre Rostovtsev wrote:
> On Mon, 2011-12-19 at 01:08 +0100, Ulrich Mueller wrote:
> > [Why are there different Reply-To: headers in -dev and in -pms MLs?
> > Following up to both lists.]
> 
> I apologize for the mess; I had intended to bring the question up before
> a wider audience, but failed to think through the consequences of two
> mailing lists ending up in the reply-to.
> 
> For the sake of keeping discussion in one thread, I ask that further
> replies should be made to gentoo-dev, not gentoo-pms.
> 
> > How do you handle FEATURES="nodoc" if you spread the documentation all
> > over the filesystem? Should Portage learn about all the special cases?
> > IMHO it would make more sense to leave the documentation under
> > /usr/share/doc and either configure the documentation viewer to find
> > it there, or (if that's not possible) create symlinks.
> 
> It's not "all over the filesystem"; in practice, the number of locations
> I believe is fairly small (/usr/share/gtk-doc and /usr/lib/monodoc for
> API documentation, and /usr/share/help, /usr/share/omf,
> and /usr/share/doc/HTML for end-user help files are the only ones that I
> know of), and adding them to portage's nodoc list seems much easier than
> editing hundreds of ebuilds that already install docs there.
> 
> Documentation in Gentoo-specific /usr/share/doc subdirectories would not
> be able to link to documentation pages in other packages without
> fragile, hard-to-maintain scripts - and even with the best scripts,
> things would break on package renames. Symlinks could work, but (if the
> nodoc situation is resolved) would give package maintainers extra work
> for no real benefit.
> 
> > Can we please avoid the bloat of another directory level here?
> > ${CATEGORY}/${PN} will be even longer than ${PF} in most cases.
> 
> The problem is that ($PN, $CATEGORY) pairs are not unique.

I still think we should even make PN an unique identifier in order to be able 
to purge categories... that's different story though...

-- 
regards
MM

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: [gentoo-dev] RFC: deprecate /usr/share/doc/$PF

[Mike Frysinger]

[-- Attachment #1: Type: Text/Plain, Size: 296 bytes --]

On Tuesday 20 December 2011 20:44:03 Maciej Mrozowski wrote:
> I still think we should even make PN an unique identifier in order to be
> able to purge categories... that's different story though...

a world without categories is a *lot* worse than a world with $PN free 
collisions
-mike

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: [gentoo-dev] RFC: deprecate /usr/share/doc/$PF

[Maciej Mrozowski]

[-- Attachment #1: Type: Text/Plain, Size: 675 bytes --]

On Wednesday 21 of December 2011 04:40:09 Mike Frysinger wrote:
> On Tuesday 20 December 2011 20:44:03 Maciej Mrozowski wrote:
> > I still think we should even make PN an unique identifier in order to be
> > able to purge categories... that's different story though...
> 
> a world without categories is a *lot* worse than a world with $PN free
> collisions

Yeah.. because?
Users of other distros seem to disagree with you however. (no other package 
manager I know relies on fixed package categories and uses tags to describe 
package purpose and PN-PV to identify it).
But like I said, that's different story, let's not hijack this topic.

-- 
regards
MM

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: [gentoo-dev] RFC: deprecate /usr/share/doc/$PF

[Mike Frysinger]

[-- Attachment #1: Type: Text/Plain, Size: 1219 bytes --]

On Tuesday 27 December 2011 12:29:09 Maciej Mrozowski wrote:
> On Wednesday 21 of December 2011 04:40:09 Mike Frysinger wrote:
> > On Tuesday 20 December 2011 20:44:03 Maciej Mrozowski wrote:
> > > I still think we should even make PN an unique identifier in order to
> > > be able to purge categories... that's different story though...
> > 
> > a world without categories is a *lot* worse than a world with $PN free
> > collisions
> 
> Yeah.. because?
> Users of other distros seem to disagree with you however. (no other package
> manager I know relies on fixed package categories and uses tags to describe
> package purpose and PN-PV to identify it).

the fact that other distros are working based on their restrictions does not 
mean they enjoy it.  flat listing package names hits collisions in the $PN 
which $CATEGORY has for the most part made a non-issue whereas other distros 
arbitrarily mangle the name.  it also makes organization a lot cleaner.  i've 
used other distros and their flat lists and makes finding stuff harder (obviously 
imo).

> But like I said, that's different story, let's not hijack this topic.

you can't open the door and shut it in the same breath :P
-mike

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: [gentoo-dev] RFC: deprecate /usr/share/doc/$PF

[Andreas K. Huettel]

[-- Attachment #1: Type: Text/Plain, Size: 350 bytes --]


OK, now that we have bikeshedded this to death, let me state that I 
fullheartedly support the original proposal by Alexandre, as unmodified as 
possible. We should IMHO implement it as soon as possible.

Andreas

-- 
Andreas K. Huettel
Gentoo Linux developer 
kde, sci, tex, arm, printing
dilfridge@gentoo.org
http://www.akhuettel.de/

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 836 bytes --]