[gentoo-dev] glep 42 (news) round six

[gentoo-dev] glep 42 (news) round six

[Ciaran McCreesh]

[-- Attachment #1.1: Type: text/plain, Size: 1017 bytes --]

Here we go again... Changes:

* We're now supporting overlays, multiple repositories and magic flying
chickens. To do this, we're shoving a whole load of new requirements
onto Portage. Said requirements are documented under `Required Portage
Enhancements`_. I now expect to get heavily flamed by a different set
of Portage developers.

* Allow underscores in names too.

* Change to yyyy-mm-dd for GLEP 45 compatibility. Rereremove the
timestamp on the Posted: header.

* Tweak Display-If-{Profile,Installed} to work with multiple
repositories.

* Use ${repoid} rather than magic-chicken for news-*.* files.

* Be specific on how news-repoid.skip can work.

You are encouraged to reply to this thread saying "I agree with ciaranm
that repository IDs should not be allowed to contain spaces".

We're getting there... Slowly...

-- 
Ciaran McCreesh : Gentoo Developer (I can kill you with my brain)
Mail            : ciaranm at gentoo.org
Web             : http://dev.gentoo.org/~ciaranm


[-- Attachment #1.2: glep-0042.txt --]
[-- Type: text/plain, Size: 21540 bytes --]

GLEP: 42
Title: Critical News Reporting
Version: $Revision: $
Author: Ciaran McCreesh <ciaranm@gentoo.org>
Last-Modified: $Date: $
Status: Draft
Type: Standards Track
Content-Type: text/x-rst
Created: 31-Oct-2005
Post-History: 1-Nov-2005, 5-Nov-2005, 7-Nov-2005, 11-Dec-2005, 13-Dec-2005, 18-Dec-2005

Abstract
========

This GLEP proposes a new way of informing users about important updates and news
regarding tree-related items.

Motivation
==========

Although most package updates are clean and require little user action,
occasionally an upgrade requires user intervention during the upgrade process.
Recent examples of the latter include the ``gcc-3.4`` stabilisation on ``x86``
and the ``mysql-4.1`` database format changes.

There are currently several ways of delivering important news items to our
users, none of them particularly effective:

* Gentoo Weekly News
* The ``gentoo-announce``, ``gentoo-user`` and ``gentoo-dev`` mailing lists
* The Gentoo Forums
* The main Gentoo website
* RSS feeds of Gentoo news
* ``einfo`` and ``ewarn`` messages in ``pkg_setup`` or ``pkg_postinst``

A more reliable way of getting news of critical updates out to users is required
to avoid repeats of the various recent upgrade debacles. This GLEP proposes a
solution based around pushing news items out to the user via the ``rsync`` tree.

.. Important:: This GLEP does not seek to replace or modify ``einfo`` messages
   which are displayed post-install. That is a separate issue which is handled
   by ``elog`` [#bug-11359]_.

Requirements
============

An adequate solution must meet all of the following requirements:

Preemptive
    Users should be told of changes *before* they break a system, not after the
    damage has already been done. Ideally, the system administrator would be
    given ample warning to plan difficult upgrades and changes, rather than only
    being told just before action is necessary.

No user subscription required
    It has already been demonstrated [#forums-apache2]_ that many users do not
    read the ``gentoo-announce`` mailing list or ``RSS`` feeds. A solution which
    requires subscription has no advantage over current methods.

No user monitoring required
    It has already been demonstrated [#forums-apache2]_ that many users do not
    read news items posted to the Gentoo website, or do not read news items
    until it is too late. A solution that relies upon active monitoring of a
    particular source has no advantage over current methods.

Relevant
    System administrators who do not use a particular package should not have to
    read news items which affect purely that package. Some news items may be of
    relevance to most or all users, but those that are not should not be forced
    upon users unnecessarily.

Lightweight
    It is not reasonable to expect all users to have an MTA, web browser, email
    client, cron daemon or text processing suite available on their system.
    Users must not be forced to install unreasonable extra software to be able
    to read news items.

No privacy violations
    Users of the solution should not be required to provide information about
    their systems (for example, IP addresses or installed packages).

Multiple delivery method support
    Some users may wish to view news items via email, some via a terminal and
    some via a web browser. A solution should either support all of these
    methods or (better still) make it simple to write clients for displaying
    news items in different ways.

The following characteristics would be desirable:

Internationalisable
    Being able to provide messages in multiple languages may be beneficial.

Quality control
    There should be some way to ensure that badly written or irrelevant messages
    are not sent out, for example by inexperienced developers or those whose
    English language skills are below par.

Simple for developers
    Posting news items should be as simple as is reasonably possible.

Simple for users
    Reading relevant news items should be as simple as is reasonably possible.

Compatibility with existing and future news sources
    A news system would ideally be able to be integrated with existing news
    sources (for example, Forums, GWN, the main Gentoo website) without
    excessive difficulty. Similarly, easy interoperation with any future news
    sources should not be precluded.

Specification
=============

Overview
--------

News items are published and delivered to users as follows:

1. A news item is written. The format to be used is described below.

2. The news item is reviewed, following the process described in
   `News Item Quality Control`_.

3. The news item is committed to a CVS (or Subversion [#glep-36]_) repository.
   From here, it is merged with the rsync tree. This is described in `News Item
   Distribution`_.

4. Users fetch the news item when they sync. This ensures that the news items in
   question are pushed to the user before the user accidentally makes an
   unwanted change. No changes to the existing rsync process are required by
   this GLEP.

5. The package manager filters the news item and, if it is relevant, marks the
   news item for reading. The package manager should also display a notice
   informing the user that there are unread news items.

6. The news item is handled by the user's choice of news item reader. See `News
   Item Clients`_.

Required Portage Enhancements
-----------------------------

The following extensions to Portage are required:

TODO: ferringb wants spaces added to the first item on the list. I don't,
because it makes repo id -> filename mappings nasty.

* Every repository (including overlays) will require a unique identifier. It is
  assumed that an identifier will be a string consisting of characters from
  ``a`` to ``z``, ``A`` to ``Z``, ``0`` to ``9``, ``+`` (plus), ``-`` (hyphen),
  ``:`` (colon) and ``_`` (underscore).

* Portage must provide a way for external programs to obtain a list of all
  repository identifiers for a given system. It is assumed that this will be in
  the form of a ``portageq`` command (e.g. ``portageq get_repo_ids``).

* Portage must provide a way for external programs to obtain the base path for
  a repository with a given ID. It is assumed that this will be in the form of
  a ``portageq`` command (e.g. ``portageq get_repo_root gentoo-x86``).

* Portage must extend ``portageq has_version`` to support restrictions to a
  given repository ID.

* Portage must extend ``portageq`` to implement a command which returns whether
  or not the profile used for a given repository ID matches a certain base path
  (e.g. ``portageq profile_used default-linux/sparc/sparc64/2004.3 gentoo-x86``).

These extensions are assumed during the following specification.

News Item Identities
--------------------

Each news item will have a unique identifier. This identifier will be in the
form ``yyyy-mm-dd-short-name``, where ``yyyy`` is the year (e.g. ``2005``),
``mm`` is the month (``01`` through ``12``) and dd is the day of the month
(``01`` through ``31``). The ``short-name`` is a very short name describing the
news item (e.g. ``yoursql-updates``), consisting only of the characters ``a-z``,
``0-9``, ``+`` (plus), ``:`` (colon), ``-`` (hyphen) and ``_`` (underscore).

News Item Directories
---------------------

Each news item will be represented by a directory whose name is the same as the
news item's identifier.

The directory will contain a file named ``yyyy-mm-dd-short-name.en.txt``, which
contains the text of the news item, in English, in the format described below.

If a news item is translated, other files named ``yyyy-mm-dd-short-name.xx.txt``
(where ``xx`` is the ISO 639 [#iso-639]_ two letter country code) will also be
provided. However, only the English version of a news item is authoritative.
This anglocentricity is justified by precedent [#glep-34]_.

News Item Files
---------------

A news item file is a text file, encoded using UTF-8 [#rfc-3629]_ for
compatibility with and for the same reasons as existing Gentoo documentation
[#docs-policy]_ and the tree [#glep-31]_.

News items should be signed with a detached GPG signature: ::

    gpg --armour --detach-sign ????-??-??-*.??.txt

A news item file's content will consist of an RFC 822 style header [#rfc-822]_
followed by the main body of the message as plain text. This GLEP defines
various optional and mandatory headers. Future GLEPs may propose new headers —
tools handling these news items must ignore any unrecognised header.

News Item Headers
'''''''''''''''''

The following headers describe the purpose and format of the news item:

``Title:``
    A short (maximum 44 characters) descriptive title. Mandatory.

``Author:``
    Author's name and email address, in the form ``Real Name <email@address>``.
    Mandatory; multiple author headers may be specified if appropriate.

``Translator:``
    For translated news items, the translator's name and email address. Multiple
    translator headers may be specified if appropriate.

``Content-Type:``
    Must be ``text/plain``. Mandatory.

``Posted:``
    Date of posting, in ``yyyy-mm-dd`` format (e.g. 2005-12-18) for
    compatibility with GLEP 45 [#glep-45]_. Mandatory.

``Revision:``
    Initially 1. Incremented every time a non-trivial change is made.  Changes
    which require a re-read of the news item should instead use a new news item
    file. Mandatory.

``News-Item-Format:``
    Must be ``1.0``. Future revisions to the format may increment the minor
    number for backwards-compatible changes, or the major number for major
    changes.

The following headers are used for filtering:

``Display-If-Installed:``
    A dependency atom or simple package name (for example,
    ``<dev-lang/php-5_alpha`` or ``net-www/apache``). If the user has the
    package specified installed from the repository from which the news item was
    obtained, the news item should be displayed.

``Display-If-Keyword:``
    A keyword [#glep-22]_ name, for example ``mips`` or ``x86-fbsd``. If the
    user is on the keyword in question, the news item should be displayed.

``Display-If-Profile:``
    A profile path, for example ``default-linux/sparc/sparc64/server``. If the
    user is using the exact profile in question, or a subprofile of this
    profile, the news item should be displayed. This header may be used to
    replace ``deprecated`` files in the future.

.. Note:: When performing package moves, developers must also update any
   relevant ``Display-If-Installed`` headers in news files.

The algorithm used to determine whether a news item is 'relevant' is as
follows:

* For each ``Display-If-`` header type which occurs at least once:

  + The news item is not relevant if none of the headers of this type are
    successfully matched.

* Otherwise the news item is relevant.

In particular, if no ``Display-If-`` header is specified, a news item will be
relevant for all users.

This algorithm was chosen because it makes conditions like "display this news
item for ``YourSQL`` users who are on ``sparc`` or ``x86-obsd`` relatively
simple to specify — it is believed that these kinds of condition are far more
likely to occur than "display this news item for people using ``YourSQL``, or
for people on ``sparc`` or ``x86-obsd``\" or "display these news items for
people who use ``YourSQL`` and who are on both ``sparc`` and ``x86-obsd``\".

News Item Body
''''''''''''''

The header section must be followed by a blank line, then the main body of the
text.

The text body should be wrapped at 72 characters. No fancy formatting or tab
characters should be used — the news item may be being displayed directly to a
terminal. Paragraphs should be separated by a blank line.

Hyperlinks may be used to refer to further information (for example, an upgrade
guide). However, the main body of the news item should be descriptive and not
simply a "read this link" text. It is assumed that the user will have access to
a web browser *somewhere*, but not necessarily on the box which is being
administrated — this will be the case on may servers and routers, for example.

Example News Item
'''''''''''''''''

`This hypothetical news item`__ could be used for an upgrade to the
``YourSQL`` database format which breaks forward compatibility.

.. __: glep-0042-extras/example-news-item.txt

News Item Quality Control
-------------------------

There have been complaints regarding the comprehensibility of some upgrade
notices and news items in the past. This is understandable — not every Gentoo
developer speaks English as a first language. However, for the sake of clarity,
professionalism and avoiding making us look like prats, it is important that any
language problems be corrected before inflicting a news item upon end users.

Thus, at least 72 hours before a proposed news item is committed, it must be
posted to the ``gentoo-dev`` mailing list and ``Cc:``\ed to ``pr@gentoo.org``
(exceptions may be made in exceptional circumstances). Any complaints — for
example regarding wording, clarity or accuracy — **must** be addressed before
the news item goes live.

.. Note:: A previous draft of this GLEP allowed news items to be sent to
   ``gentoo-core`` instead of ``gentoo-dev``. It is possible that a situation
   may arise where this will be necessary (for example, a security update which
   must break backwards compatibility and which cannot be revealed to the public
   before a given date).

News items must only be for **important** changes that may cause serious upgrade
or compatibility problems. Ordinary upgrade messages and non-critical news items
should remain in ``einfo`` notices. The importance of the message to its
intended audience should be justified with the proposal.

.. Important:: The filtering system means that it is appropriate to send out
   news items which are aimed at users of an uncommon package or architecture.
   Thus, the justification should be in the form "this message is important to
   YourSQL users because ...", not "YourSQL is important because ...".

News Item Distribution
----------------------

Server Side
'''''''''''

News items are to be made available via the standard rsync tree. This removes
any need for polling of a remote source.

A new repository will be created for news items. The type (CVS or Subversion),
location and access controls on this repository are beyond the scope of this
GLEP.

.. Note:: A previous draft of this GLEP instead used the main ``gentoo-x86``
   tree. This was changed following advice from Infrastructure
   [#ramereth-repo]_. Both solutions have the same end result.

This repository will contain directories named ``yyyy/mm/``, where ``yyyy`` is
the current year and ``mm`` is the current month number (01 for January through
12 for December). This separation will help keep news items more manageable.

The contents of this repository will automatically be merged with the main rsync
tree, placing the items in a ``metadata/news/`` directory. The method used for
merging these items is beyond the scope of this GLEP — a similar setup is
already used for merging GLSAs into the rsync tree.

The main rsync tree will **not** use the ``yyyy/mm/`` subdirectory layout.

Client Side
'''''''''''

Whenever relevant unread news items are found, the package manager will create a
file named ``/var/lib/gentoo/news/news-repoid.unread`` (if it does not
already exist) and append the news item identifier (eg
``2005-11-01-yoursql-updates``) on a new line.

Notification that new relevant news items will be displayed via the
``emerge`` tool in a similar way to the existing "configuration files need
updating" messages:

::

    * Important: there are 5 unread news items.
    * Type emerge --help news to learn how to read news files.

Checks for new news messages should be displayed:

* After an ``emerge sync``
* After an ``emerge --pretend``
* Before an ``emerge <target>`` (which may also include a red warning message)
* Before an ``emerge --ask <target>`` sequence

The package manager may use a timestamp check file to avoid having to process
news items unnecessarily.

The package manager must keep track of news items that have already been added
to the unread list to avoid repeatedly marking a deleted news item. This could
be handled via a ``news-repoid.skip`` file containing the IDs of news items that
have already been added to a ``news-repoid.unread`` file, but this method is not
required by this GLEP.

Users who really don't care about news items can use ``rsync_excludes`` to
filter out the ``metadata/news/`` directory.

News Item Clients
-----------------

Once a news item is marked for reading, third party tools (or traditional core
Unix tools) can be used to display and view the news files.

When a news item is read, its name should be removed from the
``news-repoid.unread`` file. If a news client acts as an interactive reader
rather than a gateway, it should then add the name to a ``news-repoid.read``
file in the same directory with the same file format.

An ``eselect`` [#eselect]_ module shall be created as the 'suggested' display
tool; other display tools (for example, a news to email forwarder, which would
be ideal for users who sync on a ``cron``) are left as options for those who
desire them.

News Item Removal
-----------------

News items can be removed (by removing the news file from the main tree) when
they are no longer relevant, if they are made obsolete by a future news item or
after a long period of time. This is the same as the method used for ``updates``
entries.

Integration with Existing Systems
=================================

It would be simple to convert these news items into the format used for news
items on the Gentoo website or posts for the ``gentoo-announce`` mailing list.

There is an existing automated tool [#forums-glsa]_ for posting GLSAs to the
forums. A similar tool can be used for these news items.

Backwards Compatibility
=======================

Backwards compatibility is not a concern here. Existing tools will simply ignore
the ``news/`` directory.

Reference Implementation
========================

Portage Code
------------

TODO

Simple ``eselect`` News Client
------------------------------

TODO Removed until the exact format details are figured out.

Simple News to Mail Forwarder
-----------------------------

TODO Removed until the exact format details are figured out.

Credits
=======

The idea behind notifying users of news updates via Portage comes from Stuart
Herbert [#stuart-blog]_.

Thanks to Lance Albertson, Stephen Bennett, Donnie Berkholz, Grant Goodyear,
Brian Harring, Dan Meltzer, Jason Stubbs, Paul de Vrieze and Alec Warner for
input.  Some of the ideas presented here are theirs, others go completely
against their suggestions.

Example Files
=============

TODO Removed until the exact format details are figured out.

References
==========

.. [#bug-11359] Bugzilla Bug 11359
   "[NEW FEATURE] pkg_postinst/pkg_preinst ewarn/einfo logging",
   https://bugs.gentoo.org/show_bug.cgi?id=11359
.. [#docs-policy] Gentoo XML Guide, Daniel Robbins et al.,
     http://www.gentoo.org/doc/en/xml-guide.xml
.. [#eselect] eselect modular framework for configuration and
     administration utilities,
     http://www.gentoo.org/proj/en/eselect/index.xml
.. [#forums-glsa] Forums user GLSA,
     http://forums.gentoo.org/profile.php?mode=viewprofile&u=55648
.. [#forums-apache2] Forums thread "Gentoo Apache2 Config Change Idiocy",
     http://forums.gentoo.org/viewtopic-t-384368.html
.. [#glep-22] GLEP 22: "New "keyword" system to incorporate various
     userlands/kernels/archs", Grant Goodyear,
     http://www.gentoo.org/proj/en/glep/glep-0022.html
.. [#glep-31] GLEP 31: "Character Sets for Portage Tree Items", Ciaran
     McCreesh,
     http://www.gentoo.org/proj/en/glep/glep-0031.html
.. [#glep-34] GLEP 34: "Per-Category metadata.xml Files", Ciaran McCreesh,
     http://www.gentoo.org/proj/en/glep/glep-0034.html
.. [#glep-36] GLEP 36: "Subversion/CVS for Gentoo Hosted Projects", Aaron
     Walker,
     http://www.gentoo.org/proj/en/glep/glep-0036.html
.. [#glep-45] GLEP 45: "GLEP date format", Henrik Brix Andersen,
     http://www.gentoo.org/proj/en/glep/glep-0045.html
.. [#iso-639] ISO 639 "Code for the representation of names of languages"
.. [#ramereth-repo] "Re: [gentoo-dev] GLEP ??: Critical News Reporting", Lance
     Albertson,
     http://marc.theaimsgroup.com/?l=gentoo-dev&m=113111585907703&w=2
.. [#rfc-822] RFC 822 "Standard for the format of ARPA Internet text messages"
.. [#rfc-3629] RFC 3629: "UTF-8, a transformation format of ISO 10646"
       http://www.ietf.org/rfc/rfc3629.txt
.. [#stuart-blog] "Favouring an automatic news mechanism", Stuart Herbert,
     http://stu.gnqs.org/diary/gentoo.php/2005/10/28/favouring_an_automatic_news_mechanism

Copyright
=========

This document has been placed in the public domain.

.. vim: set tw=80 fileencoding=utf-8 spell spelllang=en et :

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] glep 42 (news) round six

[Curtis Napier]

I agree with ciaranm that repository IDs should not be allowed to 
contain spaces. Seriously.
-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] glep 42 (news) round six

[Andrew Gaffney]

I agree with ciaranm (did I really just say that?) that repository IDs should 
not be allowed to contain spaces.

-- 
Andrew Gaffney                            http://dev.gentoo.org/~agaffney/
Gentoo Linux Developer                                   Installer Project
-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] glep 42 (news) round six

[John Myers]

[-- Attachment #1: Type: text/plain, Size: 475 bytes --]

Not that my lowly user opinion matters, but I agree with ciaranm that 
repository IDs should not be allowed to contain spaces

GLEP 42 wrote:
> * Before an ``emerge --ask <target>`` sequence
What, exactly, does this mean? I'm guessing it means between the package list 
and the prompt, but it's not quite clear.

Also, there's a typo in the last line of the "News Item Body" section: 
s/may/many/

-- 
# 
# electronerd, the electronerdian from electronerdia
#

[-- Attachment #2: Type: application/pgp-signature, Size: 198 bytes --]

Re: [gentoo-dev] glep 42 (news) round six

[Ciaran McCreesh]

[-- Attachment #1: Type: text/plain, Size: 675 bytes --]

On Sat, 17 Dec 2005 20:50:43 -0800 John Myers
<electronerd@electronerdia.net> wrote:
| GLEP 42 wrote:
| > * Before an ``emerge --ask <target>`` sequence
| What, exactly, does this mean? I'm guessing it means between the
| package list and the prompt, but it's not quite clear.

I dunno, I refuse to use emerge --ask for religious reasons. Feel free
to rewrite that bullet point.

| Also, there's a typo in the last line of the "News Item Body"
| section: s/may/many/

Not any more there isn't.

-- 
Ciaran McCreesh : Gentoo Developer (I can kill you with my brain)
Mail            : ciaranm at gentoo.org
Web             : http://dev.gentoo.org/~ciaranm


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] glep 42 (news) round six

[John Myers]

[-- Attachment #1: Type: text/plain, Size: 657 bytes --]

On Saturday 17 December 2005 20:57, Ciaran McCreesh wrote:
> On Sat, 17 Dec 2005 20:50:43 -0800 John Myers
>
> <electronerd@electronerdia.net> wrote:
> | GLEP 42 wrote:
> | > * Before an ``emerge --ask <target>`` sequence
> |
> | What, exactly, does this mean? I'm guessing it means between the
> | package list and the prompt, but it's not quite clear.
>
> I dunno, I refuse to use emerge --ask for religious reasons. Feel free
> to rewrite that bullet point.
>
How about

* During ``emerge --ask``, the same as for ``emerge --pretend`` (i.e. after 
the package list)

?
-- 
# 
# electronerd, the electronerdian from electronerdia
#

[-- Attachment #2: Type: application/pgp-signature, Size: 198 bytes --]

Re: [gentoo-dev] glep 42 (news) round six

[Brian Harring]

[-- Attachment #1: Type: text/plain, Size: 14079 bytes --]

On Sun, Dec 18, 2005 at 04:15:45AM +0000, Ciaran McCreesh wrote:
> Here we go again... Changes:
> 
> * We're now supporting overlays, multiple repositories and magic flying
> chickens. To do this, we're shoving a whole load of new requirements
> onto Portage. Said requirements are documented under `Required Portage
> Enhancements`_. I now expect to get heavily flamed by a different set
> of Portage developers.

You forgot the magic mushroom badgers and snake.


> * Change to yyyy-mm-dd for GLEP 45 compatibility. Rereremove the
> timestamp on the Posted: header.
> 
> * Tweak Display-If-{Profile,Installed} to work with multiple
> repositories.
> 
> * Use ${repoid} rather than magic-chicken for news-*.* files.

Drop the magic-chicken crap (especially in light of your comments 
about 'professional' news inline in the glep).

Killjoy maybe, but it detracts from the point of the glep.

> * Be specific on how news-repoid.skip can work.

Still haven't gotten into specifics, merely a different bit of hand 
waving.


> You are encouraged to reply to this thread saying "I agree with ciaranm
> that repository IDs should not be allowed to contain spaces".
>
> TODO: ferringb wants spaces added to the first item on the list. I don't,
> because it makes repo id -> filename mappings nasty.
>
> * Every repository (including overlays) will require a unique identifier. It is
>   assumed that an identifier will be a string consisting of characters from
>   ``a`` to ``z``, ``A`` to ``Z``, ``0`` to ``9``, ``+`` (plus), ``-`` (hyphen),
>   ``:`` (colon) and ``_`` (underscore).

Not really.  Just requires your code to be space safe.

You write good code, right?  Especially since you need to keep our 
path handling safe for osx (/volumes) and for users who do strange 
things?

The news handling crap should be able to take spaces- remember the 
comments about user aliasing of repostory ID's down the line.  I don't 
care if the actual metadata/repo_id lacks spaces, but the handling 
code must be able to take spaces, as such the requirement that spaces 
not be used is arbitrary and uneeded.


> * Portage must provide a way for external programs to obtain a list of all
>   repository identifiers for a given system. It is assumed that this will be in
>   the form of a ``portageq`` command (e.g. ``portageq get_repo_ids``).
> 
> * Portage must provide a way for external programs to obtain the base path for
>   a repository with a given ID. It is assumed that this will be in the form of
>   a ``portageq`` command (e.g. ``portageq get_repo_root gentoo-x86``).
> 
> * Portage must extend ``portageq has_version`` to support restrictions to a
>   given repository ID.
> 
> * Portage must extend ``portageq`` to implement a command which returns whether
>   or not the profile used for a given repository ID matches a certain base path
>   (e.g. ``portageq profile_used default-linux/sparc/sparc64/2004.3 gentoo-x86``).

profile_in_use, maybe.


> These extensions are assumed during the following specification.
> 
> News Item Identities
> --------------------
> 
> Each news item will have a unique identifier. This identifier will be in the
> form ``yyyy-mm-dd-short-name``, where ``yyyy`` is the year (e.g. ``2005``),
> ``mm`` is the month (``01`` through ``12``) and dd is the day of the month
> (``01`` through ``31``). The ``short-name`` is a very short name describing the
> news item (e.g. ``yoursql-updates``), consisting only of the characters ``a-z``,
> ``0-9``, ``+`` (plus), ``:`` (colon), ``-`` (hyphen) and ``_`` (underscore).
> 
> News Item Directories
> ---------------------
> 
> Each news item will be represented by a directory whose name is the same as the
> news item's identifier.
> 
> The directory will contain a file named ``yyyy-mm-dd-short-name.en.txt``, which
> contains the text of the news item, in English, in the format described below.
> 
> If a news item is translated, other files named ``yyyy-mm-dd-short-name.xx.txt``
> (where ``xx`` is the ISO 639 [#iso-639]_ two letter country code) will also be
> provided. However, only the English version of a news item is authoritative.
> This anglocentricity is justified by precedent [#glep-34]_.
> 
> News Item Files
> ---------------
> 
> A news item file is a text file, encoded using UTF-8 [#rfc-3629]_ for
> compatibility with and for the same reasons as existing Gentoo documentation
> [#docs-policy]_ and the tree [#glep-31]_.
> 
> News items should be signed with a detached GPG signature: ::

why should vs must?
Should leaves open the possibility for a tree compromise and a news 
item with Very Bad(tm) instructions stuck into it.

Moving towards getting the whole tree signed, introducing new 
components that aren't required signed works against that.

> News Item Headers
> '''''''''''''''''
> 
> The following headers describe the purpose and format of the news item:
> 
> ``Title:``
>     A short (maximum 44 characters) descriptive title. Mandatory.
> 
> ``Author:``
>     Author's name and email address, in the form ``Real Name <email@address>``.
>     Mandatory; multiple author headers may be specified if appropriate.
> 
> ``Translator:``
>     For translated news items, the translator's name and email address. Multiple
>     translator headers may be specified if appropriate.
> 
> ``Content-Type:``
>     Must be ``text/plain``. Mandatory.
> 
> ``Posted:``
>     Date of posting, in ``yyyy-mm-dd`` format (e.g. 2005-12-18) for
>     compatibility with GLEP 45 [#glep-45]_. Mandatory.
> 
> ``Revision:``
>     Initially 1. Incremented every time a non-trivial change is made.  Changes
>     which require a re-read of the news item should instead use a new news item
>     file. Mandatory.
non-trivial changes that don't require a re-read sounds like a 
contradiction.  Clarify, especially since portage will mark this as 
read _once_ and only once.

> The following headers are used for filtering:
> 
> ``Display-If-Installed:``
>     A dependency atom or simple package name (for example,

Drop the 'simple package name'; it still is an atom.

>     ``<dev-lang/php-5_alpha`` or ``net-www/apache``). If the user has the
>     package specified installed from the repository from which the news item was
>     obtained, the news item should be displayed.

This isn't incredibly useful if ranged versions are ever introduced.  
Ammending the glep for that seems stupid, looser language might be 
wise.

> ``Display-If-Keyword:``
>     A keyword [#glep-22]_ name, for example ``mips`` or ``x86-fbsd``. If the
>     user is on the keyword in question, the news item should be displayed.
> 
> ``Display-If-Profile:``
>     A profile path, for example ``default-linux/sparc/sparc64/server``. If the
>     user is using the exact profile in question, or a subprofile of this
>     profile, the news item should be displayed. This header may be used to
>     replace ``deprecated`` files in the future.
> 
> .. Note:: When performing package moves, developers must also update any

>    relevant ``Display-If-Installed`` headers in news files.
Grounds for someone to get off their ass and do some work, but this 
should be automated in some fashion- specifically detection of it 
(auto-updating won't work with signing).

> The algorithm used to determine whether a news item is 'relevant' is as
> follows:
> 
> * For each ``Display-If-`` header type which occurs at least once:
> 
>   + The news item is not relevant if none of the headers of this type are
>     successfully matched.
> 
> * Otherwise the news item is relevant.
> 
> In particular, if no ``Display-If-`` header is specified, a news item will be
> relevant for all users.

implementation issue= be aware this must be cached.  No caching == 
slowing down portage pissing off users.

All news items *will* need to be cached in some format here- just 
because the keyword isn't what the user is running at the time of 
sync, doesn't mean that the keyword won't be used for ROOT!="/" cross 
compilation.

That's an implementation note, but I'm bringing it up since the time 
to do the filter/cache instantiation is during portage initialization 
(yes it sucks, but your stuck with stable)... so start thinking about 
ways to make it fast, since python -c'import portage' is the slowest 
part of portage queries


> News Item Quality Control
> -------------------------
> 
> There have been complaints regarding the comprehensibility of some upgrade
> notices and news items in the past. This is understandable ??? not every Gentoo

'???' ?

> .. Note:: A previous draft of this GLEP allowed news items to be sent to
>    ``gentoo-core`` instead of ``gentoo-dev``. It is possible that a situation
>    may arise where this will be necessary (for example, a security update which
>    must break backwards compatibility and which cannot be revealed to the public
>    before a given date).

Drop that and just state that -core doesn't fly sans security 
consideration; referencing one of the previous 5 versions isn't needed 
(yes it's minor, but it's uneeded info).

> Server Side
> '''''''''''
> 
> News items are to be made available via the standard rsync tree. This removes
> any need for polling of a remote source.
> 
> A new repository will be created for news items. The type (CVS or Subversion),
> location and access controls on this repository are beyond the scope of this
> GLEP.

We generate a tree every 30 minutes.  You aiming for every update, or 
for less frequent (once an hour fex)?

Matters due to the fact rsync tree generation has a window it must not 
overrun- minor but continuing the "lets be explicit" goal of yours.


> This repository will contain directories named ``yyyy/mm/``, where ``yyyy`` is
> the current year and ``mm`` is the current month number (01 for January through
> 12 for December). This separation will help keep news items more manageable.
> 
> The contents of this repository will automatically be merged with the main rsync
> tree, placing the items in a ``metadata/news/`` directory. The method used for
> merging these items is beyond the scope of this GLEP ??? a similar setup is
> already used for merging GLSAs into the rsync tree.
> 
> The main rsync tree will **not** use the ``yyyy/mm/`` subdirectory layout.

What shall it use?  Again, be explicit.


> Client Side
> '''''''''''
> 
> Whenever relevant unread news items are found, the package manager will create a
> file named ``/var/lib/gentoo/news/news-repoid.unread`` (if it does not
-file named ``/var/lib/gentoo/news/news-repoid.unread`` (if it does not
+file named ``/var/lib/gentoo/news/news-${repoid}.unread`` (if it does not

Clearer at a first read though imo.

> already exist) and append the news item identifier (eg
> ``2005-11-01-yoursql-updates``) on a new line.

Implementation details, but this handling will need to be integrated 
into portage (eg, no external scripts).  External scripts will slow 
things down (shell overhead/fork/exec), plus portage is going to have 
to query this cruft already for the notifications on emerge targets 
(no we're not execing everytime for that either ;)

> Checks for new news messages should be displayed:
> 
> * After an ``emerge sync``
> * After an ``emerge --pretend``
> * Before an ``emerge <target>`` (which may also include a red warning message)
> * Before an ``emerge --ask <target>`` sequence

ask is superfluous, nuke it.

You haven't stated how the 'package manager' will trigger the user's 
reader of choice for these targets.  Should also extend this to allow 
a way to disable any news notices, lest someone's cronjob get hung 
displaying news (feature or not, it's needed).


> The package manager must keep track of news items that have already been added
> to the unread list to avoid repeatedly marking a deleted news item. This could
> be handled via a ``news-repoid.skip`` file containing the IDs of news items that
> have already been added to a ``news-repoid.unread`` file, but this method is not
> required by this GLEP.

Specifics.


> Once a news item is marked for reading, third party tools (or traditional core
> Unix tools) can be used to display and view the news files.
> 
> When a news item is read, its name should be removed from the
> ``news-repoid.unread`` file. If a news client acts as an interactive reader
> rather than a gateway, it should then add the name to a ``news-repoid.read``
> file in the same directory with the same file format.

implementation issue; you need locking on this.  If portage has to 
farm out to the users reader of choice, then it needs to lock the file 
to keep readers/writers from screwing with each other.

Flock preferably, since it's faster then resorting to hardlink 
trickery (yes this may be harder for shell crap, but so it goes since 
hardlink locking sucks).


> News Item Removal
> -----------------
> 
> News items can be removed (by removing the news file from the main tree) when
> they are no longer relevant, if they are made obsolete by a future news item or
> after a long period of time. This is the same as the method used for ``updates``
> entries.

implementation issue; updating unread/skip crap so it doesn't bloat is 
required, but time frame also matters (crap sync resulting in news 
getting removed, yet it still being valid, just missing from the local 
tree).

> Integration with Existing Systems
> =================================
> 
> It would be simple to convert these news items into the format used for news
> items on the Gentoo website or posts for the ``gentoo-announce`` mailing list.
> 
> There is an existing automated tool [#forums-glsa]_ for posting GLSAs to the
> forums. A similar tool can be used for these news items.

Pawned it off on someone, or something you'll be doing?

~harring

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] glep 42 (news) round six

[Ciaran McCreesh]

[-- Attachment #1: Type: text/plain, Size: 8750 bytes --]

On Sat, 17 Dec 2005 21:50:47 -0800 Brian Harring <ferringb@gentoo.org>
wrote:
| Drop the magic-chicken crap (especially in light of your comments 
| about 'professional' news inline in the glep).

Eh, there aren't any magic chickens in the glep.

| Not really.  Just requires your code to be space safe.
| 
| You write good code, right?  Especially since you need to keep our 
| path handling safe for osx (/volumes) and for users who do strange 
| things?

The problem isn't my code. My code's fine. So is eselect. The problem is
people who write their own handler scripts to suit their own needs, and
then get nastily bitten because they don't realise we're allowing
spaces in filenames.

Do you remember that Apple installer program that ended up in effect
doing rm -fr / if you tried to install a particular OS X program to a
volume whose name contained a space?

| > * Portage must extend ``portageq`` to implement a command which
| > returns whether or not the profile used for a given repository ID
| > matches a certain base path (e.g. ``portageq profile_used
| > default-linux/sparc/sparc64/2004.3 gentoo-x86``).
| 
| profile_in_use, maybe.

Mmm, that use looks too much like USE. *shrug* Exact names don't matter
at this stage anyway.

| > News items should be signed with a detached GPG signature: ::
| 
| why should vs must?
| Should leaves open the possibility for a tree compromise and a news 
| item with Very Bad(tm) instructions stuck into it.
| 
| Moving towards getting the whole tree signed, introducing new 
| components that aren't required signed works against that.

I don't want to introduce any signing requirements that we don't have
already. When signing for everything else becomes mandatory, signing
for news would be too. If I make this a 'must', someone will ask me to
specify how we're handling the Gentoo keyring...

| > ``Revision:``
| >     Initially 1. Incremented every time a non-trivial change is
| > made.  Changes which require a re-read of the news item should
| > instead use a new news item file. Mandatory.
|
| non-trivial changes that don't require a re-read sounds like a 
| contradiction.  Clarify, especially since portage will mark this as 
| read _once_ and only once.

Hrm, word it as "Changes other than minor formatting tweaks", or remove
"non-trivial"?

| > The following headers are used for filtering:
| > 
| > ``Display-If-Installed:``
| >     A dependency atom or simple package name (for example,
| 
| Drop the 'simple package name'; it still is an atom.

Ok.

| This isn't incredibly useful if ranged versions are ever introduced.  
| Ammending the glep for that seems stupid, looser language might be 
| wise.

What's the syntax for ranged versions? And how do they differ from SLOT
versions?

| 
| > .. Note:: When performing package moves, developers must also
| > update any
| >    relevant ``Display-If-Installed`` headers in news files.
|
| Grounds for someone to get off their ass and do some work, but this 
| should be automated in some fashion- specifically detection of it 
| (auto-updating won't work with signing).

Moving packages in general requires a re-sign anyway. I'm guessing that
epkgmove will handle this, if it ever starts working again...

| That's an implementation note, but I'm bringing it up since the time 
| to do the filter/cache instantiation is during portage initialization 
| (yes it sucks, but your stuck with stable)... so start thinking about 
| ways to make it fast, since python -c'import portage' is the slowest 
| part of portage queries

Looks like I might have to do that thing in Python rather than bash
then...

| > News Item Quality Control
| > -------------------------
| > 
| > There have been complaints regarding the comprehensibility of some
| > upgrade notices and news items in the past. This is
| > understandable ??? not every Gentoo
| 
| '???' ?

It's a dash. It'll show properly in the HTML version.

| 
| > .. Note:: A previous draft of this GLEP allowed news items to be
| > sent to ``gentoo-core`` instead of ``gentoo-dev``. It is possible
| > that a situation may arise where this will be necessary (for
| > example, a security update which must break backwards compatibility
| > and which cannot be revealed to the public before a given date).
| 
| Drop that and just state that -core doesn't fly sans security 
| consideration; referencing one of the previous 5 versions isn't
| needed (yes it's minor, but it's uneeded info).

Ok, the entire Note's gone. If we ever really have to do a nasty update
for security reasons, whichever poor shmuck ends up handling it should
be smart enough to know what to do...

| We generate a tree every 30 minutes.  You aiming for every update, or 
| for less frequent (once an hour fex)?
|
| Matters due to the fact rsync tree generation has a window it must
| not overrun- minor but continuing the "lets be explicit" goal of
| yours.

Once an hour would work fine. On the other hand, the merge is just
copying a few small files -- time-wise, it's less than generating the
cache for a couple of ebuilds.

| > The main rsync tree will **not** use the ``yyyy/mm/`` subdirectory
| > layout.
| 
| What shall it use?  Again, be explicit.

+ The news item directories will be moved into the single top level
directory.

Not sure whether we really want to do that. It makes the client code
slightly easier...

| -file named ``/var/lib/gentoo/news/news-repoid.unread`` (if it does
| not +file named ``/var/lib/gentoo/news/news-${repoid}.unread`` (if it
| does not
| 
| Clearer at a first read though imo.

Ok.

| ask is superfluous, nuke it.

Waah! I was told last time around that I had to add it. Looks like I
might have to taint myself by trying emerge --ask to see what it really
does...

| You haven't stated how the 'package manager' will trigger the user's 
| reader of choice for these targets.  Should also extend this to allow 
| a way to disable any news notices, lest someone's cronjob get hung 
| displaying news (feature or not, it's needed).

The same way the package manager handles updating config files: it
won't. It'll just tell the user that some news items need reading.


| > The package manager must keep track of news items that have already
| > been added to the unread list to avoid repeatedly marking a deleted
| > news item. This could be handled via a ``news-repoid.skip`` file
| > containing the IDs of news items that have already been added to a
| > ``news-repoid.unread`` file, but this method is not required by
| > this GLEP.
| 
| Specifics.

I'm trying to avoid forcing a particular implementation on the skip
file, since I can think of at least three different ways of doing it
and I'm not sure which will be fastest...

| > When a news item is read, its name should be removed from the
| > ``news-repoid.unread`` file. If a news client acts as an
| > interactive reader rather than a gateway, it should then add the
| > name to a ``news-repoid.read`` file in the same directory with the
| > same file format.
| 
| implementation issue; you need locking on this.  If portage has to 
| farm out to the users reader of choice, then it needs to lock the
| file to keep readers/writers from screwing with each other.

We do? Marius told me it wasn't worth it.

| Flock preferably, since it's faster then resorting to hardlink 
| trickery (yes this may be harder for shell crap, but so it goes since 
| hardlink locking sucks).

What's wrong with mkdir?

| > News items can be removed (by removing the news file from the main
| > tree) when they are no longer relevant, if they are made obsolete
| > by a future news item or after a long period of time. This is the
| > same as the method used for ``updates`` entries.
| 
| implementation issue; updating unread/skip crap so it doesn't bloat
| is required, but time frame also matters (crap sync resulting in news 
| getting removed, yet it still being valid, just missing from the
| local tree).

Hrm. We can't take the same approach here as we do with expiring
updates entries?

| > There is an existing automated tool [#forums-glsa]_ for posting
| > GLSAs to the forums. A similar tool can be used for these news
| > items.
| 
| Pawned it off on someone, or something you'll be doing?

Hopefully the former. I have it on reasonably good authority that it
won't take more than half an hour if I end up having to do it though...

-- 
Ciaran McCreesh : Gentoo Developer (I can kill you with my brain)
Mail            : ciaranm at gentoo.org
Web             : http://dev.gentoo.org/~ciaranm


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] glep 42 (news) round six

[Brian Harring]

[-- Attachment #1: Type: text/plain, Size: 8874 bytes --]

On Sun, Dec 18, 2005 at 06:23:55AM +0000, Ciaran McCreesh wrote:
> On Sat, 17 Dec 2005 21:50:47 -0800 Brian Harring <ferringb@gentoo.org>
> wrote:
> | Drop the magic-chicken crap (especially in light of your comments 
> | about 'professional' news inline in the glep).
> 
> Eh, there aren't any magic chickens in the glep.

Intention was a nudge about keeping snarky comments out of the glep, but yep.


> | Not really.  Just requires your code to be space safe.
> | 
> | You write good code, right?  Especially since you need to keep our 
> | path handling safe for osx (/volumes) and for users who do strange 
> | things?
> 
> The problem isn't my code. My code's fine. So is eselect. The problem is
> people who write their own handler scripts to suit their own needs, and
> then get nastily bitten because they don't realise we're allowing
> spaces in filenames.

People writing their own plugins/readers are responsible for what they 
create.  They *still* need to handle space's in file paths, thus as I 
stated the requirement is arbitrary and uneeded.

Already addressed this in irc; you provide a framework.  If plugins to 
it suck, that's not your problem, nor is it a valid reason to 
constrain things just because someone might manage something stupid 
(remember this is gentoo, that line of logic would lock CFLAGS down 
to a sane set).

To head off the "don't make features that are easily screwed up", this 
isn't one of them- this is expecting code to behave correctly from the 
path standpoint.

Yes it's harder on bash crap, but that's also a reflection of bash 
(it's not an issue in python :-P ).


> | > News items should be signed with a detached GPG signature: ::
> | 
> | why should vs must?
> | Should leaves open the possibility for a tree compromise and a news 
> | item with Very Bad(tm) instructions stuck into it.
> | 
> | Moving towards getting the whole tree signed, introducing new 
> | components that aren't required signed works against that.
> 
> I don't want to introduce any signing requirements that we don't have
> already. When signing for everything else becomes mandatory, signing
> for news would be too. If I make this a 'must', someone will ask me to
> specify how we're handling the Gentoo keyring...

Pawn the keyring off on others.  The issues isn't an established trust 
ring, it's required signing- yes, a trust ring makes things a helluva 
lot easier on the user front, but it's useless without a required 
signing policy.

We've already had this conversation also btw, in the 
beginning of glep42 iirc.  Obviously I don't agree 
with your reasoning "I'll do it when it's required I do it".  It's 
useful now, it becomes massively more useful when a trust ring is 
available.


> | > ``Revision:``
> | >     Initially 1. Incremented every time a non-trivial change is
> | > made.  Changes which require a re-read of the news item should
> | > instead use a new news item file. Mandatory.
> |
> | non-trivial changes that don't require a re-read sounds like a 
> | contradiction.  Clarify, especially since portage will mark this as 
> | read _once_ and only once.
> 
> Hrm, word it as "Changes other than minor formatting tweaks", or remove
> "non-trivial"?

It's not a wording thing, I'm pointing out sans spelling corrections 
and trivial word mangling, any new info jammed in requires a new item 
bump so readers can display the changes.

In light of that, wording above needs correction.


> | This isn't incredibly useful if ranged versions are ever introduced.  
> | Ammending the glep for that seems stupid, looser language might be 
> | wise.
> 
> What's the syntax for ranged versions? And how do they differ from SLOT
> versions?

>=kde-base/kde-libs-3.0 <=kde-base/kde-libs-4.0 
It's not syntax as much as a boolean and of atoms.


> | That's an implementation note, but I'm bringing it up since the time 
> | to do the filter/cache instantiation is during portage initialization 
> | (yes it sucks, but your stuck with stable)... so start thinking about 
> | ways to make it fast, since python -c'import portage' is the slowest 
> | part of portage queries
> 
> Looks like I might have to do that thing in Python rather than bash
> then...

Do it in bash if you like being on the receiving end of atomic 
wedgies.


> Once an hour would work fine. On the other hand, the merge is just
> copying a few small files -- time-wise, it's less than generating the
> cache for a couple of ebuilds.

More then a couple; this beast will bloat up to hundred or so files 
I'd expect (remember translation serves as a multiplier).

Any signed item would need to be verified also, although fortunately 
this chunk can be done in parallel to regen run.


> | > The main rsync tree will **not** use the ``yyyy/mm/`` subdirectory
> | > layout.
> | 
> | What shall it use?  Again, be explicit.
> 
> + The news item directories will be moved into the single top level
> directory.
> 
> Not sure whether we really want to do that. It makes the client code
> slightly easier...

Your decision.


> | You haven't stated how the 'package manager' will trigger the user's 
> | reader of choice for these targets.  Should also extend this to allow 
> | a way to disable any news notices, lest someone's cronjob get hung 
> | displaying news (feature or not, it's needed).
> 
> The same way the package manager handles updating config files: it
> won't. It'll just tell the user that some news items need reading.

And you'll personally handle all of the bug spam from feature requests 
that --ask trigger $news_reader?

It's a logical extension, thus people will ask for it.


> | > The package manager must keep track of news items that have already
> | > been added to the unread list to avoid repeatedly marking a deleted
> | > news item. This could be handled via a ``news-repoid.skip`` file
> | > containing the IDs of news items that have already been added to a
> | > ``news-repoid.unread`` file, but this method is not required by
> | > this GLEP.
> | 
> | Specifics.
> 
> I'm trying to avoid forcing a particular implementation on the skip
> file, since I can think of at least three different ways of doing it
> and I'm not sure which will be fastest...

Name them, and let others in on the sekret...


> | > When a news item is read, its name should be removed from the
> | > ``news-repoid.unread`` file. If a news client acts as an
> | > interactive reader rather than a gateway, it should then add the
> | > name to a ``news-repoid.read`` file in the same directory with the
> | > same file format.
> | 
> | implementation issue; you need locking on this.  If portage has to 
> | farm out to the users reader of choice, then it needs to lock the
> | file to keep readers/writers from screwing with each other.
> 
> We do? Marius told me it wasn't worth it.

I disagree.  It's mainly contingent upon $news_reader being spawned 
via --ask, although it *also* matters heavily if the user is flipping 
through their news while a sync in the background is running- if the 
utility updates on the way out, unless their is some advisorial 
locking involved, you've just lost all of the new synced unread news.


> | Flock preferably, since it's faster then resorting to hardlink 
> | trickery (yes this may be harder for shell crap, but so it goes since 
> | hardlink locking sucks).
> 
> What's wrong with mkdir?

Note the 'preferably'.  mkdir's reliant on all code cleaning up on the 
way out (that and sleep polling).  Flock and friends don't suffer that 
class of issues.


> | > News items can be removed (by removing the news file from the main
> | > tree) when they are no longer relevant, if they are made obsolete
> | > by a future news item or after a long period of time. This is the
> | > same as the method used for ``updates`` entries.
> | 
> | implementation issue; updating unread/skip crap so it doesn't bloat
> | is required, but time frame also matters (crap sync resulting in news 
> | getting removed, yet it still being valid, just missing from the
> | local tree).
> 
> Hrm. We can't take the same approach here as we do with expiring
> updates entries?

We expire updates?  If so, someone might want to look at the updates 
from 3 years back...


> | > There is an existing automated tool [#forums-glsa]_ for posting
> | > GLSAs to the forums. A similar tool can be used for these news
> | > items.
> | 
> | Pawned it off on someone, or something you'll be doing?
> 
> Hopefully the former. I have it on reasonably good authority that it
> won't take more than half an hour if I end up having to do it though...

Get cracking then (regardless if it's pawning or coding).

~harring

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] glep 42 (news) round six

[Marius Mauch]

Brian Harring wrote:
> On Sun, Dec 18, 2005 at 06:23:55AM +0000, Ciaran McCreesh wrote:
> 
>>On Sat, 17 Dec 2005 21:50:47 -0800 Brian Harring <ferringb@gentoo.org>
>>| You haven't stated how the 'package manager' will trigger the user's 
>>| reader of choice for these targets.  Should also extend this to allow 
>>| a way to disable any news notices, lest someone's cronjob get hung 
>>| displaying news (feature or not, it's needed).
>>
>>The same way the package manager handles updating config files: it
>>won't. It'll just tell the user that some news items need reading.
> 
> 
> And you'll personally handle all of the bug spam from feature requests 
> that --ask trigger $news_reader?
> 
> It's a logical extension, thus people will ask for it.

I don't think so.
How many people have asked for etc-update integration?

>>| implementation issue; you need locking on this.  If portage has to 
>>| farm out to the users reader of choice, then it needs to lock the
>>| file to keep readers/writers from screwing with each other.
>>
>>We do? Marius told me it wasn't worth it.
> 
> I disagree.  It's mainly contingent upon $news_reader being spawned 
> via --ask, although it *also* matters heavily if the user is flipping 
> through their news while a sync in the background is running- if the 
> utility updates on the way out, unless their is some advisorial 
> locking involved, you've just lost all of the new synced unread news.

To quote myself:
"Granted race conditions might be an issue (where the solution
complicates tools), but that's so minor I wouldn't really care about
it."
That I wouldn't care about it isn't a general recommendation to ignore 
the issue. Yes, for a perfect solution it is required, but do we aim for 
a perfect solution?

>>| > News items can be removed (by removing the news file from the main
>>| > tree) when they are no longer relevant, if they are made obsolete
>>| > by a future news item or after a long period of time. This is the
>>| > same as the method used for ``updates`` entries.
>>| 
>>| implementation issue; updating unread/skip crap so it doesn't bloat
>>| is required, but time frame also matters (crap sync resulting in news 
>>| getting removed, yet it still being valid, just missing from the
>>| local tree).
>>
>>Hrm. We can't take the same approach here as we do with expiring
>>updates entries?
> 
> We expire updates?  If so, someone might want to look at the updates 
> from 3 years back...

Yeah, mainly me dropping old files sometimes (happened two times so far 
IIRC), so not a general policy documented anywhere (just doing it when I 
get annoyed by portage re-applying ancient entries).

Another new issue nobody has mentioned yet:
The GLEP doesn't say anything about file permissions/ownership as in who 
  will/should be able to a) read news and b) modify the news-* files. 
Without thinking too much about it I'd say a) users in portage group and 
b) root.

Marius
-- 
gentoo-dev@gentoo.org mailing list

[gentoo-dev] Re: glep 42 (news) round six

[Duncan]

John Myers posted <200512172119.03178.electronerd@electronerdia.net>,
excerpted below,  on Sat, 17 Dec 2005 21:18:57 -0800:

> On Saturday 17 December 2005 20:57, Ciaran McCreesh wrote:
>> On Sat, 17 Dec 2005 20:50:43 -0800 John Myers
>>
>> <electronerd@electronerdia.net> wrote:
>> | GLEP 42 wrote:
>> | > * Before an ``emerge --ask <target>`` sequence
>> |
>> | What, exactly, does this mean? I'm guessing it means between the
>> | package list and the prompt, but it's not quite clear.
>>
>> I dunno, I refuse to use emerge --ask for religious reasons. Feel free
>> to rewrite that bullet point.
>>
> How about
> 
> * During ``emerge --ask``, the same as for ``emerge --pretend`` (i.e.
> after the package list)
> 
> ?

I had mentioned this earlier.

+1

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman in
http://www.linuxdevcenter.com/pub/a/linux/2004/12/22/rms_interview.html


-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] glep 42 (news) round six

[Brian Harring]

[-- Attachment #1: Type: text/plain, Size: 1920 bytes --]

On Sun, Dec 18, 2005 at 12:18:16PM +0200, Marius Mauch wrote:
> Brian Harring wrote:
> >On Sun, Dec 18, 2005 at 06:23:55AM +0000, Ciaran McCreesh wrote:
> >
> >>On Sat, 17 Dec 2005 21:50:47 -0800 Brian Harring <ferringb@gentoo.org>
> >>| You haven't stated how the 'package manager' will trigger the user's 
> >>| reader of choice for these targets.  Should also extend this to allow 
> >>| a way to disable any news notices, lest someone's cronjob get hung 
> >>| displaying news (feature or not, it's needed).
> >>
> >>The same way the package manager handles updating config files: it
> >>won't. It'll just tell the user that some news items need reading.
> >
> >
> >And you'll personally handle all of the bug spam from feature requests 
> >that --ask trigger $news_reader?
> >
> >It's a logical extension, thus people will ask for it.
> 
> I don't think so.
> How many people have asked for etc-update integration?

etc-update style notices are at exit of emerge.

This proposal has news item warnings prior to the merge actually 
occuring (no specification of sleep however).

Different scenario, thus etc-update isn't a good indicator.

> To quote myself:
> "Granted race conditions might be an issue (where the solution
> complicates tools), but that's so minor I wouldn't really care about
> it."
> That I wouldn't care about it isn't a general recommendation to ignore 
> the issue. Yes, for a perfect solution it is required, but do we aim for 
> a perfect solution?

We do if it potentially allows for 'critical' news to be unseen, at 
least imo.

> Another new issue nobody has mentioned yet:
> The GLEP doesn't say anything about file permissions/ownership as in who 
>  will/should be able to a) read news and b) modify the news-* files. 
> Without thinking too much about it I'd say a) users in portage group and 
> b) root.
*cough* good point. ;)
~harring

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] glep 42 (news) round six

[Philip Webb]

051218 Ciaran McCreesh wrote:
> * Change to yyyy-mm-dd for GLEP 45 compatibility.

Good to see Gentoo adopting correct international date format.

> You are encouraged to reply to this thread
> saying "I agree with ciaranm
> that repository IDs should not be allowed to contain spaces".

No problem at all there (smile): spaces in names are A Bad Thing for Unix,
as they conflict with the basic format of the command-line
& were introduced by M$ (Mac ?) to make things easier for idiots.
The proper procedure for Unix-type systems is to use an underline symbol.

> Ciaran McCreesh : Gentoo Developer (I can kill you with my brain)

Does your brain really contain that many viruses ... ?

-- 
========================,,============================================
SUPPORT     ___________//___,  Philip Webb : purslow@chass.utoronto.ca
ELECTRIC   /] [] [] [] [] []|  Centre for Urban & Community Studies
TRANSIT    `-O----------O---'  University of Toronto
-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] glep 42 (news) round six

[Patrick Lauer]

[-- Attachment #1: Type: text/plain, Size: 1081 bytes --]

On Sun, 2005-12-18 at 09:57 -0500, Philip Webb wrote:
> > You are encouraged to reply to this thread
> > saying "I agree with ciaranm
> > that repository IDs should not be allowed to contain spaces".
> 
> No problem at all there (smile): spaces in names are A Bad Thing for Unix,
> as they conflict with the basic format of the command-line
> & were introduced by M$ (Mac ?) to make things easier for idiots.
So you're saying long filenames were invented by Microsoft for Windows
95? ;-)
As long as programmers don't assume that filenames won't have spaces I
don't see the problem

> The proper procedure for Unix-type systems is to use an underline symbol.
Which "standard" says that, and how silly is that?
We're past the 80s, there's no reason to limit filenames to alphanumeric
(as I think with the same reasoning you'd forbid unicode ...)
> > Ciaran McCreesh : Gentoo Developer (I can kill you with my brain)
> Does your brain really contain that many viruses ... ?
Only because it runs Windows ;-)
-- 
Stand still, and let the rest of the universe move

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: [gentoo-dev] glep 42 (news) round six

[Ciaran McCreesh]

[-- Attachment #1: Type: text/plain, Size: 5079 bytes --]

On Sat, 17 Dec 2005 23:27:32 -0800 Brian Harring <ferringb@gentoo.org>
wrote:
| To head off the "don't make features that are easily screwed up",
| this isn't one of them- this is expecting code to behave correctly
| from the path standpoint.

Hrm, so will we be allowing spaces in package and category names too?

| > I don't want to introduce any signing requirements that we don't
| > have already. When signing for everything else becomes mandatory,
| > signing for news would be too. If I make this a 'must', someone
| > will ask me to specify how we're handling the Gentoo keyring...
| 
| Pawn the keyring off on others.  The issues isn't an established
| trust ring, it's required signing- yes, a trust ring makes things a
| helluva lot easier on the user front, but it's useless without a
| required signing policy.
| 
| We've already had this conversation also btw, in the 
| beginning of glep42 iirc.  Obviously I don't agree 
| with your reasoning "I'll do it when it's required I do it".  It's 
| useful now, it becomes massively more useful when a trust ring is 
| available.

Ok, how about I change it to "must", and add a note under Backwards
Compatibility along the lines of:

At the time of writing, there is no standardised mechanism for handling
GPG signatures in Gentoo. Until such a mechanism exists, GPG signing
cannot be considered mandatory.

| > | > ``Revision:``
| > | >     Initially 1. Incremented every time a non-trivial change is
| > | > made.  Changes which require a re-read of the news item should
| > | > instead use a new news item file. Mandatory.
| > |
| > | non-trivial changes that don't require a re-read sounds like a 
| > | contradiction.  Clarify, especially since portage will mark this
| > | as read _once_ and only once.
| > 
| > Hrm, word it as "Changes other than minor formatting tweaks", or
| > remove "non-trivial"?
| 
| It's not a wording thing, I'm pointing out sans spelling corrections 
| and trivial word mangling, any new info jammed in requires a new item 
| bump so readers can display the changes.
| 
| In light of that, wording above needs correction.

Ok, how's this?

``Revision:``
    Initially 1. Should be incremented every time a change is made to
the news item. Changes that require a re-read of the news item (for
example, most changes that are not spelling or formatting related)
should instead use a new news item. Mandatory.

| > | This isn't incredibly useful if ranged versions are ever
| > | introduced. Ammending the glep for that seems stupid, looser
| > | language might be wise.
| > 
| > What's the syntax for ranged versions? And how do they differ from
| > SLOT versions?
| 
| >=kde-base/kde-libs-3.0 <=kde-base/kde-libs-4.0 
|
| It's not syntax as much as a boolean and of atoms.

Hrm, ok. Wouldn't this resolve as true if you have kde-libs-2.0 and
kde-libs-5.0 installed (assuming SLOTted kde-libs)?

| > Once an hour would work fine. On the other hand, the merge is just
| > copying a few small files -- time-wise, it's less than generating
| > the cache for a couple of ebuilds.
| 
| More then a couple; this beast will bloat up to hundred or so files 
| I'd expect (remember translation serves as a multiplier).

Yup, but it's just a case of copying a few small text files.

| Any signed item would need to be verified also, although fortunately 
| this chunk can be done in parallel to regen run.

Hrm, is signing verification done for tree items?

| > | You haven't stated how the 'package manager' will trigger the
| > | user's reader of choice for these targets.  Should also extend
| > | this to allow a way to disable any news notices, lest someone's
| > | cronjob get hung displaying news (feature or not, it's needed).
| > 
| > The same way the package manager handles updating config files: it
| > won't. It'll just tell the user that some news items need reading.
| 
| And you'll personally handle all of the bug spam from feature
| requests that --ask trigger $news_reader?
| 
| It's a logical extension, thus people will ask for it.

What does emerge --ask do currently for config files?

| We expire updates?  If so, someone might want to look at the updates 
| from 3 years back...

Yes. Once a year, Seemant shows up and says "hey, does anyone ever
expire really really old updates entries?".

| > | > There is an existing automated tool [#forums-glsa]_ for posting
| > | > GLSAs to the forums. A similar tool can be used for these news
| > | > items.
| > | 
| > | Pawned it off on someone, or something you'll be doing?
| > 
| > Hopefully the former. I have it on reasonably good authority that it
| > won't take more than half an hour if I end up having to do it
| > though...
| 
| Get cracking then (regardless if it's pawning or coding).

Eh, that one can be left until the last minute.

-- 
Ciaran McCreesh : Gentoo Developer (I can kill you with my brain)
Mail            : ciaranm at gentoo.org
Web             : http://dev.gentoo.org/~ciaranm


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] glep 42 (news) round six

[Ciaran McCreesh]

[-- Attachment #1: Type: text/plain, Size: 672 bytes --]

On Sun, 18 Dec 2005 12:18:16 +0200 Marius Mauch <genone@gentoo.org>
wrote:
| Another new issue nobody has mentioned yet:
| The GLEP doesn't say anything about file permissions/ownership as in
| who will/should be able to a) read news and b) modify the news-*
| files. Without thinking too much about it I'd say a) users in portage
| group and b) root.

My prototype hack has the news files as 664, so even non-Portage-group
users can read things -- they just can't update the news-*.read files.

-- 
Ciaran McCreesh : Gentoo Developer (I can kill you with my brain)
Mail            : ciaranm at gentoo.org
Web             : http://dev.gentoo.org/~ciaranm


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] glep 42 (news) round six

[Brian Harring]

[-- Attachment #1: Type: text/plain, Size: 6371 bytes --]

On Sun, Dec 18, 2005 at 04:52:23PM +0000, Ciaran McCreesh wrote:
> On Sat, 17 Dec 2005 23:27:32 -0800 Brian Harring <ferringb@gentoo.org>
> wrote:
> | To head off the "don't make features that are easily screwed up",
> | this isn't one of them- this is expecting code to behave correctly
> | from the path standpoint.
> 
> Hrm, so will we be allowing spaces in package and category names too?

No, due to atoms in *depend being seperated by whitespace.  Wouldn't 
work without introducing escaping into it- beyond that the cat/pkg 
standard is in place already.

Your repo_id however, is not, thus it's mallable.

It's irrevelant either way- as I stated, your code *needs to be* space 
safe.  Existing repo label'ing code in saviour _already_ allows spaces 
(it's just a fricking name), dissallowing spaces due to a concern that 
someone might screw up is daft.

So... don't point at other things that are already set in stone and 
disallow spaces for their own reasons; state why it must be disallowed 
here, or merely state "I hate spaces".

Like I said a couple of emails back, block spaces in repo_id if you 
like, either way the code involved has to be space safe for paths, so 
it's an arbitrary restriction...

Dunno.  Either way, path handling *must* be space safe anyways- if you 
restrict repo_id to lack spaces, your choice, still going to allow external 
aliasing of the repo_id to have spaces.


> | > I don't want to introduce any signing requirements that we don't
> | > have already. When signing for everything else becomes mandatory,
> | > signing for news would be too. If I make this a 'must', someone
> | > will ask me to specify how we're handling the Gentoo keyring...
> | 
> | Pawn the keyring off on others.  The issues isn't an established
> | trust ring, it's required signing- yes, a trust ring makes things a
> | helluva lot easier on the user front, but it's useless without a
> | required signing policy.
> | 
> | We've already had this conversation also btw, in the 
> | beginning of glep42 iirc.  Obviously I don't agree 
> | with your reasoning "I'll do it when it's required I do it".  It's 
> | useful now, it becomes massively more useful when a trust ring is 
> | available.
> 
> Ok, how about I change it to "must", and add a note under Backwards
> Compatibility along the lines of:
> 
> At the time of writing, there is no standardised mechanism for handling
> GPG signatures in Gentoo. Until such a mechanism exists, GPG signing
> cannot be considered mandatory.

And provisions for going back and signing everything that was _not_ 
signed while you delaying waiting for a keyring?

That's why I'm pushing it.  Mandate it as required now, keyring down 
the line just makes it more useful.  Make it 'suggested' (which this 
is, you've changed nothing but words), you've left a mess that needs 
to be addressed when keyring comes about.

Same scenario as before, think forward- force it from the get go, less 
crap to deal with down the line.  Mandate it, no mess- just the 
pre-existing problem of getting a keyring collected.

Delay it, keyring + going back and trying to get folk to re-sign their 
releases.  That and any unsigned material released during that period 
cannot be verified, because we're waiting for a keyring.

See the gains?  Might be unpalatable, but it is the path of least work 
long term.


> Ok, how's this?
> 
> ``Revision:``
>     Initially 1. Should be incremented every time a change is made to
> the news item. Changes that require a re-read of the news item (for
> example, most changes that are not spelling or formatting related)
> should instead use a new news item. Mandatory.

Works, thank you.


> | > | This isn't incredibly useful if ranged versions are ever
> | > | introduced. Ammending the glep for that seems stupid, looser
> | > | language might be wise.
> | > 
> | > What's the syntax for ranged versions? And how do they differ from
> | > SLOT versions?
> | 
> | >=kde-base/kde-libs-3.0 <=kde-base/kde-libs-4.0 
> |
> | It's not syntax as much as a boolean and of atoms.
> 
> Hrm, ok. Wouldn't this resolve as true if you have kde-libs-2.0 and
> kde-libs-5.0 installed (assuming SLOTted kde-libs)?

Note I said boolean and.  Resolution of that string *should* result 
in 3-4 via portage processing of it; doesn't handle it perfectly, but 
the reason I brought it up is that via limiting it to a single atom, 
you block (if/when) ranged versions.


> | Any signed item would need to be verified also, although fortunately 
> | this chunk can be done in parallel to regen run.
> 
> Hrm, is signing verification done for tree items?

*If* a component was fully signed, verification prior to dissemination 
should occur.  Reliant on a keyring to automate it however, plus some 
voodoo to make as much as possible go in parallel (so as not to 
overrun the window).


> | > | You haven't stated how the 'package manager' will trigger the
> | > | user's reader of choice for these targets.  Should also extend
> | > | this to allow a way to disable any news notices, lest someone's
> | > | cronjob get hung displaying news (feature or not, it's needed).
> | > 
> | > The same way the package manager handles updating config files: it
> | > won't. It'll just tell the user that some news items need reading.
> | 
> | And you'll personally handle all of the bug spam from feature
> | requests that --ask trigger $news_reader?
> | 
> | It's a logical extension, thus people will ask for it.
> 
> What does emerge --ask do currently for config files?

see email responding to marius about why etc-update and friends aren't 
the best example...

Personally, I'm not much for triggering the news reader, but I'd 
expect folks will want a way to trigger the user's preferred reader.  
That's the crux of this question- you've bound this cruft into 
eselect, would assume there is a way to trigger the default reader 
(guessing at least).

If there is some way to trigger whatever the default news reader is, 
then the --ask cruft can be ignored; just asking about something as 
simple as a symlink, so if the request comes up (it will, imo), it's 
doable rather then trying to go and modify the existing clients to 
support it.

</ramble>

~harring

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] glep 42 (news) round six

[Paul de Vrieze]

[-- Attachment #1: Type: text/plain, Size: 444 bytes --]

On Sunday 18 December 2005 05:15, Ciaran McCreesh wrote:
> You are encouraged to reply to this thread saying "I agree with ciaranm
> that repository IDs should not be allowed to contain spaces".

I agree with ciaranm that repository IDs should not be allowed to contain 
spaces.

Paul

ps. Thanks for the perseverence on this glep. 

-- 
Paul de Vrieze
Gentoo Developer
Mail: pauldv@gentoo.org
Homepage: http://www.devrieze.net

[-- Attachment #2: Type: application/pgp-signature, Size: 198 bytes --]