[gentoo-dev] RFD: News item format 2.0

[gentoo-dev] RFD: News item format 2.0

[Ulrich Mueller]

[-- Attachment #1: Type: text/plain, Size: 2172 bytes --]

In its last meeting the council has accepted an extension of the
news item format which allows EAPI=5 style package dependency
specifications. This has triggered a discussion if this change is
backwards or forwards compatible, and what should be the new format's
version number [1]. Also it is not entirely clear if the term
"backwards-compatible" used in GLEP 42 correctly describes what was
originally intended [2].

In any case, both portage [3] and paludis [4] will have to be updated
for the new format because the change is not forwards-compatible.

Therefore, we could use the opportunity to add some other features.
So far, this includes:

1. Display-If-Installed: Allow EAPI=5 style package dependency
   specifications (see above).

2. Display-If-Profile: Allow wildcards like default-linux/* [5].

3. Title: Increase maximum length. In the past, devs repeatedly
   struggled with the current 44 character limit. (Can anyone
   enlighten me where this originates from? I cannot find anything in
   the discussions around the time when GLEP 42 was submitted.)
   The eselect news reader could still display a 51 character title
   in one line for the "list" and "read" commands, on an 80 character
   wide terminal.
   I suggest we round this down to a maximum length of 50 characters,
   which (together with the 72 character limit for the body) would
   nicely agree with the limits recommended for git commit messages.

4. Content-Type: Only one value is allowed for this header, namely
   text/plain. We might as well drop it, because any changes there
   will require an increment of the News-Item-Format.

If these changes find agreement, I would prepare a new GLEP for news
item format 2.0.

Ulrich


[1] https://bugs.gentoo.org/show_bug.cgi?id=568068#c4
[2] https://archives.gentoo.org/gentoo-dev/message/d30de011db9067ae3cc298de2b4ee1b2
[3] https://gitweb.gentoo.org/proj/portage.git/tree/pym/portage/news.py?id=7c94014a32d173ae61919b762140ac1c32d3b522#n273
[4] http://git.exherbo.org/paludis/paludis.git/tree/paludis/repositories/e/e_repository_news.cc?id=1684b446715907515359cd310c1e7bd93bad5a2e#n326
[5] https://bugs.gentoo.org/show_bug.cgi?id=290038

[-- Attachment #2: Type: application/pgp-signature, Size: 490 bytes --]

Re: [gentoo-dev] RFD: News item format 2.0

[Daniel Campbell (zlg)]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA512

I don't see why backwards compatibility would be a problem. The older news format spec supports fewer features, so the new spec should be much like newer EAPIs and 'just work'.

I'm in favor of all the changes you laid out. A good number of us are already familiar with git-style limitations, and they're rather sane to begin with. I could see issues with bigger changes fitting into 50 characters, but I guess that's what creative wording is for. :)

It's odd that news items even have the Content-Type attribute. Before removing it, are there any other formats we could feasibly want in the future? I can't think of any formats we'd want to use, but maybe someone else has an idea.

In general, though, I'm in favor of the changes if it makes life easier for those writing news. I don't write news items right now, but I may end up doing it in the future.

Sorry for top-posting. I didn't see a way to do proper replies in K-9.

On January 12, 2016 10:13:39 AM PST, Ulrich Mueller <ulm@gentoo.org> wrote:
>In its last meeting the council has accepted an extension of the
>news item format which allows EAPI=5 style package dependency
>specifications. This has triggered a discussion if this change is
>backwards or forwards compatible, and what should be the new format's
>version number [1]. Also it is not entirely clear if the term
>"backwards-compatible" used in GLEP 42 correctly describes what was
>originally intended [2].
>
>In any case, both portage [3] and paludis [4] will have to be updated
>for the new format because the change is not forwards-compatible.
>
>Therefore, we could use the opportunity to add some other features.
>So far, this includes:
>
>1. Display-If-Installed: Allow EAPI=5 style package dependency
>   specifications (see above).
>
>2. Display-If-Profile: Allow wildcards like default-linux/* [5].
>
>3. Title: Increase maximum length. In the past, devs repeatedly
>   struggled with the current 44 character limit. (Can anyone
>   enlighten me where this originates from? I cannot find anything in
>   the discussions around the time when GLEP 42 was submitted.)
>   The eselect news reader could still display a 51 character title
>   in one line for the "list" and "read" commands, on an 80 character
>   wide terminal.
>   I suggest we round this down to a maximum length of 50 characters,
>   which (together with the 72 character limit for the body) would
>   nicely agree with the limits recommended for git commit messages.
>
>4. Content-Type: Only one value is allowed for this header, namely
>   text/plain. We might as well drop it, because any changes there
>   will require an increment of the News-Item-Format.
>
>If these changes find agreement, I would prepare a new GLEP for news
>item format 2.0.
>
>Ulrich
>
>
>[1] https://bugs.gentoo.org/show_bug.cgi?id=568068#c4
>[2]
>https://archives.gentoo.org/gentoo-dev/message/d30de011db9067ae3cc298de2b4ee1b2
>[3]
>https://gitweb.gentoo.org/proj/portage.git/tree/pym/portage/news.py?id=7c94014a32d173ae61919b762140ac1c32d3b522#n273
>[4]
>http://git.exherbo.org/paludis/paludis.git/tree/paludis/repositories/e/e_repository_news.cc?id=1684b446715907515359cd310c1e7bd93bad5a2e#n326
>[5] https://bugs.gentoo.org/show_bug.cgi?id=290038

- --
Sent from my Android device with K-9 Mail. Please excuse my brevity.
-----BEGIN PGP SIGNATURE-----

iQJRBAEBCgA7NBxEYW5pZWwgQ2FtcGJlbGwgKEdlbnRvbyBEZXZlbG9wZXIpIDx6
bGdAZ2VudG9vLm9yZz4FAlaVkQkACgkQASQOlFA54XAYuQ/9F0qt4BaPTan+rMBd
MkbjA1A4EeaLgGT0BAlv5oKhfR5VC4fNoWcfCB+Z11Bkyi+8DRGLWoEqF1KvFKsb
CLu8a8Q/+T34JuvKznCJrHfnkFYwARVXpOVX70t3Sr50BSpe5lvmQ4PQ8PpOFTnJ
O4bo9GjxKTujVvu1LxYSv3CLJ6AV9HANsl5rkBQ+TKi4qrwqBTWK4VGNTCon/DFt
EUTRuz7TXlpE6quMTTk7Wx8yldRgC7mL2SwYsH0KosrBaMTv5yVWipZMdEP6oPRp
ovYhPNgPYPrct8DuiOOXvNJms8Ll5oS/m07w5MUr7KzWAoWjFluQAVR+HOACfXuk
7YiSk9FbH+a3t6aEGNrEX7VRcG8A2UG6nDsc5GNXLc+ObPvfsykXZs6vchfC8J+j
wJp8R/dXecAyw9HmjQ6cx3N3lw65YG+3I1cK883GuvGGb7P9BOeWuloouhqEfOc5
OaptMrWWJM9T8FNzgZVXc8tJmrCexw1HFKUfjhcJinIffHfRoeIjmSJcTtKliW/k
0VHtmhjr4OkBr+wcjf5uZEAstq/4/Jp1ArS3URaXtDEBuY0xKaR/WLMPtcuu4X1K
fFkah8qS/jR8qqE9KecULE25c8C47im6EJGk6Wgzb/8MNx3J4iZ8P0PHS+kLEZFq
uyrSyEwwvx2ES92sDb+EaXW0B08=
=7A0/
-----END PGP SIGNATURE-----

Re: [gentoo-dev] RFD: News item format 2.0

[Rich Freeman]

On Tue, Jan 12, 2016 at 1:13 PM, Ulrich Mueller <ulm@gentoo.org> wrote:
>
> Therefore, we could use the opportunity to add some other features.
> So far, this includes:
>

I don't have a solution for this in mind, but I do see a problem that
could use a solution with our current approach to news in our stable
and unstable branches.

Using the current criteria we have for displaying news items it is
very hard to prepare a news item that is displayed at an appropriate
time for both stable and unstable users of a package.  Either the
unstable users don't get the news item at all, or everybody gets it at
the same time.  Then stable users end up getting confused and
eventually forgetting about the notice, only to be hit with the update
after a significant time lag (it could be a year or more in some
cases) without any advance news.

One way to do this (and I'm certainly open to others) is a
Display-If-Installable header which takes a keyword string and an atom
(typically a specific PV).  The package manager would determine if a
package with that keyword string and PV would be accepted or not based
on the user's configuration, and if so display the news.  So, if the
string "~amd64 ~x86", =www-client/apache-2.4.19 were passed, users
running ~arch would generally get the message, as would a user running
stable but with <www-client/apache-2.5 in their package.keywords, but
not a user running stable with <www-client/apache-2.4.19 in their
package.keywords.  If a user already was running
www-client/apache-2.4.21 the news would not display since the package
manager wouldn't attempt to install 2.4.19 if it showed up in the tree
(I'm open to argument as to whether it should show up if 2.4.19 is
already installed as there are pros and cons here).  Note that this is
all hypothetical and the news should be triggered even if 2.4.19 isn't
in the tree yet, or if it is in the tree with a different set of
keywords.

Now, this would still potentially require putting the same news item
into the tree multiple times to trigger it for the unstable/stable
users, but the key would be that the right users get the message when
this happens.  Since the details of the news and what PV it applies to
might change between unstable/stable this is probably appropriate
anyway.  Likewise if three archs are going stable now and two more a
year from now the news could be re-triggered just for those archs.

The main downside to this I see is complexity - you can't just set up
one news item and have it do the right thing.  The main advantage I
see to this is correctness - it handles both the case where keywords
are set in make.conf and package.keywords, because it provides enough
of a hint to the package manager about what is about to be introduced
into the tree.

There could easily be a better solution for this.  However, I think it
is a problem worth solving.

-- 
Rich

Re: [gentoo-dev] RFD: News item format 2.0

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 4349 bytes --]

On Thu, 14 Jan 2016 07:28:43 -0500
Rich Freeman <rich0@gentoo.org> wrote:

> On Tue, Jan 12, 2016 at 1:13 PM, Ulrich Mueller <ulm@gentoo.org> wrote:
> >
> > Therefore, we could use the opportunity to add some other features.
> > So far, this includes:
> >  
> 
> I don't have a solution for this in mind, but I do see a problem that
> could use a solution with our current approach to news in our stable
> and unstable branches.
> 
> Using the current criteria we have for displaying news items it is
> very hard to prepare a news item that is displayed at an appropriate
> time for both stable and unstable users of a package.  Either the
> unstable users don't get the news item at all, or everybody gets it at
> the same time.  Then stable users end up getting confused and
> eventually forgetting about the notice, only to be hit with the update
> after a significant time lag (it could be a year or more in some
> cases) without any advance news.

I was going to speak about this, and you beat me up to it...

> One way to do this (and I'm certainly open to others) is a
> Display-If-Installable header which takes a keyword string and an atom
> (typically a specific PV).  The package manager would determine if a
> package with that keyword string and PV would be accepted or not based
> on the user's configuration, and if so display the news.  So, if the
> string "~amd64 ~x86", =www-client/apache-2.4.19 were passed, users
> running ~arch would generally get the message, as would a user running
> stable but with <www-client/apache-2.5 in their package.keywords, but
> not a user running stable with <www-client/apache-2.4.19 in their
> package.keywords.  If a user already was running
> www-client/apache-2.4.21 the news would not display since the package
> manager wouldn't attempt to install 2.4.19 if it showed up in the tree
> (I'm open to argument as to whether it should show up if 2.4.19 is
> already installed as there are pros and cons here).  Note that this is
> all hypothetical and the news should be triggered even if 2.4.19 isn't
> in the tree yet, or if it is in the tree with a different set of
> keywords.
> 
> Now, this would still potentially require putting the same news item
> into the tree multiple times to trigger it for the unstable/stable
> users, but the key would be that the right users get the message when
> this happens.  Since the details of the news and what PV it applies to
> might change between unstable/stable this is probably appropriate
> anyway.  Likewise if three archs are going stable now and two more a
> year from now the news could be re-triggered just for those archs.
> 
> The main downside to this I see is complexity - you can't just set up
> one news item and have it do the right thing.  The main advantage I
> see to this is correctness - it handles both the case where keywords
> are set in make.conf and package.keywords, because it provides enough
> of a hint to the package manager about what is about to be introduced
> into the tree.
> 
> There could easily be a better solution for this.  However, I think it
> is a problem worth solving.

...and I think you've found a pretty good solution to the problem.
However, I get lost in your wall of text and I don't really see
the problems you see.

Based on your idea, this is how I'd do it:

1. 'Display-If-Visible' that enables news items if given atom is
visible for PM (i.e. in repo, with right keywords and not masked).
I would avoid using 'Installable' as that could get confusing wrt
conflicts and so on.

2. Combine that with 'Display-If-Installed' -- and we should be able to
get somewhat sane behavior.

For example, for foo-1.0 to -2.0 upgrade notes, we'd do:

  Display-if-installed: <foo-2.0
  Display-if-visible: >=foo-2.0

which would basically mean that all people having old foo installed
would get the item as soon as they foo-2.0 becomes visible to them.
But people installing straight 2.0 without previous versions don't get
the item.

We could also do:

  Display-if-installed: foo
  Display-if-visible: >=foo-2.0

which would cause the item to be displayed to both people who have old
foo installed and can upgrade, or installed 2.0 directly.

-- 
Best regards,
Michał Górny
<http://dev.gentoo.org/~mgorny/>

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 949 bytes --]

Re: [gentoo-dev] RFD: News item format 2.0

[Ulrich Mueller]

[-- Attachment #1: Type: text/plain, Size: 1296 bytes --]

>>>>> On Thu, 14 Jan 2016, Michał Górny wrote:

> On Thu, 14 Jan 2016 07:28:43 -0500
> Rich Freeman <rich0@gentoo.org> wrote:

>> One way to do this (and I'm certainly open to others) is a
>> Display-If-Installable header which takes a keyword string and an
>> atom (typically a specific PV).  The package manager would
>> determine if a package with that keyword string and PV would be
>> accepted or not based on the user's configuration, and if so
>> display the news.

> Based on your idea, this is how I'd do it:

> 1. 'Display-If-Visible' that enables news items if given atom is
> visible for PM (i.e. in repo, with right keywords and not masked).
> I would avoid using 'Installable' as that could get confusing wrt
> conflicts and so on.

I had omitted the Display-If-Visible header (which has been discussed
since 2007 at least) deliberately, because we want to get format 2.0
done in a timely manner.

The problem with any visibility filtering is that visibility depends
on user configuration [1], and I don't know what changes in the
package manager would be necessary to make this work correctly and
efficiently. For example, how does portage's --autounmask-write option
interact with it?

Ulrich

[1] https://bugs.gentoo.org/show_bug.cgi?id=290038#c9

[-- Attachment #2: Type: application/pgp-signature, Size: 490 bytes --]

Re: [gentoo-dev] RFD: News item format 2.0

[Rich Freeman]

On Thu, Jan 14, 2016 at 4:30 PM, Michał Górny <mgorny@gentoo.org> wrote:
> For example, for foo-1.0 to -2.0 upgrade notes, we'd do:
>
>   Display-if-installed: <foo-2.0
>   Display-if-visible: >=foo-2.0
>
> which would basically mean that all people having old foo installed
> would get the item as soon as they foo-2.0 becomes visible to them.
> But people installing straight 2.0 without previous versions don't get
> the item.

This works, but does require 2.0 to be in the tree already.  It is
obviously simpler as a result.

I think it still makes sense - users just need to check news before
they update.  We couldn't put out the news items a few days in advance
this way.

So, ++ from me.

-- 
Rich

Re: [gentoo-dev] RFD: News item format 2.0

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 2191 bytes --]

On Fri, 15 Jan 2016 00:00:02 +0100
Ulrich Mueller <ulm@gentoo.org> wrote:

> >>>>> On Thu, 14 Jan 2016, Michał Górny wrote:  
> 
> > On Thu, 14 Jan 2016 07:28:43 -0500
> > Rich Freeman <rich0@gentoo.org> wrote:  
> 
> >> One way to do this (and I'm certainly open to others) is a
> >> Display-If-Installable header which takes a keyword string and an
> >> atom (typically a specific PV).  The package manager would
> >> determine if a package with that keyword string and PV would be
> >> accepted or not based on the user's configuration, and if so
> >> display the news.  
> 
> > Based on your idea, this is how I'd do it:  
> 
> > 1. 'Display-If-Visible' that enables news items if given atom is
> > visible for PM (i.e. in repo, with right keywords and not masked).
> > I would avoid using 'Installable' as that could get confusing wrt
> > conflicts and so on.  
> 
> I had omitted the Display-If-Visible header (which has been discussed
> since 2007 at least) deliberately, because we want to get format 2.0
> done in a timely manner.

That doesn't really make sense to me. There is no real urgency in
getting the new format right now, and omitting the most important issue
is not really the solution here.

> The problem with any visibility filtering is that visibility depends
> on user configuration [1], and I don't know what changes in the
> package manager would be necessary to make this work correctly and
> efficiently. For example, how does portage's --autounmask-write option
> interact with it?

Leave solving this to Portage developers. Autounmasking is not
something you do often on stable systems, and we have better QA to
avoid issues that could require it.

What's most important right now is to have Portage process news items
after 'emerge --sync' (or any other emerge operation), check visibility
of packages (with current configuration) and enable news items that
apply.

We could also process changed news files on emerge start, and warn
verbosely that new news items start applying.

> [1] https://bugs.gentoo.org/show_bug.cgi?id=290038#c9

-- 
Best regards,
Michał Górny
<http://dev.gentoo.org/~mgorny/>

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 949 bytes --]

Re: [gentoo-dev] RFD: News item format 2.0

[Ulrich Mueller]

[-- Attachment #1: Type: text/plain, Size: 1456 bytes --]

>>>>> On Sat, 16 Jan 2016, Michał Górny wrote:

> That doesn't really make sense to me. There is no real urgency in
> getting the new format right now, and omitting the most important
> issue is not really the solution here.

The idea was to introduce EAPI 5 dependencies for the
Display-If-Installed header. Anything else is an added bonus at this
stage, and can be included if it is trivial to implement.

Nothing prevents us from creating a format 2.1 or 3.0 later for any
features that will take more time. Or, if we are thinking in long time
scales, we could even incorporate the news item spec into PMS with the
next EAPI (and replace the format number by the EAPI).

>> The problem with any visibility filtering is that visibility
>> depends on user configuration [1], and I don't know what changes in
>> the package manager would be necessary to make this work correctly
>> and efficiently. For example, how does portage's --autounmask-write
>> option interact with it?

> Leave solving this to Portage developers. Autounmasking is not
> something you do often on stable systems, and we have better QA to
> avoid issues that could require it.

Yeah, if I wasn't sure about Display-If-Visible, then this statement
would finally convince me that we should skip that feature, for the
time being. That something does not happen often is no excuse for not
having an implementation that can handle such rare cases.

Ulrich

[-- Attachment #2: Type: application/pgp-signature, Size: 490 bytes --]

Re: [gentoo-dev] RFD: News item format 2.0

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 1106 bytes --]

On Sat, 16 Jan 2016 16:03:55 +0100
Ulrich Mueller <ulm@gentoo.org> wrote:

> >> The problem with any visibility filtering is that visibility
> >> depends on user configuration [1], and I don't know what changes in
> >> the package manager would be necessary to make this work correctly
> >> and efficiently. For example, how does portage's --autounmask-write
> >> option interact with it?  
> 
> > Leave solving this to Portage developers. Autounmasking is not
> > something you do often on stable systems, and we have better QA to
> > avoid issues that could require it.  
> 
> Yeah, if I wasn't sure about Display-If-Visible, then this statement
> would finally convince me that we should skip that feature, for the
> time being. That something does not happen often is no excuse for not
> having an implementation that can handle such rare cases.

Implementation != specification. I'm not saying the implementation
shouldn't handle them.

It's also nice how you stripped the more important part of my answer.

-- 
Best regards,
Michał Górny
<http://dev.gentoo.org/~mgorny/>

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 949 bytes --]