[gentoo-dev] RFC: security updates only? (security-1.0.ebuild)

[gentoo-dev] RFC: security updates only? (security-1.0.ebuild)

[Karsten Schulz]

Hi all, 

At bugs.gentoo.org I attached an tarball with my version of an ebuild, which 
automatically will install *only* those security updates, which are relevant 
to your system (see <http://bugs.gentoo.org/show_bug.cgi?id=5835>).

The trick is the following code in the ebuild:
DEPEND="
$(has_version 'nfs-utils' && echo '>=nfs-utils-1.0.4')
$(has_version 'gnupg' && echo '>=gnupg-1.2.2-r1')
"
this will create dependencies on the fly: Only if you have nfs-utils 
installed, the new version with the bugfix will appear as a dependency. Same 
with gnupg. 
There is no need for new 'emerge' options or KEYWORDS.

For his own system, the admin has to type in only:
emerge rsync
emerge security

or, if he is interested in this stuff:
emerge rsync
emerge -p security
and followed by 'emerge security', if necessary.

The only drawback at the moment is, that there is no automated way to generate 
the value of the DEPEND variable by the GLSAs. Maybe here can Marius' XML 
format help. With GLSAs in such a format, a simple script can generate the 
security-ebuilds, the postings at forums.gentoo.org and the mails in 
gentoo-announce! I would love to hear from some core developers and others 
what you think (is it critical to use an ebuild, which does not compile and 
install its own sourcecode, because it has none?).

I know, that this can (and must) be improved in the future (more information, 
references to advisories, signing the ebuild, a.s.o.), but at the moment, 
this way seems to me to be an easy one, isn't it?

Comments?

Karsten


--
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] RFC: security updates only? (security-1.0.ebuild)

[Paul de Vrieze]

[-- Attachment #1: signed data --]
[-- Type: text/plain, Size: 2099 bytes --]

On Saturday 16 August 2003 11:46, Karsten Schulz wrote:
> Hi all,
>
> At bugs.gentoo.org I attached an tarball with my version of an ebuild,
> which automatically will install *only* those security updates, which are
> relevant to your system (see
> <http://bugs.gentoo.org/show_bug.cgi?id=5835>).
>
> The trick is the following code in the ebuild:
> DEPEND="
> $(has_version 'nfs-utils' && echo '>=nfs-utils-1.0.4')
> $(has_version 'gnupg' && echo '>=gnupg-1.2.2-r1')
> "
> this will create dependencies on the fly: Only if you have nfs-utils
> installed, the new version with the bugfix will appear as a dependency.
> Same with gnupg.
> There is no need for new 'emerge' options or KEYWORDS.
>
> For his own system, the admin has to type in only:
> emerge rsync
> emerge security
>
> or, if he is interested in this stuff:
> emerge rsync
> emerge -p security
> and followed by 'emerge security', if necessary.
>
> The only drawback at the moment is, that there is no automated way to
> generate the value of the DEPEND variable by the GLSAs. Maybe here can
> Marius' XML format help. With GLSAs in such a format, a simple script can
> generate the security-ebuilds, the postings at forums.gentoo.org and the
> mails in gentoo-announce! I would love to hear from some core developers
> and others what you think (is it critical to use an ebuild, which does not
> compile and install its own sourcecode, because it has none?).
>
> I know, that this can (and must) be improved in the future (more
> information, references to advisories, signing the ebuild, a.s.o.), but at
> the moment, this way seems to me to be an easy one, isn't it?
>
> Comments?

Unfortunately dynamic dependencies in this style are not possible. I also 
believe that a more fundamental solution should be implemented. Probably in 
emerge itself. It should not really be hard. It should basically do what you 
propose from a dynamically generated list of insecure ebuilds.

Paul

-- 
Paul de Vrieze
Gentoo Developer
Mail: pauldv@gentoo.org
Homepage: http://www.devrieze.net

[-- Attachment #2: signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] RFC: security updates only? (security-1.0.ebuild)

[Karsten Schulz]

Am Samstag, 16. August 2003 12:20 Paul de Vrieze wrote:
> Unfortunately dynamic dependencies in this style are not possible.

Why not? Please, could you explain me what will break?
(or show me the lines in ebuild.sh were things will break?)

On my machine the generation of the dependencies work (Portage 2.0.48-r5 
(default-x86-1.4, gcc-3.2.3, glibc-2.3.2-r1)). 
I tried it with my security-1.0.ebuild.
I did some quick tests with several package combinations and the only thing I 
noticed was, that the compile and install stages cause some heavy cpu cycles, 
which I can not explain. (maybe the empty SRC_URI var, but why?)

> I also
> believe that a more fundamental solution should be implemented. Probably in
> emerge itself. 

I agree on that for future developments. But when will that be implemented? I 
tried a simple solution for now. I really like to see an 'security' argument 
like 'system' or 'world' in Portage 2.x.
OTOH I wish to keep server administration simpler now, because easier security 
administration is nothing, which must wait for tomorrow (imho).

>It should not really be hard. It should basically do what
> you propose from a dynamically generated list of insecure ebuilds.

ACK.

Karsten


--
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] RFC: security updates only? (security-1.0.ebuild)

[Karsten Schulz]

Am Samstag, 16. August 2003 13:03 Karsten Schulz wrote:
> I tried a simple solution for now. I really like to see an 'security'
> argument like 'system' or 'world' in Portage 2.x.

argh, I meant '...in Portage 2.1.x ...' - sigh

Karsten


--
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] RFC: security updates only? (security-1.0.ebuild)

[Paul de Vrieze]

[-- Attachment #1: signed data --]
[-- Type: text/plain, Size: 902 bytes --]

On Saturday 16 August 2003 13:03, Karsten Schulz wrote:
> Am Samstag, 16. August 2003 12:20 Paul de Vrieze wrote:
> > Unfortunately dynamic dependencies in this style are not possible.
>
> Why not? Please, could you explain me what will break?
> (or show me the lines in ebuild.sh were things will break?)

Well, for a user it works, but it does not work in terms of providing a cache. 
Normally a cache is built. One thing needed for that is that we can parse 
dependencies. Your dependencies are created in a bash way. That means that a 
cache can not be created because as far as the caching system is concerned 
your cache is dynamic. And worse, there would be no way of detecting this 
dynamic nature. In short putting such an ebuild in the portage tree would 
create hell.

Paul

-- 
Paul de Vrieze
Gentoo Developer
Mail: pauldv@gentoo.org
Homepage: http://www.devrieze.net

[-- Attachment #2: signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] RFC: security updates only? (security-1.0.ebuild)

[Klavs Klavsen]

[-- Attachment #1: Type: text/plain, Size: 1071 bytes --]

How about starting with putting the GLSA's in the tree?

That would be step 1. 

Then an emerge sync would give you new security information too.
or perhaps an glsa-ebuild?

After this, I agree it would be best to add a --security option.

IMHO this should work, so emerge -u world --security (or -S - or is that
too close to -s?) would look at all the upgrades suggested by the world
- and then check against the glsa's and see if any of them have are
mentioned here. The glsa's should be named after the package version
that fixes this issue, that way it would be easy to check and upgrade if
the currently installed is older.

Better solutions are welcome - but are they as easy to implement as
this?
-- 
Regards,
Klavs Klavsen, GSEC - kl@vsen.dk - http://www.vsen.dk
PGP: 7E063C62/2873 188C 968E 600D D8F8  B8DA 3D3A 0B79 7E06 3C62
See my new CMS Hosting Service at http://www.VirkPaaNettet.dk

Working with Unix is like wrestling a worthy opponent. 
Working with windows is like attacking a small whining child 
who is carrying a .38.				

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] RFC: security updates only? (security-1.0.ebuild)

[Karsten Schulz]

Am Samstag, 16. August 2003 16:31 Paul de Vrieze wrote:
> Well, for a user it works, but it does not work in terms of providing a
> cache. Normally a cache is built. One thing needed for that is that we can
> parse dependencies.

Oh, I see. As far as I understand now, it works in this way:
emerge rsyncs and then generates the cache in /var/vache/edb/dep.
At this time, no ebuild is executed as a script and therefore the dependencies 
cannot be calculated. That I did not know, thanks for enlighten me, Paul.

So, if I understand things right, it has to go this way:
On the rsync-Servers there should be a file containing GLSAs. This file 
contains for example the following lines:
VERSION	1.0
nfs-utils	>=nfs-utils-1.0.4
gnupg	>=gnupg-1.2.2-r1

*After* rsyncing, but *before* building the cache, emerge should create a 
statical version of app-admin/security/security-VERSION.ebuild with the 
dependencies corresponding to the current system and the correct version.
After creating this ebuild, the cache could be build and an emerge -p security 
will show the needed updates. That seems to me a few lines of python code and 
an introduction of the new GLSA file.

How is that?

Karsten





--
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] RFC: security updates only? (security-1.0.ebuild)

[Karsten Schulz]

Am Samstag, 16. August 2003 17:56 Klavs Klavsen wrote:
> How about starting with putting the GLSA's in the tree?
> That would be step 1.

ack. Let us talk about the format! For future enhancements I suggest using an 
xml format (Marius?).

> After this, I agree it would be best to add a --security option.
>
> IMHO this should work, so emerge -u world --security (or -S - or is that
> too close to -s?) would look at all the upgrades suggested by the world
> - and then check against the glsa's and see if any of them have are
> mentioned here. The glsa's should be named after the package version
> that fixes this issue, that way it would be easy to check and upgrade if
> the currently installed is older.

after I learned from Paul, that I cannot use dynamically generated 
dependencies, I would second this suggestion.

Karsten


--
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] RFC: security updates only? (security-1.0.ebuild)

[Paul de Vrieze]

[-- Attachment #1: signed data --]
[-- Type: text/plain, Size: 866 bytes --]

On Saturday 16 August 2003 17:59, Karsten Schulz wrote:
> *After* rsyncing, but *before* building the cache, emerge should create a
> statical version of app-admin/security/security-VERSION.ebuild with the
> dependencies corresponding to the current system and the correct version.
> After creating this ebuild, the cache could be build and an emerge -p
> security will show the needed updates. That seems to me a few lines of
> python code and an introduction of the new GLSA file.
>
> How is that?

Why not just having a utility (or integrate it into emerge) that scans all 
installed packages, compares them with the list and tries to update those 
that are with an ebuild in the same slot. If there is no such ebuild it 
issues a warning.

Paul

-- 
Paul de Vrieze
Gentoo Developer
Mail: pauldv@gentoo.org
Homepage: http://www.devrieze.net

[-- Attachment #2: signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] RFC: security updates only? (security-1.0.ebuild)

[Marius Mauch]

On Sat, 16 Aug 2003 19:11:36 +0200
Paul de Vrieze <pauldv@gentoo.org> wrote:

> On Saturday 16 August 2003 17:59, Karsten Schulz wrote:
> > *After* rsyncing, but *before* building the cache, emerge should
> > create a statical version of
> > app-admin/security/security-VERSION.ebuild with the dependencies
> > corresponding to the current system and the correct version. After
> > creating this ebuild, the cache could be build and an emerge -p
> > security will show the needed updates. That seems to me a few lines
> > of python code and an introduction of the new GLSA file.
> >
> > How is that?
> 
> Why not just having a utility (or integrate it into emerge) that scans
> all installed packages, compares them with the list and tries to
> update those that are with an ebuild in the same slot. If there is no
> such ebuild it issues a warning.

I don't know how many people actually tested my stuff
(http://gentoo.devel-net.org/glsa/ in case you haven't seen it) so I'll
explain how it works:

glsa-check is a tool to read a directory (could also be on a remote
server) of GLSA files in XML format and can do the following actions on
it:- print a list of all unchecked GLSAs, using a checkfile that is
updated on other actions
- test if the current system is affected by a GLSA
- print a plaintext version of a GLSA (similar to the current plaintext
format)
- tell the user all necessary steps to apply a GLSA to the system
- apply a GLSA automatically (not implemented yet, I just need to find
the best way to do it) and add it to the checkfile
- "inject" the GLSA into the checkfile in case the testing was wrong or
the user fixed it manually

The glsa-check.py is just a frontend for most parts, the real work is
done in the glsa.py class, integration in portage at a later stage
should be rather easy. I'll check with carpaski once we get to this.

If someone doesn't understand why I chose XML over the current plaintext
format, the reasons are
- it's IMO easier to parse in python, no custom parser needed
- allows some enhancements as specifying multiple affected/fixed
versions or implement multi-language support (not in the current DTD)
- converting to other formats (e.g. html for online viewing) is easy
with XSL

So, what is needed to get this working?
- put the GLSA in a standard location for automated retrieval, I suggest
both in the rsync tree and somewhere on www.gentoo.org
- convert all GLSA into XML format, this could be done by the authors or
a converter script
- some code additions, nothing major
- a lot of testing :)


Ok, so far the information on the current state, I'm open to any
questions/suggestions/comments. The code is only a prototype, easy to
change at the moment. I already discussed this with people on -security
and -server, so far people seem to like this, but I need the views of
some devs now.

Marius

--
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] RFC: security updates only? (security-1.0.ebuild)

[Karsten Schulz]

Am Samstag, 16. August 2003 21:38 Marius Mauch wrote:
> I don't know how many people actually tested my stuff
> (http://gentoo.devel-net.org/glsa/ in case you haven't seen it) so I'll
> explain how it works:

I did :-)

> So, what is needed to get this working?
> - put the GLSA in a standard location for automated retrieval, I suggest
> both in the rsync tree and somewhere on www.gentoo.org
> - convert all GLSA into XML format, this could be done by the authors or
> a converter script
> - some code additions, nothing major
> - a lot of testing :)

I would like to discuss some aspects of your software. I like it, and I think, 
it could make admin's life easier. 

First, for what reason do you maintain /var/cache/edb/glsa? You could ask 
portage to see, wether a special version of a package is installed in the 
system or is not. To store applied GLSAs in this file seems redundant to me, 
isn't it?

Second, the 'glsa-' prefix of the GLSA filenames seems unnecessary to me. The 
GLSA information is stored in the directory /usr/portage/glsa/. Why not name 
the file with its unique GLSA identifier (200307-07 or perhaps 
200307-07.xml), shouldn't that be enough?

In your 'readme' file you wrote, that there is no installation needed. 
Nevertheless I would provide a little install-script, which checks the 
presence of the needed files and directories (/etc/portage/, 
/usr/portage/glsa/  and /var/cache/edb/glsa). Without this dirs and the file, 
glsa.py fails.

Please feel free to contact me for further testing!

have fun!
Karsten



--
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] RFC: security updates only? (security-1.0.ebuild)

[Marius Mauch]

On Mon, 18 Aug 2003 12:29:59 +0200
Karsten Schulz <kaschu@t800.ping.de> wrote:

> Am Samstag, 16. August 2003 21:38 Marius Mauch wrote:
> > So, what is needed to get this working?
> > - put the GLSA in a standard location for automated retrieval, I
> > suggest both in the rsync tree and somewhere on www.gentoo.org
> > - convert all GLSA into XML format, this could be done by the
> > authors or a converter script
> > - some code additions, nothing major
> > - a lot of testing :)
> 
> I would like to discuss some aspects of your software. I like it, and
> I think, it could make admin's life easier. 

I'm currently writing a GLEP to adress the problem, so far the feedback
on my stuff has been very positive so I'm optimistic that it will be
accepted.

> First, for what reason do you maintain /var/cache/edb/glsa? You could
> ask portage to see, wether a special version of a package is installed
> in the system or is not. To store applied GLSAs in this file seems
> redundant to me, isn't it?

Sometimes you might want to solve an issue in another way than
recommended or the test result could be wrong or you don't want to fix
the issue or ...

> Second, the 'glsa-' prefix of the GLSA filenames seems unnecessary to
> me. The GLSA information is stored in the directory
> /usr/portage/glsa/. Why not name the file with its unique GLSA
> identifier (200307-07 or perhaps 200307-07.xml), shouldn't that be
> enough?

That's open for discussion. FYI, the glsa- prefix can be
changed/disabled in /etc/portage/glsa.conf.

> In your 'readme' file you wrote, that there is no installation needed.
> Nevertheless I would provide a little install-script, which checks the
> presence of the needed files and directories (/etc/portage/, 
> /usr/portage/glsa/  and /var/cache/edb/glsa). Without this dirs and
> the file, glsa.py fails.

Right now the tool is nearly useless as there are no XMLified GLSAs
available. It's only a prototype. Once the GLEP is accepted
I'll write an ebuild for it or ask carpaski to add it to the portage
package.

Marius

--
gentoo-dev@gentoo.org mailing list