[gentoo-catalyst] PKGCACHE_PATH and os.makedirs() usage

[gentoo-catalyst] PKGCACHE_PATH and os.makedirs() usage

[Brian Dolbec]

While investigating why the stage1 build was not creating or using
binpkgs as expected.  I discovered that the PKGCACHE-PATH dir was not
being checked for existence nor being created if it didn't.

I did however discover that it was indeed checked and/or created in
livecd_stage2.py.  It should be done in the StageBase class (my rewrite)
where it it set upon the class's initialization.  This will centralize
the operation to one common location.  This portion is something easily
done in the current master and probably desirable for the next release.

Further investigation of the use of os.makedirs() reveals that none of
it's frequent use does any error checking or exception escaping.
I propose to move this common code to either a utility function or
possibly a FileOps class that the main classes can then subclass or
create an instance of.  I need to investigate what method will be best.
That way, exceptions can be controlled for the type of failure desired.
As well as proper shutdown procedures to run for a failure.

If it is moved into a class, I may also move the pack/unpack operations
into it, removing some of the complexity from the currently overloaded
base stage classes.  They are after all file operations.  Again,
something to be determined.


What say everyone?

Re: [gentoo-catalyst] PKGCACHE_PATH and os.makedirs() usage

[Brian Dolbec]

On Tue, 2013-05-28 at 19:08 -0700, Brian Dolbec wrote:
> While investigating why the stage1 build was not creating or using
> binpkgs as expected.  I discovered that the PKGCACHE-PATH dir was not
> being checked for existence nor being created if it didn't.
> 
> I did however discover that it was indeed checked and/or created in
> livecd_stage2.py.  It should be done in the StageBase class (my rewrite)
> where it it set upon the class's initialization.  This will centralize
> the operation to one common location.  This portion is something easily
> done in the current master and probably desirable for the next release.
> 

OK, turns out it is in the base class, so the one in livecd_stage2.py is
a duplicate.  Ignore this first part.

This next section below still applies.
> Further investigation of the use of os.makedirs() reveals that none of
> it's frequent use does any error checking or exception escaping.
> I propose to move this common code to either a utility function or
> possibly a FileOps class that the main classes can then subclass or
> create an instance of.  I need to investigate what method will be best.
> That way, exceptions can be controlled for the type of failure desired.
> As well as proper shutdown procedures to run for a failure.
> 
> If it is moved into a class, I may also move the pack/unpack operations
> into it, removing some of the complexity from the currently overloaded
> base stage classes.  They are after all file operations.  Again,
> something to be determined.
> 
> 
> What say everyone?
> 
> 
> 
> 
> 

Re: [gentoo-catalyst] PKGCACHE_PATH and os.makedirs() usage

[Mike Frysinger]

[-- Attachment #1: Type: Text/Plain, Size: 848 bytes --]

On Tuesday 28 May 2013 22:08:38 Brian Dolbec wrote:
> Further investigation of the use of os.makedirs() reveals that none of
> it's frequent use does any error checking or exception escaping.
> I propose to move this common code to either a utility function or
> possibly a FileOps class that the main classes can then subclass or
> create an instance of.  I need to investigate what method will be best.
> That way, exceptions can be controlled for the type of failure desired.
> As well as proper shutdown procedures to run for a failure.

i get the feeling that catalyst really needs to get away from implementing 
everything itself (either open coding it, or generating its own util libs).  
there are perfectly fine drop in util libs like snakeoil.
http://docs.snakeoil.googlecode.com/git/index.html
osutils.ensure_dirs()
-mike

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: [gentoo-catalyst] PKGCACHE_PATH and os.makedirs() usage

[Brian Dolbec]

On Tue, 2013-05-28 at 23:17 -0400, Mike Frysinger wrote:
> On Tuesday 28 May 2013 22:08:38 Brian Dolbec wrote:
> > Further investigation of the use of os.makedirs() reveals that none of
> > it's frequent use does any error checking or exception escaping.
> > I propose to move this common code to either a utility function or
> > possibly a FileOps class that the main classes can then subclass or
> > create an instance of.  I need to investigate what method will be best.
> > That way, exceptions can be controlled for the type of failure desired.
> > As well as proper shutdown procedures to run for a failure.
> 
> i get the feeling that catalyst really needs to get away from implementing 
> everything itself (either open coding it, or generating its own util libs).  
> there are perfectly fine drop in util libs like snakeoil.
> http://docs.snakeoil.googlecode.com/git/index.html
> osutils.ensure_dirs()
> -mike

yeah, good idea. I keep forgetting all what snakeoil offers, but moving
to using some of it at least was on my TODO.