[gentoo-soc] Automatically generated overlay of R packages - progress report #4

[gentoo-soc] Automatically generated overlay of R packages - progress report #4

[André Erdmann]

Hi everyone,

== Brief summary of this project ==

The aim of this project is to create scripts that automate the process
of overlay creation/maintenance for R packages from repositories such
as CRAN and Bioconductor.

Longer:
For the ebuild creation of a single package one needs to extract the
package, copy-paste data from its description file to the ebuild and
look up dependencies, which is time-consuming.
Although trivial for a few number of packages, this is practically
impossible to do by hand for repositories like CRAN (> 3500 packages),
especially 'cause it also requires tracking changes (new / updated /
removed packages).
The solution is to automate that process and this is what this project is about.

== Progress of this week ==

My plan for this week was:
* improve dependency resolution, add some functionality (e.g. listener
modules that log unresolved dependencies in a separate file) and make
it more thread-able
* add Manifest / metadata creation

And that's what I've achieved:

* dependency resolution
-> listener modules, unresolved dependencies can now be logged to an extra file
-> 'negative' result cache that stores unresolvable dependencies
-> the dependency resolver supports threads now (one master and
multiple worker threads)

Manifest creation should work (needs more testing), I'm currently
using the ebuild(1) interface to do this within a filtered
environment, which contains only necessary variables from os.environ
and adjusts portage variables (DISTDIR, FEATURES).

Metadata creation works using a data tree (metadata root, description
node, use flag list node, use flag nodes,...). This is inspired by how
portage is reading metadata files.

I'm on schedule.

== Plans for the next (this) week ==
I'm now starting with 'putting all things together'. This means I'll
first concentrate on an "R package -> ebuild in overlay" interface
(including metadata, Manifest), which will later be part of an "local
distfiles dir -> overlay" interface.

-- 
Regards,
André E.