[gentoo-qa] Re: [GSoC-status] Collagen - database schema and further changes

[Stanislav Ochotnicky <sochotnicky@gmail.com>]

[-- Attachment #1: Type: text/plain, Size: 3233 bytes --]

Heya everyone,

another (almost) week went by so here is another status report. 

As I stated in my last report one of key goals for this week was db
layer for storing information retrieved by tinderboxes. I was looking
into using various ORM frameworks. It was suggested to me to try Django
and I though "Hey, that's not even ORM framework, but a web framework".

Well one part of my project is creating web interface for database at
later stage. So in spirit of not doing same thing twice I looked into
using ORM part of Django. And guess what? I it doable, and basic
implementation is in devel branch of my repo. 

There were certain caveats of course. Django is designed to work
for web applications, not as general purpose ORM framework. So when
using its ORM part without rest of Django, I have to take care of
DB exceptions and rollback of transactions myself. I soon realized
I am doing same thing in every db function I was writing so I ended
up writing a decorator in Python (finally had a reason! :-) ). It
looks something like this:

--- CODE

def dbquery(f):
    def decor(*args, **kwargs):
        reset_queries() 
        try:
            return f(*args, **kwargs)
        except Exception, e:
            _rollback_on_exception() 
            raise e
    return decor

@dbquery
def add_package(...)

--- CODE

This way we can be sure that failed transactions are rolled back. 

Because I am using Django to generate SQL now, orignal database
schema that I commited to repository some time ago is now deprecated. We
can generate database (and initial data) by using django-admin syncdb
command now.

This approach seems fairly good so far since everything was set-up by
code that fits on one screen. I only wish using only small part of
Django was less painful.


And now comes the big part. Actually populating the database with some
meaningful data. I did some work in that part. Last week there were some
modifications to protocol I was using between Matchbox and Tinderboxes.
Most of changes were touching code dealing with log/environment
collection and fact that we have been compiling inside chroot
environment.

I also added support for packages that require certain use flags
enabled/disabled for their dependencies. Good example is dev-utils/git
(requires dev-utils/subversion[-dso]). For git this is however only
RDEPEND (runtime dependency) so compilation doesn't depend on it.

Since we already have db for storing results, and support is there let's
compile some packages! For next week I plan to finally add proper
testing to the mix. Instead of compiling fortune-mod over and over,
Matchbox will ask also for other packages to be compiled. This will bring
out some more problems that will need to be fixed I am sure. At least
that's the idea...

-- 
Stanislav Ochotnicky
Working for Gentoo Linux http://www.gentoo.org
Implementing Tree-wide collision checking and provided files database
http://soc.gentooexperimental.org/projects/show/collision-database
Blog: http://inputvalidation.blogspot.com/search/label/gsoc


jabber: sochotnicky@gmail.com
icq: 74274152
PGP: https://dl.getdropbox.com/u/165616/sochotnicky-key.asc

[-- Attachment #2: Type: application/pgp-signature, Size: 198 bytes --]