[gentoo-soc] euscan - final report

[gentoo-soc] euscan - final report

[Federico "fox" Scrinzi]

[-- Attachment #1: Type: text/plain, Size: 3381 bytes --]

Hi folks!

This is the final update for the euscan GSoC project. It was a great
summer and I'd like to thank you all for this opportunity :D
As usual I remind you that the project repo is at
https://github.com/volpino/euscan (check that for more info)


Here's a summary of my work during the summer, the main goals achieved
and the plans for the future. I'll split the goals in euscan (euscan
script) and euscanwww (web interface)

euscan:
- New handlers system: plugin system with package level handlers (use
package <upstream> metadata) and url level handlers (use package's
SRC_URIs) to scan upstream

- Added new handlers and improved existing ones, in particular for
widely used services (e.g.: sourceforge, google-code, github, ...)

- Wrote a PoC script for getting "watch" file data from Debian packages
and wrote an handler that uses it. We're still working on that and we
have to find a good and efficient way to store addition data in
metadata.xml.
Mail to gentoo-dev: http://www.gossamer-threads.com/lists/gentoo/dev/258330

- Every found version has a confidence score based on the handler used
to discover it (e.g.: PyPI API has an high confidence while bruteforcing
urls has a low confidence)

- various performance improvements

- Added a JSON output to euscan to be machine readable

- Use eix --xml and Layman API to retrieve package data


euscanwww:
- Moved all tasks from shell scripts or Django management commands to
Celery tasks; added a tasks launcher to the admin interface
(this part was quite hard, getting everything to work efficiently and
prevent deadlocks or concurrency issues wasn't easy and Ask, the
developer of Celery, helped us)

- Provide the possibility to register and login to euscan, even using
Gentoo LDAP (not active until euscan will be moved into Gentoo
infrastructure for security reasons)

- Added a "refresh button" to let users request a package scan

- Added a "watch"/"unwatch" functionality for categories, herds,
maintainers, overlays and packages

- Implemented a personalized dashboard for registered users with a
summary of the status of the watched packages

- Personalized RSS feeds for users with only watched stuff

- Options for feeds to tweak output (e.g.: don't show unstable releases,
show only upstream versions, ...)

- Email newsletter for users with new upstream version of watched packages

- Added a form to report euscan errors

Other:
- Added test for old and new code
- All code is PEP8 compliant


For the future:
I plan to use euscan and my GSoC as main subject for my bachelor thesis
so I absolutely keep on working on that. We have a proposal to
gentoo-dev still pending about adding new upstream data to metadata.xml.
First of all we'll deal with that issue and we'll keep on working on new
featues, so stay tuned!


I achieved all the goals in my project proposal with the required
success criteria and I worked on other stuff as well, not strictly in my
project but very interesting (in particular using Debian watch data for
euscan). I'm really satisfied from this summer and I hope that everybody
had a lot of fun!

Thank you guys! See you around ;)

-- 
f.

  "Always code as if the guy who ends up maintaining your code will be a
   violent psychopath who knows where you live."
  (Martin Golding)



[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 900 bytes --]