[gentoo-soc] GSoC2013 suggestion - Fix archives.g.o

[gentoo-soc] GSoC2013 suggestion - Fix archives.g.o

[Denis M.]

[-- Attachment #1: Type: text/plain, Size: 429 bytes --]

Hello,
I want to suggest something here. We all know that archives.g.o hasn't
been working for some months... From what I've heard from robbat, the
system is still receiving and processing the emails but it's not
'archiving' them... I think it's a good idea if someone can gather a
group of willing people to fix this, so we can have a gentoo 'official'
place to read the mailing lists.

Regards,
Denis M. (Phr33d0m)


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: [gentoo-soc] GSoC2013 suggestion - Fix archives.g.o

[Robin H. Johnson]

On Tue, Apr 09, 2013 at 12:11:23AM +0200, Denis M. wrote:
> Hello,
> I want to suggest something here. We all know that archives.g.o hasn't
> been working for some months... From what I've heard from robbat, the
> system is still receiving and processing the emails but it's not
> 'archiving' them... I think it's a good idea if someone can gather a
> group of willing people to fix this, so we can have a gentoo 'official'
> place to read the mailing lists.
I'll provide some more details, in the hopes that somebody will take
this on as a project.

Yes, I am willing to mentor you and provide some help, but you'll need
to be fairly self-sufficient: I didn't build the archival system, I just
ended up being one of the maintainers.

We are using Mhonarc to handle the mails, with custom templates that
convert each mail to our GuideXML (a derivative of Docbook XML). An
update got applied accidentally, and since then, each run of mhonarc
either crashes or overrides the prior index.

The overwrite is a major problem, because there's a limit as to how many
emails mhonarc can handle in a single pass, and lots of our lists have
exceeded it.

Mhonarc itself is written in Perl. Gentoo also has some modifications so
that mhonarc generates consistent URLs even when asked to reindex a list
from scratch. This is done by injecting headers into the emails when
they are received by procmail (after postfix), and then the output
filenames for mhonarc are based on those headers.

As a project, we would prefer if mhonarc were continued to be used, but
as long as we use the same consistent URLs as mhonarc, I'm not actually
set in that discussion. Likewise, GuideXML would be useful, because then
it can just slot into the existing system, but also flexible. I don't
think the GuideXML part would be difficult at all either, it's really
just extracting a few headers, and then dumping the body into a single
tag.

-- 
Robin Hugh Johnson
Gentoo Linux: Developer, Trustee & Infrastructure Lead
E-Mail     : robbat2@gentoo.org
GnuPG FP   : 11ACBA4F 4778E3F6 E4EDF38E B27B944E 34884E85

Re: [gentoo-soc] GSoC2013 suggestion - Fix archives.g.o

[Denis M.]

[-- Attachment #1: Type: text/plain, Size: 1969 bytes --]

On 04/09/2013 12:42 AM, Robin H. Johnson wrote:
> On Tue, Apr 09, 2013 at 12:11:23AM +0200, Denis M. wrote:
>> Hello,
>> I want to suggest something here. We all know that archives.g.o hasn't
>> been working for some months... From what I've heard from robbat, the
>> system is still receiving and processing the emails but it's not
>> 'archiving' them... I think it's a good idea if someone can gather a
>> group of willing people to fix this, so we can have a gentoo 'official'
>> place to read the mailing lists.
> I'll provide some more details, in the hopes that somebody will take
> this on as a project.
Thanks for the valuable information.

> Yes, I am willing to mentor you and provide some help, but you'll need
> to be fairly self-sufficient: I didn't build the archival system, I just
> ended up being one of the maintainers.
This is a whole new terminology to me that I'm not familiar with at all.
I could study the information on the software you have mentioned and
most probably be of some help, although it will be almost impossible for
me to take care of the entirety of this project, unfortunately.

If we can gather a (small) group of knowledgeable people I'd be very
interested in participating and be of any help possible. If that does
not happen, as I said above, it will be impossible for me to take care
of this issue by myself.
> [..] An
> update got applied accidentally, and since then, each run of mhonarc
> either crashes or overrides the prior index.
Just of out curiosity (and of course someone has already thought of
that, but...) - isn't it possible to revert that update and get it
running smoothly like before with the less effort possible? I guess
there must be some reason behind not have done that already, do you
happen to know what the reason is?

And of course, this is a lame solution to the problem, as what we want
to achieve is do a proper fix of it.


Regards,
Denis M. (Phr33d0m)


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: [gentoo-soc] GSoC2013 suggestion - Fix archives.g.o

[Robin H. Johnson]

On Tue, Apr 09, 2013 at 02:57:40AM +0200, Denis M. wrote:
> > [..] An
> > update got applied accidentally, and since then, each run of mhonarc
> > either crashes or overrides the prior index.
> Just of out curiosity (and of course someone has already thought of
> that, but...) - isn't it possible to revert that update and get it
> running smoothly like before with the less effort possible? I guess
> there must be some reason behind not have done that already, do you
> happen to know what the reason is?
Nobody has a copy of the file beforehand :-(. It had a patch on it to
fix a previous crash, but said patch seems to have been lost.

-- 
Robin Hugh Johnson
Gentoo Linux: Developer, Trustee & Infrastructure Lead
E-Mail     : robbat2@gentoo.org
GnuPG FP   : 11ACBA4F 4778E3F6 E4EDF38E B27B944E 34884E85

Re: [gentoo-soc] GSoC2013 suggestion - Fix archives.g.o

[Rich Freeman]

On Mon, Apr 8, 2013 at 9:07 PM, Robin H. Johnson <robbat2@gentoo.org> wrote:
> Nobody has a copy of the file beforehand :-(. It had a patch on it to
> fix a previous crash, but said patch seems to have been lost.
>

Hopefully something has changed so that this sort of thing doesn't happen again?

We had a situation like this happen at work - we actually had to pay
to have something re-developed from requirements because nobody
bothered to check in the source code.  As an added bonus it was legacy
code that nobody even used any longer, but which needed to be present
so that old data in the system could still be retrieved.  Calculate
the ROI on that...

Just another reason developers shouldn't have access to production
systems.  I'd hope that most would think twice before writing an
install script that says "Step 4 - get Joe to email you the file...."
:)

Rich