[gentoo-dev] [RFC] Removing support for mercurial repos in repositories.xml

[gentoo-dev] [RFC] Removing support for mercurial repos in repositories.xml

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 1180 bytes --]

Hi, everyone.

I'd like to ask Gentoo repository owners to switch off Mercurial
and remove all Mercurial repositories from repositories.xml.  There are
two reasons for that:

1. Portage does not support syncing from Mercurial repos, and needs to
use external tools (e.g. layman) for that.

2. Mercurial is buggy and maintaining support for those repos is PITA.

If you noticed that Gentoo repository mirrors did not update for 10
hours a few days ago -- Mercurial was the reason.  It is very fragile,
and if some server chokes during sync, it hangs the whole process until
somebody (which means me) kills it.  And it's not the first time it
killed the whole system.

Yes, I could add process timeouts.  But small timeouts are going to
break the occasional necessity of cloning big repos, and big timeouts
are going to make little difference when Mercurial starts hanging again.

If someone really cares about this horrible piece of software, I'd
appreciate patches (preferably going upstream) to make it timeout sanely
when something hangs.  Otherwise, I'd like to announce discontinuation
of Mercurial support soon.

-- 
Best regards,
Michał Górny

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 963 bytes --]

Re: [gentoo-dev] [RFC] Removing support for mercurial repos in repositories.xml

[Alec Warner]

[-- Attachment #1: Type: text/plain, Size: 1405 bytes --]

On Sun, Sep 23, 2018 at 4:42 PM Michał Górny <mgorny@gentoo.org> wrote:

> Hi, everyone.
>
> I'd like to ask Gentoo repository owners to switch off Mercurial
> and remove all Mercurial repositories from repositories.xml.  There are
> two reasons for that:
>
> 1. Portage does not support syncing from Mercurial repos, and needs to
> use external tools (e.g. layman) for that.
>
> 2. Mercurial is buggy and maintaining support for those repos is PITA.
>
> If you noticed that Gentoo repository mirrors did not update for 10
> hours a few days ago -- Mercurial was the reason.  It is very fragile,
> and if some server chokes during sync, it hangs the whole process until
> somebody (which means me) kills it.  And it's not the first time it
> killed the whole system.
>
> Yes, I could add process timeouts.  But small timeouts are going to
> break the occasional necessity of cloning big repos, and big timeouts
> are going to make little difference when Mercurial starts hanging again.
>

So you are against timeouts altogether, or you just don't want to implement
them?


>
> If someone really cares about this horrible piece of software, I'd
> appreciate patches (preferably going upstream) to make it timeout sanely
> when something hangs.  Otherwise, I'd like to announce discontinuation
> of Mercurial support soon.
>

> --
> Best regards,
> Michał Górny
>

[-- Attachment #2: Type: text/html, Size: 1993 bytes --]

Re: [gentoo-dev] [RFC] Removing support for mercurial repos in repositories.xml

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 1752 bytes --]

On Sun, 2018-09-23 at 18:46 -0400, Alec Warner wrote:
> On Sun, Sep 23, 2018 at 4:42 PM Michał Górny <mgorny@gentoo.org> wrote:
> 
> > Hi, everyone.
> > 
> > I'd like to ask Gentoo repository owners to switch off Mercurial
> > and remove all Mercurial repositories from repositories.xml.  There are
> > two reasons for that:
> > 
> > 1. Portage does not support syncing from Mercurial repos, and needs to
> > use external tools (e.g. layman) for that.
> > 
> > 2. Mercurial is buggy and maintaining support for those repos is PITA.
> > 
> > If you noticed that Gentoo repository mirrors did not update for 10
> > hours a few days ago -- Mercurial was the reason.  It is very fragile,
> > and if some server chokes during sync, it hangs the whole process until
> > somebody (which means me) kills it.  And it's not the first time it
> > killed the whole system.
> > 
> > Yes, I could add process timeouts.  But small timeouts are going to
> > break the occasional necessity of cloning big repos, and big timeouts
> > are going to make little difference when Mercurial starts hanging again.
> > 
> 
> So you are against timeouts altogether, or you just don't want to implement
> them?

I'm against dumb timeouts.  Good timeout = die if nothing happens for T.
 Bad timeout = die if process doesn't finish for T (yet it may still be 
doing something).

> 
> 
> > 
> > If someone really cares about this horrible piece of software, I'd
> > appreciate patches (preferably going upstream) to make it timeout sanely
> > when something hangs.  Otherwise, I'd like to announce discontinuation
> > of Mercurial support soon.
> > 
> > --
> > Best regards,
> > Michał Górny
> > 

-- 
Best regards,
Michał Górny

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 963 bytes --]

Re: [gentoo-dev] [RFC] Removing support for mercurial repos in repositories.xml

[Kent Fredric]

[-- Attachment #1: Type: text/plain, Size: 963 bytes --]

On Mon, 24 Sep 2018 07:59:46 +0200
Michał Górny <mgorny@gentoo.org> wrote:

> I'm against dumb timeouts.  Good timeout = die if nothing happens for T.
>  Bad timeout = die if process doesn't finish for T (yet it may still be 
> doing something).

Could you perhaps adjust it so that when the timeout limit is exceeded,
the sync is aborted, and the sync status of that repository is flagged
as "bad", and notifies you somehow to fix it manually at your leisure,
but otherwise ceases to impede the progress of all the other
repositories?

And if you're worried that a sync interruption midway could cause a
dirty state, maybe you could do an rsync trick where mercurial repos
are rsynced into a sandbox location, and then only rsynced back into
place on success?

This process lets you go "sure, it may be doing something, but it took
too long anyway, so we'll let somebody with brains work it out and
pretend it was broken in the interim"


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [gentoo-dev] [RFC] Removing support for mercurial repos in repositories.xml

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 1104 bytes --]

On Fri, 2018-09-28 at 01:50 +1200, Kent Fredric wrote:
> On Mon, 24 Sep 2018 07:59:46 +0200
> Michał Górny <mgorny@gentoo.org> wrote:
> 
> > I'm against dumb timeouts.  Good timeout = die if nothing happens for T.
> >  Bad timeout = die if process doesn't finish for T (yet it may still be 
> > doing something).
> 
> Could you perhaps adjust it so that when the timeout limit is exceeded,
> the sync is aborted, and the sync status of that repository is flagged
> as "bad", and notifies you somehow to fix it manually at your leisure,
> but otherwise ceases to impede the progress of all the other
> repositories?
> 
> And if you're worried that a sync interruption midway could cause a
> dirty state, maybe you could do an rsync trick where mercurial repos
> are rsynced into a sandbox location, and then only rsynced back into
> place on success?
> 
> This process lets you go "sure, it may be doing something, but it took
> too long anyway, so we'll let somebody with brains work it out and
> pretend it was broken in the interim"

No.

-- 
Best regards,
Michał Górny

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 963 bytes --]

Re: [gentoo-dev] [RFC] Removing support for mercurial repos in repositories.xml

[desultory]

On 09/27/18 10:21, Michał Górny wrote:
> On Fri, 2018-09-28 at 01:50 +1200, Kent Fredric wrote:
>> On Mon, 24 Sep 2018 07:59:46 +0200
>> Michał Górny <mgorny@gentoo.org> wrote:
>>
>>> I'm against dumb timeouts.  Good timeout = die if nothing happens for T.
>>>  Bad timeout = die if process doesn't finish for T (yet it may still be 
>>> doing something).
>>
>> Could you perhaps adjust it so that when the timeout limit is exceeded,
>> the sync is aborted, and the sync status of that repository is flagged
>> as "bad", and notifies you somehow to fix it manually at your leisure,
>> but otherwise ceases to impede the progress of all the other
>> repositories?
>>
>> And if you're worried that a sync interruption midway could cause a
>> dirty state, maybe you could do an rsync trick where mercurial repos
>> are rsynced into a sandbox location, and then only rsynced back into
>> place on success?
>>
>> This process lets you go "sure, it may be doing something, but it took
>> too long anyway, so we'll let somebody with brains work it out and
>> pretend it was broken in the interim"
> 
> No.
> 
Elucidate.

Re: [gentoo-dev] [RFC] Removing support for mercurial repos in repositories.xml

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 1442 bytes --]

On Fri, 2018-09-28 at 23:44 -0400, desultory wrote:
> On 09/27/18 10:21, Michał Górny wrote:
> > On Fri, 2018-09-28 at 01:50 +1200, Kent Fredric wrote:
> > > On Mon, 24 Sep 2018 07:59:46 +0200
> > > Michał Górny <mgorny@gentoo.org> wrote:
> > > 
> > > > I'm against dumb timeouts.  Good timeout = die if nothing happens for T.
> > > >  Bad timeout = die if process doesn't finish for T (yet it may still be 
> > > > doing something).
> > > 
> > > Could you perhaps adjust it so that when the timeout limit is exceeded,
> > > the sync is aborted, and the sync status of that repository is flagged
> > > as "bad", and notifies you somehow to fix it manually at your leisure,
> > > but otherwise ceases to impede the progress of all the other
> > > repositories?
> > > 
> > > And if you're worried that a sync interruption midway could cause a
> > > dirty state, maybe you could do an rsync trick where mercurial repos
> > > are rsynced into a sandbox location, and then only rsynced back into
> > > place on success?
> > > 
> > > This process lets you go "sure, it may be doing something, but it took
> > > too long anyway, so we'll let somebody with brains work it out and
> > > pretend it was broken in the interim"
> > 
> > No.
> > 
> 
> Elucidate.

My time is better spent elsewhere.  You want mercurial, you fix it. 
This is open source, you can write the patches.

-- 
Best regards,
Michał Górny

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 963 bytes --]

Re: [gentoo-dev] [RFC] Removing support for mercurial repos in repositories.xml

[Benda Xu]

[-- Attachment #1: Type: text/plain, Size: 521 bytes --]

Michał Górny <mgorny@gentoo.org> writes:

> If you noticed that Gentoo repository mirrors did not update for 10
> hours a few days ago -- Mercurial was the reason.  It is very fragile,
> and if some server chokes during sync, it hangs the whole process until
> somebody (which means me) kills it.  And it's not the first time it
> killed the whole system.

While I agree with you that mercurial is less well-maintained than git,
this seems to me that the sync framework is more buggy than mercurial.

Benda

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]

Re: [gentoo-dev] [RFC] Removing support for mercurial repos in repositories.xml

[Dirkjan Ochtman]

[-- Attachment #1: Type: text/plain, Size: 519 bytes --]

On Sun, Sep 23, 2018 at 10:42 PM Michał Górny <mgorny@gentoo.org> wrote:

> 2. Mercurial is buggy and maintaining support for those repos is PITA.
>

As a former Mercurial maintainer, I'm very skeptical of claims that
Mercurial is that buggy or generally less well-maintained than Git. Still,
reality is that Gentoo is mostly focused on Git (and Mercurial use is
relatively limited in the broader free software community), so I could see
it makes sense to focus our resources on Git.

Regards,

Dirkjan

[-- Attachment #2: Type: text/html, Size: 802 bytes --]

Re: [gentoo-dev] [RFC] Removing support for mercurial repos in repositories.xml

[Cédric Krier]

[-- Attachment #1: Type: text/plain, Size: 862 bytes --]

On 2018-09-23 22:42, Michał Górny wrote:
> Yes, I could add process timeouts.  But small timeouts are going to
> break the occasional necessity of cloning big repos, and big timeouts
> are going to make little difference when Mercurial starts hanging again.
> 
> If someone really cares about this horrible piece of software, I'd
> appreciate patches (preferably going upstream) to make it timeout sanely
> when something hangs.  Otherwise, I'd like to announce discontinuation
> of Mercurial support soon.

FYI, I send an email to mercurial list to talk about this issue [1]. And it
seems that for HTTP connections there is no timeout on the connection.
I'm working to provide to the upstream a solution to be able to
configure a timeout.


[1] https://www.mercurial-scm.org/pipermail/mercurial/2018-September/051003.html
-- 
Cédric Krier

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 376 bytes --]

Re: [gentoo-dev] [RFC] Removing support for mercurial repos in repositories.xml

[Cédric Krier]

[-- Attachment #1: Type: text/plain, Size: 967 bytes --]

On 2018-09-29 11:42, Cédric Krier wrote:
> On 2018-09-23 22:42, Michał Górny wrote:
> > Yes, I could add process timeouts.  But small timeouts are going to
> > break the occasional necessity of cloning big repos, and big timeouts
> > are going to make little difference when Mercurial starts hanging again.
> > 
> > If someone really cares about this horrible piece of software, I'd
> > appreciate patches (preferably going upstream) to make it timeout sanely
> > when something hangs.  Otherwise, I'd like to announce discontinuation
> > of Mercurial support soon.
> 
> FYI, I send an email to mercurial list to talk about this issue [1]. And it
> seems that for HTTP connections there is no timeout on the connection.
> I'm working to provide to the upstream a solution to be able to
> configure a timeout.

Normally, the feature should be available for the next release of
mercurial: https://phab.mercurial-scm.org/D4878

-- 
Cédric Krier

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 376 bytes --]

Re: [gentoo-dev] [RFC] Removing support for mercurial repos in repositories.xml

[Cédric Krier]

[-- Attachment #1: Type: text/plain, Size: 1164 bytes --]

On 2018-10-04 18:05, Cédric Krier wrote:
> On 2018-09-29 11:42, Cédric Krier wrote:
> > On 2018-09-23 22:42, Michał Górny wrote:
> > > Yes, I could add process timeouts.  But small timeouts are going to
> > > break the occasional necessity of cloning big repos, and big timeouts
> > > are going to make little difference when Mercurial starts hanging again.
> > > 
> > > If someone really cares about this horrible piece of software, I'd
> > > appreciate patches (preferably going upstream) to make it timeout sanely
> > > when something hangs.  Otherwise, I'd like to announce discontinuation
> > > of Mercurial support soon.
> > 
> > FYI, I send an email to mercurial list to talk about this issue [1]. And it
> > seems that for HTTP connections there is no timeout on the connection.
> > I'm working to provide to the upstream a solution to be able to
> > configure a timeout.
> 
> Normally, the feature should be available for the next release of
> mercurial: https://phab.mercurial-scm.org/D4878

For the record, the feature is now in the release 4.8 of Mercurial:
https://www.mercurial-scm.org/wiki/Release4.8

-- 
Cédric Krier

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 376 bytes --]