[gentoo-user] IMAP servers

[gentoo-user] IMAP servers

[Steve [Gentoo]]

This is not so much a problem as an 'I wonder'...

I currently use dovecot imap with maildir to support several dozen
folders containing several gigabytes of mail using Thunderbird as a mail
client.  This "works" - but I am surprised by the number of imap
processes that the mail server is running to support a single
Thunderbird client... it appears that there is one connection per folder
and one process per connection... which seems (to me at least) to be
overkill.

I'm aware that there are lots of IMAP clients I might have chosen -
Courier; UWImap, Cyrus and dovecot... but I'm finding it difficult to
establish details which would help me choose the most appropriate server
for my purposes.  In order of importance, I'm interested in:

    1.   Stability and reliability.
    2.   Scalability to handle a very large number of 'folders' for a
relatively small number of end-users.
    3.   Performance (primarily with respect to 'new mail'
notifications, but also, to a lesser extent, with searching archived
messages.

I'd be interested to hear any anecdotes from Gentooists who have
addressed a similar question recently...

Steve



-- 
gentoo-user@gentoo.org mailing list

Re: [gentoo-user] IMAP servers

[Joost Roeleveld]

> This is not so much a problem as an 'I wonder'...
>
> I currently use dovecot imap with maildir to support several dozen
> folders containing several gigabytes of mail using Thunderbird as a mail
> client.  This "works" - but I am surprised by the number of imap
> processes that the mail server is running to support a single
> Thunderbird client... it appears that there is one connection per folder
> and one process per connection... which seems (to me at least) to be
> overkill.

Having never used dovecot, I can't comment on this. But maybe this can be
changed in a configuration file somewhere?
But I think this is caused by the client, not by the server.

> I'm aware that there are lots of IMAP clients I might have chosen -
> Courier; UWImap, Cyrus and dovecot... but I'm finding it difficult to
> establish details which would help me choose the most appropriate server
> for my purposes.  In order of importance, I'm interested in:
>
>     1.   Stability and reliability.
>     2.   Scalability to handle a very large number of 'folders' for a
> relatively small number of end-users.
>     3.   Performance (primarily with respect to 'new mail'
> notifications, but also, to a lesser extent, with searching archived
> messages.
>
> I'd be interested to hear any anecdotes from Gentooists who have
> addressed a similar question recently...
>
> Steve

I have a similar requirement for my emails, esp. with the mailinglists I
can end up with some very large folders (100,000+ messages in one of my
folders)

I found that Courier-IMAP didn't manage to handle these folders very well,
and I had to split these folders up into seperate folders for this to
work.
I have since moved to Cyrus and I find that this works easier, even with
only 2 users using it, it currently handles over 7Gb of stored email.

Only once had a problem with one of the folders, which was easily fixed.
The cause for this was most likely a minor corruption on the filesystem
when my server lost power. (UPS is unreliable, but that's a different
issue)

As for the performance with regards to notifications, I find that the
mailclient is responsible for this, rather then the server. Doing a quick
check for new emails is very quick with Cyrus, Courier didn't scale very
well in my experience.

I hope this helps, and that others can come with more information,
including some benchmarks, if these exist.

As a final note, I don't think either Courier or Cyrus have ever lost a
single email, but I have never done a full check for this.

--
Joost Roeleveld

-- 
gentoo-user@gentoo.org mailing list

Re: [gentoo-user] IMAP servers

[Roman Zilka]

On Mon, Oct 16, 2006 at 11:26:23AM +0100, Joost Roeleveld wrote:
> > This is not so much a problem as an 'I wonder'...
> >
> > I currently use dovecot imap with maildir to support several dozen
> > folders containing several gigabytes of mail using Thunderbird as a mail
> > client.  This "works" - but I am surprised by the number of imap
> > processes that the mail server is running to support a single
> > Thunderbird client... it appears that there is one connection per folder
> > and one process per connection... which seems (to me at least) to be
> > overkill.
> 
> Having never used dovecot, I can't comment on this. But maybe this can be
> changed in a configuration file somewhere?

Confirm. There are a few settings regarding exactly this
issue. Check out all the "login_process*" and "login_max*" options at least.

Regards
-Roman
-- 
gentoo-user@gentoo.org mailing list

Re: [gentoo-user] IMAP servers

[Régis Décamps]

On 10/16/06, Roman Zilka <rzilka@gvid.cz> wrote:
> On Mon, Oct 16, 2006 at 11:26:23AM +0100, Joost Roeleveld wrote:
> > > This is not so much a problem as an 'I wonder'...
> > >
> > > I currently use dovecot imap with maildir to support several dozen
> > > folders containing several gigabytes of mail using Thunderbird as a mail
> > > client.  This "works" - but I am surprised by the number of imap
> > > processes that the mail server is running to support a single
> > > Thunderbird client... it appears that there is one connection per folder
> > > and one process per connection... which seems (to me at least) to be
> > > overkill.
> >
> > Having never used dovecot, I can't comment on this. But maybe this can be
> > changed in a configuration file somewhere?
>
> Confirm. There are a few settings regarding exactly this
> issue. Check out all the "login_process*" and "login_max*" options at least.
>

Exactly, and on the client side as well. If you reduce the number of
processes per IP on the server, the client might have some
difficulties and complain aboout "connection impossible".

All IMAP open several threads like this, and I believe it improves
performance. Why would you change this?

Be warned though that IMAP servers handle the INBOX differently (some
pretend everything is in INBOX, like INBOX/Sent ; others present
folders at the same level than INBOX)

That said, I like dovecot and courier-imap equally ; and I don't like UW-imap.

-- 
Régis

http://regis.decamps.info/blog/

-- 
gentoo-user@gentoo.org mailing list

Re: [gentoo-user] IMAP servers

[Steve [Gentoo]]

Roman Zilka wrote:
> On Mon, Oct 16, 2006 at 11:26:23AM +0100, Joost Roeleveld wrote:
>   
>>> it appears that there is one connection per folder
>>> and one process per connection... which seems (to me at least) to be overkill.
>>>       
>> Having never used dovecot, I can't comment on this. But maybe this can be changed in a configuration file somewhere?
>>     
> Confirm. There are a few settings regarding exactly this
> issue. Check out all the "login_process*" and "login_max*" options at least.
>   
This sounded very promising... and I've now spent some time tweaking
those parameters (I'm not sure why I'd previously overlooked them.)

The parameter login_processes_count seems to control the number of
/usr/libexec/dovecot/imap-login processes that are run (when idle)
awaiting connections.  The parameter login_max_processes_count appears
to relate to the same imap_login process.  It was not the imap_login
processes which concern me from a scalability perspective - but rather
the imap (/usr/libexec/dovecot/imap) processes.  I'm no longer so sure
about when imap processes are spawned or terminated - but there seem to
be a non-deterministic "lots" of them for each Thunderbird session.




-- 
gentoo-user@gentoo.org mailing list

Re: [gentoo-user] IMAP servers

[Régis Décamps]

On 10/17/06, Steve [Gentoo] <gentoo_steve@shic.co.uk> wrote:
>  I'm no longer so sure
> about when imap processes are spawned or terminated - but there seem to
> be a non-deterministic "lots" of them for each Thunderbird session.

There is one process create each time Thunderbird opens a new
connection, up to a limit which should be configured in your imap
server (MAXPERIP for courier-imap)

Thunderbird opens a connection each time a new folder is accessed, and
closes them when it has reach a limit which is configured in "Account
Setting", <account>, "Server settings", "Advanced", "maximum unmber of
connections to cache"


-- 
Régis

http://regis.decamps.info/blog/

-- 
gentoo-user@gentoo.org mailing list

Re: [gentoo-user] IMAP servers

[Steve [Gentoo]]

Régis Décamps wrote:
> On 10/17/06, Steve [Gentoo] <gentoo_steve@shic.co.uk> wrote:
>>  I'm no longer so sure about when imap processes are spawned or
>> terminated - but there seem to
>> be a non-deterministic "lots" of them for each Thunderbird session.
> There is one process create each time Thunderbird opens a new
> connection, up to a limit which should be configured in your imap
> server (MAXPERIP for courier-imap)
I couldn't find a setting for this in dovecot...
> Thunderbird opens a connection each time a new folder is accessed, and
> closes them when it has reach a limit which is configured in "Account
> Setting", <account>, "Server settings", "Advanced", "maximum unmber of
> connections to cache"
...but, given that gem, it doesn't matter to me any longer.  I've
reduced the setting in Thunderbird to something sensibly small and
things are running far more smoothly now.

Thanks!


-- 
gentoo-user@gentoo.org mailing list