Re: Hypermail incremental mode

---------

From: Daniel Stenberg (daniel.stenberg@autodiagnos.se)
Date: Mon Apr 12 1999 - 06:30:00 CDT


John Finlay wrote:

> At the time, I didn't discover any others who were working on hypermail
> so I just sat on those fixes.

We were many in exactly the same position. :-)

> Now that I have discovered that others are
> still interested in hypermail, I'm trying to come up to speed on the
> latest version(s) and their features.

You're very welcome to our team! We need more people with skill and
some time over to make improvements.

> I see that there have been a number of useful enhancements to hypermail
> and am wondering about upgrading to a newer version but want to make
> sure that someone has already added in fixes to make the incremental
> mode robust and to make hypermail performant.

In my eyes, the current version 2 is not any longer comparable with 1.02
when it comes to features, such as MIME decoding etc. So I wouldn't even
consider to use 1.02 today.

> 1. added archive locking

This exists in v2. In the recent 2a18 you're now able to specify how long
to wait for a lock before deliberatly breaking it.

> 2. added an optional file that contained the summary info of the archive
> to avoid opening all the archive files during an update. [this removed a
> big performance hit on large archives]

This does not exist, and I have not added any such on purpose. I believe
this is left for discussions whether we should add one or not. I'm in
favour of a solution where such a file is used if existent, but where the
HTML files are scanned and that info is used if the "cache" is for some
reason not there.

> 3. fixed up the crossindexing of threads, etc. [more reliable indexing]

The entire thread indexing was rewritten for 2a18.

> 4. cleaned up the print.c routines. [rationalized these]

They are still a bit weird but modified a lot since 1.02.

> 5. fixed lots of string handling errors in string.c [source of
> coredumps]

Since beyond 2.0b3 all string handling is dynamic and should not
cause any buffer overflows anymore. Instead we got a bunch of NULL
referings that have been and will be taken care of.

> 6. changed the addhash function and introduced a mail struct to avoid
> replicating each header string 4 times. [big saving on memory usage with
> large archives]

Done in 2a18.

> 7. eliminated the on-the-fly sorting of messages by using qsort on the
> tables. [big performance enhancement for large archives]

I surely wouldn't mind if you added that functionality to the 2a18
package and sent me a patch for it! ;-)

> 8. fixed up the date routines which seemed to broken in a number of ways

The date routines are pretty much modified for 2a18 too. Paul Haldane has
been friendly enough to shoulder the date stuff for now so I'm hoping
you can coordinate your ideas with his new stuff.

I'm awaiting some further fixes in that area.

> 9. fixed lots of string handling errors in the parse.c routines.

I believe parse.c has been modified pretty much since 1.02. If you had
specific problems with 1.02 I think the best way would be to test those
cases against 2a18 to see if they're corrected or not.

> I took a quick look at 20b3 and it seems not to have fixes equivalent to
> 1, 2, 6, or 7. I didn't check for 3, 4, 5, 8, or 9 but I would hope
> that these have been fixed since they are a source of coredumps.

The beta 3 is dated August '98 and is by all measures an old version.

> After a quick look at 2a18, I noticed fixes equivalent to 1 and 6. I
> didn't see fixes equivalent to 2 or 7. I didn't check for the rest
> though I hope these are fixed.

You're correct.

> - is my analysis of the current releases wrt my concerns somewhat
> correct?

It seems so, yes.

> - does anyone else care about/use incremental mode of hypermail?

The bug report flow shows that many people use hypermail in incremental
mode, yes. I also believe that one of the main reason people use
hypermail at all and not any competitor, is speed. We must keep hypermail
speedy or else we have no reason to exist at all.

> - does anyone care about large archives or is the current practice to
> slice the archives by month or week to avoid the inherent problems?

I think it depends a little on what "large archives" mean. I think
hypermail should be made to work nicely and swiftly with archives
containing at least a couple of thousand mails, if that is a large
archive or not by your means I don't know.

> - does the 2a18 represent the current development effort? Is 20b3 an
> earlier dead release?

Yes and yes. Have a look at www.fts.frontec.se/~dast/hypermail/. It is
my hypermail page that blurbs a little about what versions that do what
and it also explains how to get the very latest source off the CVS
server.

It has turned out that I am the current maintainer of the developing
since Kent Landfield has simply vanished.

I do not wish to take credit for all of the hypermail v2 developing, but
I am collecting all the patches I get and add them into the archive and
I also publish tar-balls of the development version with some random
intervals ;-)

We do need and appriciate all help and support we can get with developing
hypermail v2 into a stable and fast version. Right now, I tend to
prioritize fixes that improve stability, instead of adding more and more
fancy features before a stable v2 has been released. We also need a lot
of help with docs and setting up a decent web page (Kent is the maintainer
of landfield.com and it is terribly outdated).

 / Daniel

(please don't write me at this address, my real address is
Daniel.Stenberg@sth.frontec.se)


---------

This archive was generated by hypermail 2.1.5.