public inbox for gentoo-user@lists.gentoo.org
 help / color / mirror / Atom feed
* [gentoo-user] Upgrading without breaking LVM
@ 2008-06-18 17:41 Matt Harrison
  2008-06-18 20:46 ` Florian Philipp
  2008-06-24 14:32 ` Daniel Iliev
  0 siblings, 2 replies; 15+ messages in thread
From: Matt Harrison @ 2008-06-18 17:41 UTC (permalink / raw
  To: gentoo-user

[-- Attachment #1: Type: text/plain, Size: 1143 bytes --]


Hi all,

Some of you may remember my problems with lvm after an update in 
http://www.archivum.info/gentoo-user@lists.gentoo.org/2008-04/msg00899.html

I'm now headed back towards the same situation. I have a load of updates to do
according to emerge -DNavu world but I've had to mask all upgraded versions of
udev, mdadm and lvm as every time I upgrade them, it breaks my disk access.

Whenever I upgrade these packages (I'm not which one of those 3 is actually to
blame, and I don't want to chance it) I can no longer access the lvm volumes on
my disks. I have no idea what is causing it.

This means I cannot do a world update as those manually masked packages are
blocking a lot of other packages. 

A few months on, is anyone able to shed light on why updating any of those 3
packages causes my system to be unbootable and my data to be inaccessible?

I would really like to keep my system up to date, but as you can understand, I'm
loathe to make changes that will give me a borked system yet again.

Grateful for advice anyone can give as my system is now getting quite out of
date.

Thanks

Matt Harrison

[-- Attachment #2: Type: application/pgp-signature, Size: 197 bytes --]

^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: [gentoo-user] Upgrading without breaking LVM
  2008-06-18 21:01   ` Matt Harrison
@ 2008-06-18 20:25     ` Alan McKinnon
  2008-06-18 21:48     ` Florian Philipp
  1 sibling, 0 replies; 15+ messages in thread
From: Alan McKinnon @ 2008-06-18 20:25 UTC (permalink / raw
  To: gentoo-user

On Wednesday 18 June 2008, Matt Harrison wrote:
> On Wed, Jun 18, 2008 at 10:46:25PM +0200, Florian Philipp wrote:
> > Which versions are you using, to which shall they be updated and on
> > which arch are you? Knowing emerge --info and your profile might
> > also help.
>
> Thanks for the reply.
>
> Ok, versions in use:
>
> lvm2-2.02.10
> mdadm-2.6.4-r1
> udev-104-r12

Hang on a minute ....

Those are stable packages. IIRC you are using baselayout-2 right?

I'm not surprised you are having trouble. What happens if you mark lvm2, 
mdadm and udev unstable and re-merge? Does it just update those 
packages, or does it pull in a ton of other stuff as well?

AFAIK, there isn't an easy way to downgrade baselayout, and I wouldn't 
like to try either....

-- 
Alan McKinnon
alan dot mckinnon at gmail dot com

-- 
gentoo-user@lists.gentoo.org mailing list



^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: [gentoo-user] Upgrading without breaking LVM
  2008-06-18 17:41 [gentoo-user] Upgrading without breaking LVM Matt Harrison
@ 2008-06-18 20:46 ` Florian Philipp
  2008-06-18 21:01   ` Matt Harrison
  2008-06-24 14:32 ` Daniel Iliev
  1 sibling, 1 reply; 15+ messages in thread
From: Florian Philipp @ 2008-06-18 20:46 UTC (permalink / raw
  To: gentoo-user

[-- Attachment #1: Type: text/plain, Size: 1541 bytes --]

On Wed, 18 Jun 2008 18:41:12 +0100
Matt Harrison <iwasinnamuknow@genestate.com> wrote:

> 
> Hi all,
> 
> Some of you may remember my problems with lvm after an update in 
> http://www.archivum.info/gentoo-user@lists.gentoo.org/2008-04/msg00899.html
> 
> I'm now headed back towards the same situation. I have a load of
> updates to do according to emerge -DNavu world but I've had to mask
> all upgraded versions of udev, mdadm and lvm as every time I upgrade
> them, it breaks my disk access.
> 
> Whenever I upgrade these packages (I'm not which one of those 3 is
> actually to blame, and I don't want to chance it) I can no longer
> access the lvm volumes on my disks. I have no idea what is causing it.
> 
> This means I cannot do a world update as those manually masked
> packages are blocking a lot of other packages. 
> 
> A few months on, is anyone able to shed light on why updating any of
> those 3 packages causes my system to be unbootable and my data to be
> inaccessible?
> 
> I would really like to keep my system up to date, but as you can
> understand, I'm loathe to make changes that will give me a borked
> system yet again.
> 
> Grateful for advice anyone can give as my system is now getting quite
> out of date.
> 
> Thanks
> 
> Matt Harrison

Which versions are you using, to which shall they be updated and on
which arch are you? Knowing emerge --info and your profile might also
help.

By the way: Did you report it as a bug? If not, I would certainly do so
if I were you.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 197 bytes --]

^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: [gentoo-user] Upgrading without breaking LVM
  2008-06-18 20:46 ` Florian Philipp
@ 2008-06-18 21:01   ` Matt Harrison
  2008-06-18 20:25     ` Alan McKinnon
  2008-06-18 21:48     ` Florian Philipp
  0 siblings, 2 replies; 15+ messages in thread
From: Matt Harrison @ 2008-06-18 21:01 UTC (permalink / raw
  To: gentoo-user

[-- Attachment #1: Type: text/plain, Size: 962 bytes --]

On Wed, Jun 18, 2008 at 10:46:25PM +0200, Florian Philipp wrote:
> Which versions are you using, to which shall they be updated and on
> which arch are you? Knowing emerge --info and your profile might also
> help.

Thanks for the reply.

Ok, versions in use:

lvm2-2.02.10
mdadm-2.6.4-r1
udev-104-r12

I've masked updates to an versions after those above. Emerge wants to
upgrade to the following:

lvm2-2.02.28-r2
udev-119

mdadm doesn't actually want to be upgraded so it must be a problem with one of
those two packages.

All I know is when I do a world update, my lvm dies and I have to reinstall or
go through a painful downgrade procedure.

> By the way: Did you report it as a bug? If not, I would certainly do so
> if I were you.

I haven't but that's because no-one else has reported problems like this, and
I can work out why its affecting me in enough detail. I don't want to submit
a but report unless I can back it up with some useful information.

[-- Attachment #2: Type: application/pgp-signature, Size: 197 bytes --]

^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: [gentoo-user] Upgrading without breaking LVM
  2008-06-18 21:01   ` Matt Harrison
  2008-06-18 20:25     ` Alan McKinnon
@ 2008-06-18 21:48     ` Florian Philipp
  2008-06-18 22:15       ` Matt Harrison
  1 sibling, 1 reply; 15+ messages in thread
From: Florian Philipp @ 2008-06-18 21:48 UTC (permalink / raw
  To: gentoo-user

[-- Attachment #1: Type: text/plain, Size: 931 bytes --]

On Wed, 18 Jun 2008 22:01:03 +0100
Matt Harrison <iwasinnamuknow@genestate.com> wrote:

> On Wed, Jun 18, 2008 at 10:46:25PM +0200, Florian Philipp wrote:
> > Which versions are you using, to which shall they be updated and on
> > which arch are you? Knowing emerge --info and your profile might
> > also help.
> 
> Thanks for the reply.
> 
> Ok, versions in use:
> 
> lvm2-2.02.10
> mdadm-2.6.4-r1
> udev-104-r12
> 
> I've masked updates to an versions after those above. Emerge wants to
> upgrade to the following:
> 
> lvm2-2.02.28-r2
> udev-119
> 

udev-104 ?! That's not even in the portage tree any longer, right?

In any case: Opening a bug might help you faster than we could. The only
advice I can give you is to upgrade udev first so you know which
package really has the bug and then to try newer versions of lvm, I'd
try 2.02.36 which is marked stable for amd64 (I guess, you're on x86?). 

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 197 bytes --]

^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: [gentoo-user] Upgrading without breaking LVM
  2008-06-18 21:48     ` Florian Philipp
@ 2008-06-18 22:15       ` Matt Harrison
  2008-06-19  5:33         ` Dirk Heinrichs
  0 siblings, 1 reply; 15+ messages in thread
From: Matt Harrison @ 2008-06-18 22:15 UTC (permalink / raw
  To: gentoo-user

[-- Attachment #1: Type: text/plain, Size: 1930 bytes --]

On Wed, Jun 18, 2008 at 11:48:19PM +0200, Florian Philipp wrote:
> On Wed, 18 Jun 2008 22:01:03 +0100
> Matt Harrison <iwasinnamuknow@genestate.com> wrote:
> 
> > On Wed, Jun 18, 2008 at 10:46:25PM +0200, Florian Philipp wrote:
> > > Which versions are you using, to which shall they be updated and on
> > > which arch are you? Knowing emerge --info and your profile might
> > > also help.
> > 
> > Thanks for the reply.
> > 
> > Ok, versions in use:
> > 
> > lvm2-2.02.10
> > mdadm-2.6.4-r1
> > udev-104-r12
> > 
> > I've masked updates to an versions after those above. Emerge wants to
> > upgrade to the following:
> > 
> > lvm2-2.02.28-r2
> > udev-119
> > 
> 
> udev-104 ?! That's not even in the portage tree any longer, right?

Quite possibly, its installed from the 2007.0 stage3 tarball.

And in reply to Alan's post, no I'm not (intentionally) useing BL2. Nothing is
marked unstable on this machine with the example of a ruby module i quite like.

This was the problem before, if I let emerge upgrade mdadm and lvm2, its starts
trying to run BL2 initscripts and complaining that I'm not using BL2.

> In any case: Opening a bug might help you faster than we could. The only
> advice I can give you is to upgrade udev first so you know which
> package really has the bug and then to try newer versions of lvm, I'd
> try 2.02.36 which is marked stable for amd64 (I guess, you're on x86?). 

Yes I'm on x86. I've got a few other amd64 systems that does have any problems,
but I'm installing a test machine now with a layout as close to this problem 
system as possible. I'll try upgrading udev, then lvm2 and see if I can break
it as well.

As I say its annoying because obviously I'm about the only one with this problem.
I didn't think my setup was that peculiar :P

I'll look into a bug report as soon as I can pin down which package is causing the 
real probs.

Thanks

[-- Attachment #2: Type: application/pgp-signature, Size: 197 bytes --]

^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: [gentoo-user] Upgrading without breaking LVM
  2008-06-18 22:15       ` Matt Harrison
@ 2008-06-19  5:33         ` Dirk Heinrichs
  2008-06-19 14:20           ` Matt Harrison
  0 siblings, 1 reply; 15+ messages in thread
From: Dirk Heinrichs @ 2008-06-19  5:33 UTC (permalink / raw
  To: gentoo-user

[-- Attachment #1: Type: text/plain, Size: 1354 bytes --]

Am Donnerstag, 19. Juni 2008 schrieb ext Matt Harrison:

> And in reply to Alan's post, no I'm not (intentionally) useing BL2.
> Nothing is marked unstable on this machine with the example of a ruby
> module i quite like.
>
> This was the problem before, if I let emerge upgrade mdadm and lvm2, its
> starts trying to run BL2 initscripts and complaining that I'm not using
> BL2.

Well, then upgrade to BL2! And upgrade all the other packages as well. You 
only have to keep in mind to tell the initscripts that your RAID and LVM 
devices need to be activated, too.

This is done in /etc/rc.conf, like

rc_<this_init_script>_need="<another_init_script>"

In my case, I need to activate EVMS volumes, so I let fsck depend on it:

rc_fsck_need="evms"

On my laptop, which also has its disks encrypted, I have this:

rc_dmcrypt_need="evms"
rc_fsck_need="dmcrypt"

For you, this would be (guess)

rc_lvm_need="mdadm" (or whatever this init script is called)
rc_fsck_need="lvm"

HTH...

	Dirk
-- 
Dirk Heinrichs          | Tel:  +49 (0)162 234 3408
Configuration Manager   | Fax:  +49 (0)211 47068 111
Capgemini Deutschland   | Mail: dirk.heinrichs@capgemini.com
Wanheimerstraße 68      | Web:  http://www.capgemini.com
D-40468 Düsseldorf      | ICQ#: 110037733
GPG Public Key C2E467BB | Keyserver: wwwkeys.pgp.net

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: [gentoo-user] Upgrading without breaking LVM
  2008-06-19  5:33         ` Dirk Heinrichs
@ 2008-06-19 14:20           ` Matt Harrison
  2008-06-19 14:24             ` Neil Bothwick
  0 siblings, 1 reply; 15+ messages in thread
From: Matt Harrison @ 2008-06-19 14:20 UTC (permalink / raw
  To: gentoo-user

[-- Attachment #1: Type: text/plain, Size: 1157 bytes --]

On Thu, Jun 19, 2008 at 07:33:11AM +0200, Dirk Heinrichs wrote:
> Well, then upgrade to BL2! And upgrade all the other packages as well. You 
> only have to keep in mind to tell the initscripts that your RAID and LVM 
> devices need to be activated, too.

Isn't BL2 masked ~arch? I think it was when I originally ran into this problem.
Doesn't that mean that my system will upgrade stable packages that MUST be run
using a package that I have to unmask?

That sounds a little like a bug if I'm brutally honest. If this is the case I
would have at least liked an annoucement or something.

On the other hand, I don't think this is case as I've installed a test system
copying my setup as closely as possible, and it isn't having the same problems.
There must be something particular to the setup on that machine that is causing
this but I don't have the knowledge to tell what, and until I can pin it down
I'm not going to submit a bug report as it would be unreproducable. 

I'll look into BL2 and practice upgrading it, if I can satisfy myself that I can
get it working ok I'll consider it for the production machine.

Thanks

Matt

[-- Attachment #2: Type: application/pgp-signature, Size: 197 bytes --]

^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: [gentoo-user] Upgrading without breaking LVM
  2008-06-19 14:20           ` Matt Harrison
@ 2008-06-19 14:24             ` Neil Bothwick
  2008-06-19 14:50               ` Matt Harrison
  0 siblings, 1 reply; 15+ messages in thread
From: Neil Bothwick @ 2008-06-19 14:24 UTC (permalink / raw
  To: gentoo-user

[-- Attachment #1: Type: text/plain, Size: 580 bytes --]

On Thu, 19 Jun 2008 15:20:08 +0100, Matt Harrison wrote:

> Isn't BL2 masked ~arch? I think it was when I originally ran into this
> problem. Doesn't that mean that my system will upgrade stable packages
> that MUST be run using a package that I have to unmask?

No. The BL2 init script provided with LVM must be used when running BL2,
they should not be used with BL1. If you are seeing the warning messages
it is because you have added the lvm init script to your runlevel when it
should not be there.


-- 
Neil Bothwick

I@love~my,;It's%made in Taiwa~##$ ` #@

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 197 bytes --]

^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: [gentoo-user] Upgrading without breaking LVM
  2008-06-19 14:24             ` Neil Bothwick
@ 2008-06-19 14:50               ` Matt Harrison
  2008-06-23 20:32                 ` Matt Harrison
  0 siblings, 1 reply; 15+ messages in thread
From: Matt Harrison @ 2008-06-19 14:50 UTC (permalink / raw
  To: gentoo-user

[-- Attachment #1: Type: text/plain, Size: 1063 bytes --]

On Thu, Jun 19, 2008 at 03:24:29PM +0100, Neil Bothwick wrote:
> On Thu, 19 Jun 2008 15:20:08 +0100, Matt Harrison wrote:
> 
> > Isn't BL2 masked ~arch? I think it was when I originally ran into this
> > problem. Doesn't that mean that my system will upgrade stable packages
> > that MUST be run using a package that I have to unmask?
> 
> No. The BL2 init script provided with LVM must be used when running BL2,
> they should not be used with BL1. If you are seeing the warning messages
> it is because you have added the lvm init script to your runlevel when it
> should not be there.

Thats ok, I can understand that. I'm just about ready to give up on this as its
obviously too hard to debug. If upgrading to BL2 will fix my problems then
that's what I must do.

I can't test this tho as I haven't been able to break my test system with the
updates. I just hope that BL2 will fix this and not just introduce further
problems.

I'll consider it and post back my results. Fingers and toes crossed that BL2
solves it :)


Thanks

Matt

[-- Attachment #2: Type: application/pgp-signature, Size: 197 bytes --]

^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: [gentoo-user] Upgrading without breaking LVM
  2008-06-19 14:50               ` Matt Harrison
@ 2008-06-23 20:32                 ` Matt Harrison
  2008-06-23 23:18                   ` Neil Bothwick
  0 siblings, 1 reply; 15+ messages in thread
From: Matt Harrison @ 2008-06-23 20:32 UTC (permalink / raw
  To: gentoo-user

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Sorry for replying to my own message, just had a thought. Could this all
be a udev rules problem? I mean could some new rules in the updated udev
package be causing my lvm/raid devices to not be picked up and
activated?

I've been trying to break a vmware install that mirrors my problem
machine as closely as possible but it just won't die. Maybe there's
something particular to the hardware on the main machine that udev isn't
liking?

Thanks
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (MingW32)

iEYEARECAAYFAkhgCGEACgkQxNZfa+YAUWG6OQCgnrVedUw0Aj88d+HM98tuIWmw
FEAAn1nofMr3PaveYC9MZXKHGMMk1Ukz
=PcDX
-----END PGP SIGNATURE-----
-- 
gentoo-user@lists.gentoo.org mailing list



^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: [gentoo-user] Upgrading without breaking LVM
  2008-06-23 20:32                 ` Matt Harrison
@ 2008-06-23 23:18                   ` Neil Bothwick
  2008-06-23 23:40                     ` Matt Harrison
  0 siblings, 1 reply; 15+ messages in thread
From: Neil Bothwick @ 2008-06-23 23:18 UTC (permalink / raw
  To: gentoo-user

[-- Attachment #1: Type: text/plain, Size: 462 bytes --]

On Mon, 23 Jun 2008 21:32:33 +0100, Matt Harrison wrote:

> Sorry for replying to my own message, just had a thought. Could this all
> be a udev rules problem? I mean could some new rules in the updated udev
> package be causing my lvm/raid devices to not be picked up and
> activated?

Are your LVM partitions marked accordingly in (c)fdisk?


-- 
Neil Bothwick

There's no such thing as a free lunch
  ___Steve Ballmer, choking on a linuxburger

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 197 bytes --]

^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: [gentoo-user] Upgrading without breaking LVM
  2008-06-23 23:18                   ` Neil Bothwick
@ 2008-06-23 23:40                     ` Matt Harrison
  2008-06-24  7:37                       ` Alan McKinnon
  0 siblings, 1 reply; 15+ messages in thread
From: Matt Harrison @ 2008-06-23 23:40 UTC (permalink / raw
  To: gentoo-user

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Neil Bothwick wrote:
| On Mon, 23 Jun 2008 21:32:33 +0100, Matt Harrison wrote:
|
|> Sorry for replying to my own message, just had a thought. Could this all
|> be a udev rules problem? I mean could some new rules in the updated udev
|> package be causing my lvm/raid devices to not be picked up and
|> activated?
|
| Are your LVM partitions marked accordingly in (c)fdisk?
|
|

No they're marked as Linux Raid Autodetect, as the lvm is on top of
raid. Is this an issue?

- --
Matt Harrison
iwasinnamuknow@genestate.com
http://mattharrison.org
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (MingW32)

iEYEARECAAYFAkhgNIoACgkQxNZfa+YAUWHt1wCfc53an5LVjN+wXdZlNdbl4wPU
SPsAoKH3vL6hWq5GYSptQHymn23QISpb
=2Gr3
-----END PGP SIGNATURE-----
-- 
gentoo-user@lists.gentoo.org mailing list



^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: [gentoo-user] Upgrading without breaking LVM
  2008-06-23 23:40                     ` Matt Harrison
@ 2008-06-24  7:37                       ` Alan McKinnon
  0 siblings, 0 replies; 15+ messages in thread
From: Alan McKinnon @ 2008-06-24  7:37 UTC (permalink / raw
  To: gentoo-user

On Tuesday 24 June 2008, Matt Harrison wrote:
> Neil Bothwick wrote:
> | On Mon, 23 Jun 2008 21:32:33 +0100, Matt Harrison wrote:
> |> Sorry for replying to my own message, just had a thought. Could
> |> this all be a udev rules problem? I mean could some new rules in
> |> the updated udev package be causing my lvm/raid devices to not be
> |> picked up and activated?
> |
> | Are your LVM partitions marked accordingly in (c)fdisk?
>
> No they're marked as Linux Raid Autodetect, as the lvm is on top of
> raid. Is this an issue?

For lvm2 it is fine. It's raid that is using the native device, so you 
have it tagged correctly.



-- 
Alan McKinnon
alan dot mckinnon at gmail dot com

-- 
gentoo-user@lists.gentoo.org mailing list



^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: [gentoo-user] Upgrading without breaking LVM
  2008-06-18 17:41 [gentoo-user] Upgrading without breaking LVM Matt Harrison
  2008-06-18 20:46 ` Florian Philipp
@ 2008-06-24 14:32 ` Daniel Iliev
  1 sibling, 0 replies; 15+ messages in thread
From: Daniel Iliev @ 2008-06-24 14:32 UTC (permalink / raw
  To: gentoo-user

On Wed, 18 Jun 2008 18:41:12 +0100
Matt Harrison <iwasinnamuknow@genestate.com> wrote:

> 
> Hi all,
> 
> Some of you may remember my problems with lvm after an update in 
> http://www.archivum.info/gentoo-user@lists.gentoo.org/2008-04/msg00899.html
> 
> I'm now headed back towards the same situation. I have a load of
> updates to do according to emerge -DNavu world but I've had to mask
> all upgraded versions of udev, mdadm and lvm as every time I upgrade
> them, it breaks my disk access.
> 
> Whenever I upgrade these packages (I'm not which one of those 3 is
> actually to blame, and I don't want to chance it) I can no longer
> access the lvm volumes on my disks. I have no idea what is causing it.
> 
> This means I cannot do a world update as those manually masked
> packages are blocking a lot of other packages. 
> 
> A few months on, is anyone able to shed light on why updating any of
> those 3 packages causes my system to be unbootable and my data to be
> inaccessible?
> 
> I would really like to keep my system up to date, but as you can
> understand, I'm loathe to make changes that will give me a borked
> system yet again.
> 
> Grateful for advice anyone can give as my system is now getting quite
> out of date.
> 
> Thanks
> 
> Matt Harrison


Please, make it clear: which versions of all relevant packages are you
using?

emerge --info | grep ACC
emerge --info | grep baselayout
emerge -pv lvm2 udev baselayout mdadm
emerge -DuNpv world


-- 
Best regards,
Daniel
-- 
gentoo-user@lists.gentoo.org mailing list



^ permalink raw reply	[flat|nested] 15+ messages in thread

end of thread, other threads:[~2008-06-24 14:32 UTC | newest]

Thread overview: 15+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2008-06-18 17:41 [gentoo-user] Upgrading without breaking LVM Matt Harrison
2008-06-18 20:46 ` Florian Philipp
2008-06-18 21:01   ` Matt Harrison
2008-06-18 20:25     ` Alan McKinnon
2008-06-18 21:48     ` Florian Philipp
2008-06-18 22:15       ` Matt Harrison
2008-06-19  5:33         ` Dirk Heinrichs
2008-06-19 14:20           ` Matt Harrison
2008-06-19 14:24             ` Neil Bothwick
2008-06-19 14:50               ` Matt Harrison
2008-06-23 20:32                 ` Matt Harrison
2008-06-23 23:18                   ` Neil Bothwick
2008-06-23 23:40                     ` Matt Harrison
2008-06-24  7:37                       ` Alan McKinnon
2008-06-24 14:32 ` Daniel Iliev

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox