[gentoo-user] Hints for using Unison?

[gentoo-user] Hints for using Unison?

[Mark Knecht]

Hi,
   I'm wondering if Unison is a good tool for helping me make sure the
two copies of our music library are consistent?

Local: /dev/sda1 mounted at /home/mark/music

Remote: dragonfly:/Musiclib NFS mounted at /mnt/Musiclib

   These two directories started off identical at one point about 4-6
months ago but have been slowly diverging as my wife and I add things
to each one. I'd now like them the get back into sync and keep them
that way.

   One possibly tricky part about this will be that in some cases we
have found bad rips and have reripped files to fix that. In this case
there is going to be a newer file in each either location with the
same name but with a new size & date. Will Unison give me an option to
just accept the newer one in each location and remove or backup the
older one automatically?

   This is likely to be a sort of long process I expect. The
directories are about 50GB of ogg files, although I hope there isn't
that much copying as probably 99% of it is identical.

Thanks in advance,
Mark

-- 
gentoo-user@gentoo.org mailing list

Re: [gentoo-user] Hints for using Unison?

[Neil Bothwick]

[-- Attachment #1: Type: text/plain, Size: 757 bytes --]

On Sun, 31 Jul 2005 13:43:16 -0700, Mark Knecht wrote:

>    One possibly tricky part about this will be that in some cases we
> have found bad rips and have reripped files to fix that. In this case
> there is going to be a newer file in each either location with the
> same name but with a new size & date. Will Unison give me an option to
> just accept the newer one in each location and remove or backup the
> older one automatically?

Unison is particularly good at handling this sort of situation. It keeps
a log of the file dates each time you sync and asks for manual
intervention when a file have been updated on both sides since the last
sync.


-- 
Neil Bothwick

Did you know that eskimos have 17 different words for linguist?

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-user] Hints for using Unison?

[Mark Knecht]

On 7/31/05, Neil Bothwick <neil@digimed.co.uk> wrote:
> On Sun, 31 Jul 2005 13:43:16 -0700, Mark Knecht wrote:
> 
> >    One possibly tricky part about this will be that in some cases we
> > have found bad rips and have reripped files to fix that. In this case
> > there is going to be a newer file in each either location with the
> > same name but with a new size & date. Will Unison give me an option to
> > just accept the newer one in each location and remove or backup the
> > older one automatically?
> 
> Unison is particularly good at handling this sort of situation. It keeps
> a log of the file dates each time you sync and asks for manual
> intervention when a file have been updated on both sides since the last
> sync.
> 

Hi Neil,
   OK, so I tried Unison and, twice, it just gets stuck at the same
file. I'm running it like this:

unison /home/mark/music /mnt/Musiclib 

The gui comes up and the program gets started but then it just hangs.
There's no obvious network activity or local disk activity. If I let
the program sit long enough for the screensaver to kick in then when I
unlock the screen the program is just a grey box.

So far I cannot even kill the thing. kill -9 pid or killall -9 unison
act like they killed it but ps aux says the process is still there.
It's even there if I try killing the gui in Gnome. The gui goes away
but the process persists.

Any ideas on what I'm doing wrong?

Thanks,
Mark

-- 
gentoo-user@gentoo.org mailing list

Re: [gentoo-user] Hints for using Unison?

[Mark Knecht]

On 7/31/05, Mark Knecht <markknecht@gmail.com> wrote:
> 
> So far I cannot even kill the thing. kill -9 pid or killall -9 unison
> act like they killed it but ps aux says the process is still there.
> It's even there if I try killing the gui in Gnome. The gui goes away
> but the process persists.
> 
> Any ideas on what I'm doing wrong?
> 
> Thanks,
> Mark
> 

Even worse, I cannot even reboot the machine after this event. I have
to hit the reset button. Bummer.

Possibly this problem is really an NFS issue or a 1394 issue? Neither
of these are overly tested, at least on my system under Gentoo. I ran
the same system with FC2 for quite awhile. 1394 worked fine as far as
I could tell, but I never used NFS much and I only jsut set up NFS
today to try out Unison so maybe it's one of those systems causing the
problem?

So far I haven't found a log file, most probably because the program
crashes before it can do anything.

Cheers,
Mark

-- 
gentoo-user@gentoo.org mailing list

Re: [gentoo-user] Hints for using Unison?

[Glenn Enright]

[-- Attachment #1: Type: text/plain, Size: 649 bytes --]

On Mon, 01 Aug 2005 11:52, Mark Knecht wrote:
> On 7/31/05, Mark Knecht <markknecht@gmail.com> wrote:

> Possibly this problem is really an NFS issue or a 1394 issue? Neither
> of these are overly tested, at least on my system under Gentoo. I ran
> the same system with FC2 for quite awhile. 1394 worked fine as far as
> I could tell, but I never used NFS much and I only jsut set up NFS
> today to try out Unison so maybe it's one of those systems causing the
> problem?

Is the NFS filesystem mounted read-only?
-- 

Speak the truth.  That is always much easier,
and is often the most powerful argument.

  -- Bene Gesserit Axiom

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-user] Hints for using Unison?

[Mark Knecht]

On 7/31/05, Glenn Enright <elinar@ihug.co.nz> wrote:
> On Mon, 01 Aug 2005 11:52, Mark Knecht wrote:
> > On 7/31/05, Mark Knecht <markknecht@gmail.com> wrote:
> 
> > Possibly this problem is really an NFS issue or a 1394 issue? Neither
> > of these are overly tested, at least on my system under Gentoo. I ran
> > the same system with FC2 for quite awhile. 1394 worked fine as far as
> > I could tell, but I never used NFS much and I only jsut set up NFS
> > today to try out Unison so maybe it's one of those systems causing the
> > problem?
> 
> Is the NFS filesystem mounted read-only?
> --

No Glenn, it's mounted read/write and I tested that I Could write it
from the machine running Unison.

I tried running Unison on a directory containing a single CD. It hung
up on that also. After the hang up the machine is really unhappy
rebooting. It gives me messages about being unable to unmoust the NFS
filesystems.

Thanks,
Mark

thanks,
Mark

-- 
gentoo-user@gentoo.org mailing list

Re: [gentoo-user] Hints for using Unison?

[Glenn Enright]

[-- Attachment #1: Type: text/plain, Size: 1493 bytes --]

On Mon, 01 Aug 2005 17:16, Mark Knecht wrote:
> On 7/31/05, Glenn Enright <elinar@ihug.co.nz> wrote:
> > On Mon, 01 Aug 2005 11:52, Mark Knecht wrote:
> > > On 7/31/05, Mark Knecht <markknecht@gmail.com> wrote:
> > >
> > > Possibly this problem is really an NFS issue or a 1394 issue? Neither
> > > of these are overly tested, at least on my system under Gentoo. I ran
> > > the same system with FC2 for quite awhile. 1394 worked fine as far as
> > > I could tell, but I never used NFS much and I only jsut set up NFS
> > > today to try out Unison so maybe it's one of those systems causing the
> > > problem?
> >
> > Is the NFS filesystem mounted read-only?
> > --
>
> No Glenn, it's mounted read/write and I tested that I Could write it
> from the machine running Unison.
>
> I tried running Unison on a directory containing a single CD. It hung
> up on that also. After the hang up the machine is really unhappy
> rebooting. It gives me messages about being unable to unmoust the NFS
> filesystems.
>
> Thanks,
> Mark
>
> thanks,
> Mark

Ok just a thought :). Last time I had a dodgy program Zac Medico suggested I 
run strace on it, so in your case from a command prompt...
	# strace unison dir1 dir2

Does using a different terminal program make any difference? maybee using 
screen?. Not an expert here you understand just trying to fit peices in the 
puzzle.
-- 

"Life, loathe it or ignore it, you can't like it."
-- Marvin the paranoid android

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-user] Hints for using Unison?

[Neil Bothwick]

[-- Attachment #1: Type: text/plain, Size: 677 bytes --]

On Sun, 31 Jul 2005 16:52:15 -0700, Mark Knecht wrote:

> Possibly this problem is really an NFS issue or a 1394 issue? Neither
> of these are overly tested, at least on my system under Gentoo. I ran
> the same system with FC2 for quite awhile. 1394 worked fine as far as
> I could tell, but I never used NFS much and I only jsut set up NFS
> today to try out Unison so maybe it's one of those systems causing the
> problem?

You don't use NFS with Unison, give it hostnames and it will copy ther
files with SSH. 

e.g. unison this box:/home/somedir ssh://otherbox/home/somedir


-- 
Neil Bothwick

I couldn't repair your brakes, so I made your horn louder.

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-user] Hints for using Unison?

[Mark Knecht]

On 8/1/05, Neil Bothwick <neil@digimed.co.uk> wrote:
> On Sun, 31 Jul 2005 16:52:15 -0700, Mark Knecht wrote:
> 
> > Possibly this problem is really an NFS issue or a 1394 issue? Neither
> > of these are overly tested, at least on my system under Gentoo. I ran
> > the same system with FC2 for quite awhile. 1394 worked fine as far as
> > I could tell, but I never used NFS much and I only jsut set up NFS
> > today to try out Unison so maybe it's one of those systems causing the
> > problem?
> 
> You don't use NFS with Unison, give it hostnames and it will copy ther
> files with SSH.
> 
> e.g. unison this box:/home/somedir ssh://otherbox/home/somedir
> 

Neil,
   Thanks. This is helping me get to the root cause. It appears that
the real problem has something to do with non-standard characters for
American English. The first directory that was failing was a Spanish
Edition of a CD. When I went and looked at it there were three songs
that used different sorts of accent character over vowels. I changed
these characters on  both machines and then unison got further, so I
changed a second CD and it got further yet. Unfortunately I've been at
this about an hour and am just finishing up changing the "A"'s so I'd
like to try and understand what the issue here really is ad find a fix
that doesn't require making any changes.

   The failure in unison is a dialog box with this message:

Uncaught exception Glib.convert.error(1, "Invalid byte sequence in
conversion input")

This failure is from the name "Angelique Kidjo" where a forward accent
appears over the first 'e' in Angelique. When I remove this unison
goes further but fails for more stuff like this.

   I use a program called Aqualung (not in portage unfortunately) to
play these directories and it doesn't have any trouble with these
non-standard accent characters so why does unison?

   One last thing. I note that when I change these characters on the
local machine that it takes one backspace in a terminal to erase the
character, while on the remote machine (logged in through ssh like
unison would be) it takes two backspaces to change the character. Does
this mean anything to you? It's beyond me. Note that locales are
probably not identical between the two machines.

   Again, thanks to you and the other folks who responded. At least I
could probably get past this problem with a couple of days of work but
I'm hoping for a more internationalized solution.

Cheers,
Mark

-- 
gentoo-user@gentoo.org mailing list

Re: [gentoo-user] Hints for using Unison?

[Mark Knecht]

On 8/1/05, Mark Knecht <markknecht@gmail.com> wrote:
> On 8/1/05, Neil Bothwick <neil@digimed.co.uk> wrote:
> > On Sun, 31 Jul 2005 16:52:15 -0700, Mark Knecht wrote:
> >
> > > Possibly this problem is really an NFS issue or a 1394 issue? Neither
> > > of these are overly tested, at least on my system under Gentoo. I ran
> > > the same system with FC2 for quite awhile. 1394 worked fine as far as
> > > I could tell, but I never used NFS much and I only jsut set up NFS
> > > today to try out Unison so maybe it's one of those systems causing the
> > > problem?
> >
> > You don't use NFS with Unison, give it hostnames and it will copy ther
> > files with SSH.
> >
> > e.g. unison this box:/home/somedir ssh://otherbox/home/somedir
> >
> 
> Neil,
>    Thanks. This is helping me get to the root cause. It appears that
> the real problem has something to do with non-standard characters for
> American English. 

Maybe the problem is only with the GUI of Unison? I tried ssh'ing into
my machine and running that way since it doesn't use the GUI and keeps
a record of what's going on in the terminal. This is not failing for
any of the non-standard American english characters.

Possibly time for a bug report?

Thanks,
Mark

-- 
gentoo-user@gentoo.org mailing list

[gentoo-user] Re: Hints for using Unison?

[Moshe Kaminsky]

[-- Attachment #1: Type: text/plain, Size: 2273 bytes --]

Hi,

* Mark Knecht <markknecht@gmail.com> [01/08/05 02:42]:
> 
> On 7/31/05, Neil Bothwick <neil@digimed.co.uk> wrote:
> > On Sun, 31 Jul 2005 13:43:16 -0700, Mark Knecht wrote:
> > 
> > >    One possibly tricky part about this will be that in some cases we
> > > have found bad rips and have reripped files to fix that. In this case
> > > there is going to be a newer file in each either location with the
> > > same name but with a new size & date. Will Unison give me an option to
> > > just accept the newer one in each location and remove or backup the
> > > older one automatically?
> > 
> > Unison is particularly good at handling this sort of situation. It keeps
> > a log of the file dates each time you sync and asks for manual
> > intervention when a file have been updated on both sides since the last
> > sync.
> > 
> 
> Hi Neil,
>    OK, so I tried Unison and, twice, it just gets stuck at the same
> file. I'm running it like this:
> 
> unison /home/mark/music /mnt/Musiclib 
> 
> The gui comes up and the program gets started but then it just hangs.
> There's no obvious network activity or local disk activity. If I let
> the program sit long enough for the screensaver to kick in then when I
> unlock the screen the program is just a grey box.
> 
> So far I cannot even kill the thing. kill -9 pid or killall -9 unison
> act like they killed it but ps aux says the process is still there.
> It's even there if I try killing the gui in Gnome. The gui goes away
> but the process persists.
> 
> Any ideas on what I'm doing wrong?

This is just a guess, but I know that unison is using some lock files, 
and there might be problems with this on nfs mounted directories. I 
think the problem might be solved if you run unison on both machines, 
rather than using the mounted directory. If that's impossible, you might 
be able to change the location of the lock files by changing the root to 
a local directory.

Moshe

> 
> Thanks,
> Mark
> 
> -- 
> gentoo-user@gentoo.org mailing list
> 

-- 
I love deadlines. I like the whooshing sound they make as they fly by. 
                                        -- Douglas Adams
    
    Moshe Kaminsky <kaminsky@math.huji.ac.il>
    Home: 08-9456841


[-- Attachment #2: Type: application/pgp-signature, Size: 198 bytes --]