[gentoo-user] Using losetup & loopback devices / filesystems - changes aren't written to disk?!?!?

[Stroller <stroller@stellar.eclipse.co.uk>]

Ok,

I'm really new to loopback devices / filesystems, so please forgive me  
if I'm doing something stupid here. It wouldn't be the first time. :/

I've created this loopback filesystem, by using `dd if=/dev/zero  
of=individual.files.img` (so named because I'm hope to copy a bunch of  
"individual files" onto it) so that it is 65gig in size and I've  
attached it to a loopback device and formatted it with a filesystem.  
So far all seems good and I can write files to it and seem to be able  
to read the changes & stuff.

I've mounted CD .iso images loopback in the past to access the files  
on them, but this is the first time I've created one from scratch and  
written files to it, so I was actually quite chuffed when fsck  
actually performed its magic.

This kinda illustrates the problem I'm having:

$ date
Sun Dec  6 23:54:32 GMT 2009
$ ls -lh individual.files.img
-rw-rw-rw- 1 stroller users 65G 2009-12-06 23:36 individual.files.img
$ ls -l mnttmp/
total 0
$ sudo losetup -a
$ sudo losetup /dev/loop0 individual.files.img
$ sudo losetup -a
/dev/loop/0: [0811]:132155 (/path/to/individual.files.i*)
$ sudo ntfs-3g /dev/loop0 mnttmp/
$ ls -l mnttmp/
total 1
-rwxrwxrwx 1 root root 11 2009-12-06 23:52 foo
$ cat mnttmp/foo
wowow wow
$ echo bar >> mnttmp/foo
$ cat mnttmp/foo
wowow wow
bar
$ sudo sync
$ sudo umount -v mnttmp/
/dev/loop0 umounted
$ ls -l mnttmp/
total 0
$ sudo sync
$ sudo losetup -d /dev/loop/0
$ sudo sync
$ ls -lh individual.files.img
-rw-rw-rw- 1 stroller users 65G 2009-12-06 23:36 individual.files.img
$ date
Sun Dec  6 23:57:05 GMT 2009
$ sudo losetup -a
$


So ISTM that the above proves that the filesystem *thinks* it's  
written the file to disk (because I can modify the file contents and  
read back the changes), but that the loopback file hasn't actually  
been written, *because the file modification date doesn't change*.

I'm really seeing the evidence of this, because individual.files.img  
is shared over NFS and I then try to mount the image loopback on  
another machine (running System Rescue CD) it doesn't see the changes  
I've made. If I reattach the image loopback on the first system again,  
the changes *are* still shown. I figured that the system must  
"somehow" be caching the results, so a little Google (is a dangerous  
thing) and that's what led me to try running sync repeatedly to try  
flushing to disk. But as you can see, that's not helping.

The filesystem on which individual.files.img is stored is running on a  
3ware RAID controller, but that is set with caching off (for this array)

I'm pretty much stumbling around in the dark here, so any pointers  
would be very gratefully received. I feel like I could spent a load of  
time trying different things and not have any luck. For instance I've  
just noticed that /etc/exports I have (async) set as an option, so  
maybe that's preventing the second system from seeing the changes on  
the NGS mount; but then if the file's timestamp hasn't changed, then  
there surely won't be any changes to read. It's like the computer is  
only _pretending_ to write out to the loopback device.

Anyway: I'm an idiot, one of the experts on the list will surely see  
in an instant what obvious thing I'm overlooking.

Thanks,

Stroller.