[gentoo-user] Howto share Linux swap partition with Windows XP

[gentoo-user] Howto share Linux swap partition with Windows XP

[Grant Edwards]

For reasons that probably aren't interesting, I needed to
eliminate the Windows XP swap file from the "C:" NTFS parition
of one of my multi-boot machines [no matter how cheap and fast
disks get, partitions still seem to fill up, and backups still
take too long]. 

That machine had a 2GB Linux swap partition. After googling for
a while, I found a very slick solution that lets Windows XP use
a Linux swap partition for swap/paging/vm/whatever-MS-calls-it:

  http://db.bme.hu/~surprof/SwapFs-i/

Maybe others who do a lot of multi-booting all know about
swapfs, but my finding it took quite a bit of searching and
chasing down useless google hits.  Next time I need to find it,
hopefully I'll be able to find this posting...

-- 
Grant

Re: [gentoo-user] Howto share Linux swap partition with Windows XP

[Stroller]

On 23 Jan 2009, at 05:16, Grant Edwards wrote:
> ... I found a very slick solution that lets Windows XP use
> a Linux swap partition for swap/paging/vm/whatever-MS-calls-it:
>
>  http://db.bme.hu/~surprof/SwapFs-i/

That looks a really cool & useful idea.

However, I have a reservation. Since you NEED to use it - perhaps for  
space considerations? - the go for it, by all means. But if this  
driver is slower at reads or writes than Windows' own NTFS driver then  
it may actually slow the computer down.

In the case of a driver written by an individual, who is likely not as  
familiar with Windows' APIs, or who may not be able to use the private  
APIs used by Windows' own filesystem drivers, I find it quite possible  
the performance may be questioned.

Stroller.

[gentoo-user] Re: Howto share Linux swap partition with Windows XP

[Grant Edwards]

On 2009-01-23, Stroller <stroller@stellar.eclipse.co.uk> wrote:
>
> On 23 Jan 2009, at 05:16, Grant Edwards wrote:
>> ... I found a very slick solution that lets Windows XP use
>> a Linux swap partition for swap/paging/vm/whatever-MS-calls-it:
>>
>>  http://db.bme.hu/~surprof/SwapFs-i/
>
> That looks a really cool & useful idea.
>
> However, I have a reservation. Since you NEED to use it -
> perhaps for space considerations?

Yup.  Mainly because I use ntfsclone to keep a bunch of backup
copies of the NTFS partition, and having a 2GB swap file in
every backup copy starts to eat up a lot of disk space.

> - the go for it, by all means. But if this driver is slower at
> reads or writes than Windows' own NTFS driver then it may
> actually slow the computer down.

Could be.  My understanding is that the system is still using
the Windows NTFS driver.  I believe that swapfs is a
block-device filter driver that sits between the NTFS driver
and the bottom layer block device driver (the IDE driver or the
SATA driver or the SCSI driver).

> In the case of a driver written by an individual, who is
> likely not as familiar with Windows' APIs, or who may not be
> able to use the private APIs used by Windows' own filesystem
> drivers, I find it quite possible the performance may be
> questioned.

Well, there are several individuals who've worked on it.  That
said, I can't swear that there aren't performance implications.
There probably are, since it adds a layer between the SATA
driver (in my case) and the NTFS driver.  But, I haven't
noticed any visible slow-down, and the machine is more than
fast enough for my purposes.

Besides, if you're running MS Windows, you've already lost the
war when it comes to swapping performance: anybody who cares
about swap performance wouldn't be using a normal file on a
normal filesystem for it...

Someday when I'm bored, maybe I'll google for a file I/O
benchmark for windows and run it with and without the swapfs
layer.

-- 
Grant Edwards                   grante             Yow! INSIDE, I have the
                                  at               same personality disorder
                               visi.com            as LUCY RICARDO!!

Re: [gentoo-user] Re: Howto share Linux swap partition with Windows XP

[Stroller]

On 23 Jan 2009, at 14:58, Grant Edwards wrote:

> On 2009-01-23, Stroller <stroller@stellar.eclipse.co.uk> wrote:
>>
>> On 23 Jan 2009, at 05:16, Grant Edwards wrote:
>>> ... I found a very slick solution that lets Windows XP use
>>> a Linux swap partition for swap/paging/vm/whatever-MS-calls-it:
>>>
>>> http://db.bme.hu/~surprof/SwapFs-i/
>>
>> That looks a really cool & useful idea.
>>
>> However, I have a reservation. Since you NEED to use it -
>> perhaps for space considerations?
>
> Yup.  Mainly because I use ntfsclone to keep a bunch of backup
> copies of the NTFS partition, and having a 2GB swap file in
> every backup copy starts to eat up a lot of disk space.

It might be possible to script removing the swap file at shutdown (or  
place a wrapper script to mount the partition & remove the swapfile  
before running ntfsclone). But I appreciate this is less elegant than  
just using the same swap partition  for both o/s.

Stroller.

[gentoo-user] Re: Howto share Linux swap partition with Windows XP

[Grant Edwards]

On 2009-01-23, Stroller <stroller@stellar.eclipse.co.uk> wrote:
> On 23 Jan 2009, at 14:58, Grant Edwards wrote:
>> On 2009-01-23, Stroller <stroller@stellar.eclipse.co.uk> wrote:
>>> On 23 Jan 2009, at 05:16, Grant Edwards wrote:
>>>> ... I found a very slick solution that lets Windows XP use
>>>> a Linux swap partition for swap/paging/vm/whatever-MS-calls-it:
>>>>
>>>> http://db.bme.hu/~surprof/SwapFs-i/
>>>
>>> That looks a really cool & useful idea.
>>>
>>> However, I have a reservation. Since you NEED to use it -
>>> perhaps for space considerations?
>>
>> Yup.  Mainly because I use ntfsclone to keep a bunch of backup
>> copies of the NTFS partition, and having a 2GB swap file in
>> every backup copy starts to eat up a lot of disk space.
>
> It might be possible to script removing the swap file at shutdown (or  
> place a wrapper script to mount the partition & remove the swapfile  
> before running ntfsclone).

I thought about that.  It probably would be a bit more robust
than using the Linux swap partition, but it seemed too much
like giving in to Microsoft. :)

> But I appreciate this is less elegant than just using the same
> swap partition for both o/s.

-- 
Grant Edwards                   grante             Yow! Everybody is going
                                  at               somewhere!!  It's probably
                               visi.com            a garage sale or a disaster
                                                   Movie!!

Re: [gentoo-user] Re: Howto share Linux swap partition with Windows XP

[Paul Hartman]

On Fri, Jan 23, 2009 at 10:08 AM, Stroller
<stroller@stellar.eclipse.co.uk> wrote:
>
> On 23 Jan 2009, at 14:58, Grant Edwards wrote:
>
>> On 2009-01-23, Stroller <stroller@stellar.eclipse.co.uk> wrote:
>>>
>>> On 23 Jan 2009, at 05:16, Grant Edwards wrote:
>>>>
>>>> ... I found a very slick solution that lets Windows XP use
>>>> a Linux swap partition for swap/paging/vm/whatever-MS-calls-it:
>>>>
>>>> http://db.bme.hu/~surprof/SwapFs-i/
>>>
>>> That looks a really cool & useful idea.
>>>
>>> However, I have a reservation. Since you NEED to use it -
>>> perhaps for space considerations?
>>
>> Yup.  Mainly because I use ntfsclone to keep a bunch of backup
>> copies of the NTFS partition, and having a 2GB swap file in
>> every backup copy starts to eat up a lot of disk space.
>
> It might be possible to script removing the swap file at shutdown (or place
> a wrapper script to mount the partition & remove the swapfile before running
> ntfsclone). But I appreciate this is less elegant than just using the same
> swap partition  for both o/s.

http://support.microsoft.com/kb/314834

There is a registry setting in Windows to "clear" the pagefile.sys at
shutdown. What does "clear" mean? To overwrite with 0? To delete? I
don't know.

Paul

Re: [gentoo-user] Re: Howto share Linux swap partition with Windows XP

[Stroller]

On 23 Jan 2009, at 17:09, Paul Hartman wrote:
> ...
> http://support.microsoft.com/kb/314834
>
> There is a registry setting in Windows to "clear" the pagefile.sys at
> shutdown. What does "clear" mean? To overwrite with 0? To delete? I
> don't know.

 From memory it's just to delete it, which is perfect.

It would take too long to zero it out - I don't think that's the  
purpose. Instead, I think, it should prevent swapfile fragmentation -  
making it a very good general-purpose setting to enable.

What would be really idea for the OP is some kind of grub setting & a  
bash script that formats the partition to the appropriate format for  
the o/s being booted. But you'd have to be clever about it to avoid  
long boot times.

Stroller.

Re: [gentoo-user] Re: Howto share Linux swap partition with Windows XP

[Paul Hartman]

On Fri, Jan 23, 2009 at 2:53 PM, Stroller
<stroller@stellar.eclipse.co.uk> wrote:
>
> On 23 Jan 2009, at 17:09, Paul Hartman wrote:
>>
>> ...
>> http://support.microsoft.com/kb/314834
>>
>> There is a registry setting in Windows to "clear" the pagefile.sys at
>> shutdown. What does "clear" mean? To overwrite with 0? To delete? I
>> don't know.
>
> From memory it's just to delete it, which is perfect.
>
> It would take too long to zero it out - I don't think that's the purpose.
> Instead, I think, it should prevent swapfile fragmentation - making it a
> very good general-purpose setting to enable.
>
> What would be really idea for the OP is some kind of grub setting & a bash
> script that formats the partition to the appropriate format for the o/s
> being booted. But you'd have to be clever about it to avoid long boot times.
>
> Stroller.

After further googling, it appears it *does* fill the pagefile.sys
with zeros, and adds a significant delay to windows shutdown times. So
it won't do anything for the OP in this case.

Re: [gentoo-user] Re: Howto share Linux swap partition with Windows XP

[Stroller]

On 23 Jan 2009, at 21:10, Paul Hartman wrote:
>> ...
>> From memory it's just to delete it, which is perfect.
>>
>> It would take too long to zero it out - I don't think that's the  
>> purpose.
>> ...
>
> After further googling, it appears it *does* fill the pagefile.sys
> with zeros, and adds a significant delay to windows shutdown times. So
> it won't do anything for the OP in this case.

I don't know why I said "from memory" before, I was surely just making  
the assumption.

ISTM a bit daft, under Windows, to zero out the pagefile. If you have  
physical access to the computer, most anything in the swapfile will be  
available elsewhere on the hard-drive anyway. About the only thing you  
*might* get out of it is passwords, but that's not something for a  
very amateur hacker.

I guess writing the whole routine to (free up swap memory, check the  
registry for this setting &) zero the swapfile not to have been a mere  
5 minute job. How hard would it have been to add an option _just_ to  
delete it? This just requires freeing the inode, is surely less work,  
and would have been more useful to far more people. *sigh* Microsoft.

Stroller.

[gentoo-user] Re: Howto share Linux swap partition with Windows XP

[Grant Edwards]

On 2009-01-23, Stroller <stroller@stellar.eclipse.co.uk> wrote:
>
> On 23 Jan 2009, at 17:09, Paul Hartman wrote:
>> ...
>> http://support.microsoft.com/kb/314834
>>
>> There is a registry setting in Windows to "clear" the pagefile.sys at
>> shutdown. What does "clear" mean? To overwrite with 0? To delete? I
>> don't know.
>
>  From memory it's just to delete it, which is perfect.

That would eliminate the issue of a backup snapshot having 2GB
of pagefile.sys and 1.7GB of other stuff. 

> It would take too long to zero it out - I don't think that's
> the purpose. Instead, I think, it should prevent swapfile
> fragmentation - making it a very good general-purpose setting
> to enable.
>
> What would be really idea for the OP is some kind of grub
> setting & a bash script that formats the partition to the
> appropriate format for the o/s being booted. But you'd have to
> be clever about it to avoid long boot times.

I found some old postings from 6-8 years ago from people who
were trying to do that.  Nobody seemed to have come up with
anything that worked very well.  Since then disks have gotten
large enough that normal people don't care about a few GB.

-- 
Grant Edwards                   grante             Yow! I have many CHARTS
                                  at               and DIAGRAMS..
                               visi.com            

Re: [gentoo-user] Re: Howto share Linux swap partition with Windows XP

[Peter Humphrey]

On Friday 23 January 2009 14:58:32 Grant Edwards wrote:

> Mainly because I use ntfsclone to keep a bunch of backup copies of the
> NTFS partition, and having a 2GB swap file in every backup copy starts to
> eat up a lot of disk space. 

In the days when I ran Windows I used to have at least one partition other 
than C and force the swap file onto it, with fixed size. Then I could just 
omit that partition from the backup.

Perhaps it's still possible to do that; I don't know, but it might be worth 
a try.

-- 
Rgds
Peter

[gentoo-user] Re: Howto share Linux swap partition with Windows XP

[Grant Edwards]

On 2009-01-24, Peter Humphrey <peter@humphrey.ukfsn.org> wrote:
> On Friday 23 January 2009 14:58:32 Grant Edwards wrote:
>
>> Mainly because I use ntfsclone to keep a bunch of backup copies of the
>> NTFS partition, and having a 2GB swap file in every backup copy starts to
>> eat up a lot of disk space. 
>
> In the days when I ran Windows I used to have at least one
> partition other than C and force the swap file onto it, with
> fixed size. Then I could just omit that partition from the
> backup.
>
> Perhaps it's still possible to do that; I don't know, but it
> might be worth a try.

Yes, it's still possible to do that.  I didn't figure out I
_should_ do that until it was too late and the disk was
partitioned and several OSes installed -- I didn't have a spare
primary parition to put the swap file on.  I had a bunch of
spare extended partitions but all the docs say you can't put
the XP swap file on en extended paritition (unless you use
something like swapfs, which will work with an extended
partition).

-- 
Grant

Re: [gentoo-user] Re: Howto share Linux swap partition with Windows XP

[Peter Humphrey]

On Saturday 24 January 2009 15:35:32 Grant Edwards wrote:

> I didn't have a spare primary parition to put the swap file on.  I had a
> bunch of spare extended partitions but all the docs say you can't put the
> XP swap file on en extended paritition...

Ah, I didn't know that. In Win98, I think it was, I used to put it on drive 
E, which was a logical disk in the extended partition.

-- 
Rgds
Peter

[gentoo-user] Re: Howto share Linux swap partition with Windows XP

[Grant Edwards]

On 2009-01-24, Peter Humphrey <peter@humphrey.ukfsn.org> wrote:
> On Saturday 24 January 2009 15:35:32 Grant Edwards wrote:
>
>> I didn't have a spare primary parition to put the swap file
>> on.  I had a bunch of spare extended partitions but all the
>> docs say you can't put the XP swap file on en extended
>> paritition...
>
> Ah, I didn't know that. In Win98, I think it was, I used to
> put it on drive E, which was a logical disk in the extended
> partition.

I didn't actually try it, so maybe I was wrong -- but I swear I
read that somewhere (and it sounded like the sort of
restriction one would run into under Windows).

[some googling]

I can't find any confirmation for what I claimed about swap
files on logical paritions.  I must have mis-read something or
conflated it with the restriction that XP itself can't be
installed on a logical partition. :/

It looks like I could have created a small logical partition
for an NTFS filesystem in which I could have placed the swap
file.

I still can't believe that Windows does it's swapping using a
normal filesystem -- and by default it's the same filesystem
used for system and application files.  It seems like the
filesystem code would end up being a serious bottleneck. But I
long ago stopped trying to figure out why Windows does things...

-- 
Grant

[gentoo-user] Re: Howto share Linux swap partition with Windows XP

[ABCD]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Grant Edwards wrote:
> I still can't believe that Windows does it's swapping using a
> normal filesystem -- and by default it's the same filesystem
> used for system and application files.  It seems like the
> filesystem code would end up being a serious bottleneck. But I
> long ago stopped trying to figure out why Windows does things...

There actually is a good reason (oddly enough) for Windows using a file
on the filesystem for its swap space.  Because it is a simple file on
disk, if Windows realizes that the swap file is almost full, it can
expand your swap without having to do things like repartition.  This
makes the "swap is full - out of memory"-type problems less likely to
occur (unless it is "filesystem is full" as well :) ).

- --
ABCD
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.9 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iEYEARECAAYFAkl7Za8ACgkQOypDUo0oQOp8egCgwWyB4db6ZYJ9YwgvG/dq70Rq
64cAn3laOOtlhh7zN7ni85WpBYZDyLz6
=T1za
-----END PGP SIGNATURE-----

[gentoo-user] Re: Howto share Linux swap partition with Windows XP

[Grant Edwards]

On 2009-01-24, ABCD <en.ABCD@gmail.com> wrote:

> There actually is a good reason (oddly enough) for Windows
> using a file on the filesystem for its swap space.  Because it
> is a simple file on disk, if Windows realizes that the swap
> file is almost full, it can expand your swap without having to
> do things like repartition.  This makes the "swap is full -
> out of memory"-type problems less likely to occur

While that's a valid point in theory, I've never had a "swap is
full - out of memory" problem in all the years I've been
running Unixes that swapped to dedicated partitions.  In my
experience the system usually slows to a standstill and
requires drastic action long before swap fills up.

> (unless it is "filesystem is full" as well :) ).

That, on the other hand, I do run into quite regularly.

So it seems to me that using a swap file rather than a
paritition is increasing the liklehood of problems rather than
decreasing it while at the same time adding both system
overhead and instability.  Surely it's easier to corrupt a
swapfile that's in a normal, heavily-used filesystem than it is
to corrupt a dedicated swap partition?

The code that prevents one partition from "spilling over" into
another is much, much simpler and more bullet-proof than the
code that manages blocks/clusters within a filesystems.

If I were to guess why Windows doesn't use a swap partition, it
would be because floppy disks didn't have partitions.

-- 
Grant

Re: [gentoo-user] Re: Howto share Linux swap partition with Windows XP

[Stroller]

On 24 Jan 2009, at 17:22, Grant Edwards wrote:
> I still can't believe that Windows does it's swapping using a
> normal filesystem -- and by default it's the same filesystem
> used for system and application files.  It seems like the
> filesystem code would end up being a serious bottleneck.

    > 3. Does creating the swapfile on a journaled filesystem (e.g.
    > ext3 or reiser) incur a significant performance hit?

    None at all. The kernel generates a map of swap offset -> disk
    blocks at swapon time and from then on uses that map to perform
    swap I/O directly against the underlying disk queue, bypassing all
    caching, metadata and filesystem code.

    http://lkml.org/lkml/2005/7/7/326

Stroller.

[gentoo-user] Re: Howto share Linux swap partition with Windows XP

[Grant Edwards]

On 2009-01-24, Stroller <stroller@stellar.eclipse.co.uk> wrote:

>     > 3. Does creating the swapfile on a journaled filesystem (e.g.
>     > ext3 or reiser) incur a significant performance hit?
>
>     None at all. The kernel generates a map of swap offset -> disk
>     blocks at swapon time and from then on uses that map to perform
>     swap I/O directly against the underlying disk queue, bypassing all
>     caching, metadata and filesystem code.

I supposed that the NT kernel does something similar.

One implication of that is that the filesystem is then not
allowed to move blocks around if they are part of an "active"
swap file?  Not that I'm aware of filesystems that shuffle
blocks around while they're part of an open file, but one might
imagine something like that happening as part of some sort of
balancing algorithm.

-- 
Grant

[gentoo-user] Re: Howto share Linux swap partition with Windows XP

[ABCD]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Grant Edwards wrote:
> One implication of that is that the filesystem is then not
> allowed to move blocks around if they are part of an "active"
> swap file?  Not that I'm aware of filesystems that shuffle
> blocks around while they're part of an open file, but one might
> imagine something like that happening as part of some sort of
> balancing algorithm.
> 

I'm not sure if the swap can be moved around during normal use, but I do
know that it shows up as an "unmovable" block in XP's defragmentation
tool, suggesting that nothing is allowed to move it on disk at all,
while it is in use (which, on Windows, means the OS is running).

- --
ABCD
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.9 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iEYEARECAAYFAkl70wsACgkQOypDUo0oQOqDPgCfc78Ejvm96lonhVA581xCftXu
c9UAoL+YzrNHQ8iJL+fCmAUlD5WG9s5w
=KeKc
-----END PGP SIGNATURE-----