[gentoo-amd64] Memory usage; 32 bit vs 64 bit.

[gentoo-amd64] Memory usage; 32 bit vs 64 bit.

[Dale]

Hi folks,

I recently built me a new 64 bit system.  My old 32 bit system has 2Gbs 
and my new system has 4Gbs.  I was expecting it to use about the same 
amount of memory but noticed it uses a good bit more on the new system 
than the old one.   With just the normal stuff open, I use about 1.5Gbs 
of ram.  My old system would use a little over half that.   I have the 
same settings on both.

Is this difference because 64 bit programs use more memory, maybe they 
are larger than 32 bit programs?  Just curious.  I notice that Seamonkey 
uses more and KDE's plasma-desktop uses more.  Those are generally the 
biggest users.

I'm not complaining about the usage, just curious as to why the difference.

Dale

:-)  :-)

Re: [gentoo-amd64] Memory usage; 32 bit vs 64 bit.

[Mark Knecht]

On Mon, Jan 3, 2011 at 10:34 AM, Dale <rdalek1967@gmail.com> wrote:
> Hi folks,
>
> I recently built me a new 64 bit system.  My old 32 bit system has 2Gbs and
> my new system has 4Gbs.  I was expecting it to use about the same amount of
> memory but noticed it uses a good bit more on the new system than the old
> one.   With just the normal stuff open, I use about 1.5Gbs of ram.  My old
> system would use a little over half that.   I have the same settings on
> both.
>
> Is this difference because 64 bit programs use more memory, maybe they are
> larger than 32 bit programs?  Just curious.  I notice that Seamonkey uses
> more and KDE's plasma-desktop uses more.  Those are generally the biggest
> users.
>
> I'm not complaining about the usage, just curious as to why the difference.
>
> Dale
>
> :-)  :-)
>
>

Hi Dale,
   That seems way high to me but I cannot do a real comparison here as
I haven't had a 32-bit X86 machine in 4-5 years. My current box is
tied up running 3 copies of Windows at the moment and is using pretty
much 12GB at the moment so I'd have to reboot to take any reading.
However I seem to remember only 350MB being used right at the point of
logging into KDE, but I could be wrong about that.

   Maybe post a few numbers:

1) What is used when you first start KDE
2) The additions for a few common apps.

   I'd be happy to compare mine to yours later today.

- Mark

Re: [gentoo-amd64] Memory usage; 32 bit vs 64 bit.

[Dale]

Mark Knecht wrote:
> On Mon, Jan 3, 2011 at 10:34 AM, Dale<rdalek1967@gmail.com>  wrote:
>    
>> Hi folks,
>>
>> I recently built me a new 64 bit system.  My old 32 bit system has 2Gbs and
>> my new system has 4Gbs.  I was expecting it to use about the same amount of
>> memory but noticed it uses a good bit more on the new system than the old
>> one.   With just the normal stuff open, I use about 1.5Gbs of ram.  My old
>> system would use a little over half that.   I have the same settings on
>> both.
>>
>> Is this difference because 64 bit programs use more memory, maybe they are
>> larger than 32 bit programs?  Just curious.  I notice that Seamonkey uses
>> more and KDE's plasma-desktop uses more.  Those are generally the biggest
>> users.
>>
>> I'm not complaining about the usage, just curious as to why the difference.
>>
>> Dale
>>
>> :-)  :-)
>>
>>
>>      
> Hi Dale,
>     That seems way high to me but I cannot do a real comparison here as
> I haven't had a 32-bit X86 machine in 4-5 years. My current box is
> tied up running 3 copies of Windows at the moment and is using pretty
> much 12GB at the moment so I'd have to reboot to take any reading.
> However I seem to remember only 350MB being used right at the point of
> logging into KDE, but I could be wrong about that.
>
>     Maybe post a few numbers:
>
> 1) What is used when you first start KDE
> 2) The additions for a few common apps.
>
>     I'd be happy to compare mine to yours later today.
>
> - Mark
>
>
>    

Well, I start with a saved session.  I usually have seamonkey, Konsole, 
Konqueror, Kpatience and gkrellm running.  I also run folding at well.  
I have checked when folding is not running and it still uses more.  I 
use the same settings as on my old rig for folding.  I may change to 
larger units when I get another stick or two of ram in here.

Here is some results from top with them sorted by memory usage:

   PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
18913 dale      20   0  793m 303m  30m S    3  7.7  21:35.05 seamonkey-bin
18451 dale      20   0  871m 165m  40m S    1  4.2  40:47.95 plasma-desktop
28665 root      39  19  304m 112m 2616 S   94  2.8   1918:38 FahCore_a3.exe
25414 dale      20   0  607m 107m 9.8m S    0  2.7   1:33.53 knotify4
18453 dale      20   0  591m  99m  15m S    0  2.5   0:23.53 knotify4
18417 dale      20   0  618m  90m  41m S    2  2.3  23:59.85 kwin
26081 root      39  19  280m  80m 2616 S   99  2.0   2138:13 FahCore_a3.exe
28654 root      39  19  277m  80m 2616 S   99  2.0   1916:58 FahCore_a3.exe
27537 root      39  19  280m  80m 2616 S   96  2.0   2021:21 FahCore_a3.exe
  5081 dale      20   0  474m  73m  31m S    0  1.9   0:31.91 konqueror
18225 root      20   0  142m  61m  15m S    4  1.6  92:24.29 X
18541 dale      20   0  470m  58m  27m S    0  1.5   0:35.72 kopete
29937 root      20   0  573m  53m  35m S    0  1.4   0:26.64 konqueror
26809 dale      20   0  202m  49m  30m S    0  1.2   0:29.16 mplayer
18566 dale      20   0  367m  48m  27m S    0  1.2   0:01.00 python2.6
18564 dale      20   0  418m  45m  24m S    0  1.1   0:00.49 python2
11802 root      20   0  721m  43m  11m S    0  1.1   1:19.41 knotify4
18539 dale      20   0  408m  38m  20m S    0  1.0   2:11.97 kpat
18491 dale      20   0  584m  37m  20m S    0  0.9   0:08.12 krunner
18544 dale      20   0  356m  34m  17m S    0  0.9   0:47.28 konsole
18371 dale      20   0  230m  31m  22m S    0  0.8   0:00.48 kdeinit4
18527 dale      20   0  417m  31m  16m S    0  0.8   0:00.48 kmix
18374 dale      20   0  409m  29m  16m S    0  0.7   0:05.96 kded4

As you can tell, Seamonkey has been running a while.  It still uses a 
good bit even when freshly started.  Here is my flags:

CFLAGS="-march=native -O2 -pipe"
CXXFLAGS="${CFLAGS}"

I got them off a wiki for safe flags.  I like fast but hate crashes.  ;-)

Dale

:-)  :-)

Re: [gentoo-amd64] Memory usage; 32 bit vs 64 bit.

[Darragh Bailey]

On Mon, Jan 03, 2011 at 03:46:28PM -0600, Dale wrote:
> Well, I start with a saved session.  I usually have seamonkey, Konsole,  
> Konqueror, Kpatience and gkrellm running.  I also run folding at well.   
> I have checked when folding is not running and it still uses more.  I  
> use the same settings as on my old rig for folding.  I may change to  
> larger units when I get another stick or two of ram in here.
>
> Here is some results from top with them sorted by memory usage:
>
>   PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
> 18913 dale      20   0  793m 303m  30m S    3  7.7  21:35.05 seamonkey-bin
> 18451 dale      20   0  871m 165m  40m S    1  4.2  40:47.95 plasma-desktop
> 28665 root      39  19  304m 112m 2616 S   94  2.8   1918:38 FahCore_a3.exe
> 25414 dale      20   0  607m 107m 9.8m S    0  2.7   1:33.53 knotify4
> 18453 dale      20   0  591m  99m  15m S    0  2.5   0:23.53 knotify4
> 18417 dale      20   0  618m  90m  41m S    2  2.3  23:59.85 kwin
> 26081 root      39  19  280m  80m 2616 S   99  2.0   2138:13 FahCore_a3.exe
> 28654 root      39  19  277m  80m 2616 S   99  2.0   1916:58 FahCore_a3.exe
> 27537 root      39  19  280m  80m 2616 S   96  2.0   2021:21 FahCore_a3.exe
>  5081 dale      20   0  474m  73m  31m S    0  1.9   0:31.91 konqueror
> 18225 root      20   0  142m  61m  15m S    4  1.6  92:24.29 X
> 18541 dale      20   0  470m  58m  27m S    0  1.5   0:35.72 kopete
> 29937 root      20   0  573m  53m  35m S    0  1.4   0:26.64 konqueror
> 26809 dale      20   0  202m  49m  30m S    0  1.2   0:29.16 mplayer
> 18566 dale      20   0  367m  48m  27m S    0  1.2   0:01.00 python2.6
> 18564 dale      20   0  418m  45m  24m S    0  1.1   0:00.49 python2
> 11802 root      20   0  721m  43m  11m S    0  1.1   1:19.41 knotify4
> 18539 dale      20   0  408m  38m  20m S    0  1.0   2:11.97 kpat
> 18491 dale      20   0  584m  37m  20m S    0  0.9   0:08.12 krunner
> 18544 dale      20   0  356m  34m  17m S    0  0.9   0:47.28 konsole
> 18371 dale      20   0  230m  31m  22m S    0  0.8   0:00.48 kdeinit4
> 18527 dale      20   0  417m  31m  16m S    0  0.8   0:00.48 kmix
> 18374 dale      20   0  409m  29m  16m S    0  0.7   0:05.96 kded4
>
> As you can tell, Seamonkey has been running a while.  It still uses a  
> good bit even when freshly started.  Here is my flags:
>
> CFLAGS="-march=native -O2 -pipe"
> CXXFLAGS="${CFLAGS}"
>
> I got them off a wiki for safe flags.  I like fast but hate crashes.  ;-)
>
> Dale
>
> :-)  :-)

I could be wrong here, but aren't both seamonkey-bin & FahCore_a3.exe
32bit applications? That would mean that you're likely loading both
32bit and 64bit versions of a number of libraries. Would certainly count
towards a higher memory usage.


-- 
Darragh

"Nothing is foolproof to a sufficiently talented fool."

Re: [gentoo-amd64] Memory usage; 32 bit vs 64 bit.

[Dale]

Darragh Bailey wrote:
> I could be wrong here, but aren't both seamonkey-bin&  FahCore_a3.exe
> 32bit applications? That would mean that you're likely loading both
> 32bit and 64bit versions of a number of libraries. Would certainly count
> towards a higher memory usage.
>
>
>    

That is a compiled seamonkey.  It installs it as seamonkey-bin since 
after I compile it here, it is then a binary.  I don't have any 
precompiled binaries here that I know of.

I chose the 64 bit folding package.  That's what was on the web page at 
least.  Since it is a binary, I can't be sure of that.

Dale

:-)  :-)

Re: [gentoo-amd64] Memory usage; 32 bit vs 64 bit.

[Volker Armin Hemmann]

On Thursday 06 January 2011 14:52:34 Dale wrote:
> Darragh Bailey wrote:
> > I could be wrong here, but aren't both seamonkey-bin&  FahCore_a3.exe
> > 32bit applications? That would mean that you're likely loading both
> > 32bit and 64bit versions of a number of libraries. Would certainly count
> > towards a higher memory usage.
> 
> That is a compiled seamonkey.  It installs it as seamonkey-bin since
> after I compile it here, it is then a binary.  I don't have any
> precompiled binaries here that I know of.
> 
> I chose the 64 bit folding package.  That's what was on the web page at
> least.  Since it is a binary, I can't be sure of that.
> 
> Dale
> 
> :-)  :-)

file can tell you that.

Re: [gentoo-amd64] Memory usage; 32 bit vs 64 bit.

[Dale]

Volker Armin Hemmann wrote:
> On Thursday 06 January 2011 14:52:34 Dale wrote:
>    
>> Darragh Bailey wrote:
>>      
>>> I could be wrong here, but aren't both seamonkey-bin&   FahCore_a3.exe
>>> 32bit applications? That would mean that you're likely loading both
>>> 32bit and 64bit versions of a number of libraries. Would certainly count
>>> towards a higher memory usage.
>>>        
>> That is a compiled seamonkey.  It installs it as seamonkey-bin since
>> after I compile it here, it is then a binary.  I don't have any
>> precompiled binaries here that I know of.
>>
>> I chose the 64 bit folding package.  That's what was on the web page at
>> least.  Since it is a binary, I can't be sure of that.
>>
>> Dale
>>
>> :-)  :-)
>>      
> file can tell you that.
>
>
>    

I thought there was a way but couldn't remember what it was.  I only 
used that once.

root@fireball / # file /root/foldingathome/fah6
/root/foldingathome/fah6: ELF 64-bit LSB executable, x86-64, version 1 
(SYSV), statically linked, for GNU/Linux 2.6.9, stripped
root@fireball / #

So I guess we know now that that is 64 bit.

Dale

:-)  :-)

Re: [gentoo-amd64] Memory usage; 32 bit vs 64 bit.

[Alex Alexander]

On 3 Jan 2011, at 20:34, Dale <rdalek1967@gmail.com> wrote:

> Hi folks,
> 
> I recently built me a new 64 bit system.  My old 32 bit system has 2Gbs and my new system has 4Gbs.  I was expecting it to use about the same amount of memory but noticed it uses a good bit more on the new system than the old one.   With just the normal stuff open, I use about 1.5Gbs of ram.  My old system would use a little over half that.   I have the same settings on both.
> 
> Is this difference because 64 bit programs use more memory, maybe they are larger than 32 bit programs?  Just curious.  I notice that Seamonkey uses more and KDE's plasma-desktop uses more.  Those are generally the biggest users.
> 
> I'm not complaining about the usage, just curious as to why the difference.
> 
> Dale
> 
> :-)  :-)
> 

Are you sure you're checking your free ram correctly? run "free" and check the buffers/cache line :)

Alex | wired | sent from my i4

Re: [gentoo-amd64] Memory usage; 32 bit vs 64 bit.

[Dale]

Alex Alexander wrote:
>
> Are you sure you're checking your free ram correctly? run "free" and check the buffers/cache line :)
>
> Alex | wired | sent from my i4
>
>    

Yea, I don't count caches and buffers.  After being up a few days, it 
uses almost all the ram with those included.  Here is the output of free 
tho:

root@fireball / # free
              total       used       free     shared    buffers     cached
Mem:       4055816    3413320     642496          0     540172    1093420
-/+ buffers/cache:    1779728    2276088
Swap:       979960        148     979812
root@fireball / #


That was just taken and I been logged in for a day or so.

Dale

:-)  :-)

Re: [gentoo-amd64] Memory usage; 32 bit vs 64 bit.

[Mark Knecht]

On Mon, Jan 3, 2011 at 1:48 PM, Dale <rdalek1967@gmail.com> wrote:
> Alex Alexander wrote:
>>
>> Are you sure you're checking your free ram correctly? run "free" and check
>> the buffers/cache line :)
>>
>> Alex | wired | sent from my i4
>>
>>
>
> Yea, I don't count caches and buffers.  After being up a few days, it uses
> almost all the ram with those included.  Here is the output of free tho:
>
> root@fireball / # free
>             total       used       free     shared    buffers     cached
> Mem:       4055816    3413320     642496          0     540172    1093420
> -/+ buffers/cache:    1779728    2276088
> Swap:       979960        148     979812
> root@fireball / #
>
>
> That was just taken and I been logged in for a day or so.
>
> Dale

When you have a chance reboot the machine and try it again. Don't run
any apps, open a terminal and run free.

I just rebooted. Here's what I see before running any apps:

mark@c2stable ~ $ free
             total       used       free     shared    buffers     cached
Mem:      12303060     462120   11840940          0      12412     178712
-/+ buffers/cache:     270996   12032064
Swap:     12602976          0   12602976
mark@c2stable ~ $

It doesn't drop much running Firefox to send this email:

mark@c2stable ~ $ free
             total       used       free     shared    buffers     cached
Mem:      12303060     655776   11647284          0      13824     230160
-/+ buffers/cache:     411792   11891268
Swap:     12602976          0   12602976
mark@c2stable ~ $


Cheers,
Mark

Re: [gentoo-amd64] Memory usage; 32 bit vs 64 bit.

[Dale]

Mark Knecht wrote:
> On Mon, Jan 3, 2011 at 1:48 PM, Dale<rdalek1967@gmail.com>  wrote:
>    
>> Alex Alexander wrote:
>>      
>>> Are you sure you're checking your free ram correctly? run "free" and check
>>> the buffers/cache line :)
>>>
>>> Alex | wired | sent from my i4
>>>
>>>
>>>        
>> Yea, I don't count caches and buffers.  After being up a few days, it uses
>> almost all the ram with those included.  Here is the output of free tho:
>>
>> root@fireball / # free
>>              total       used       free     shared    buffers     cached
>> Mem:       4055816    3413320     642496          0     540172    1093420
>> -/+ buffers/cache:    1779728    2276088
>> Swap:       979960        148     979812
>> root@fireball / #
>>
>>
>> That was just taken and I been logged in for a day or so.
>>
>> Dale
>>      
> When you have a chance reboot the machine and try it again. Don't run
> any apps, open a terminal and run free.
>
> I just rebooted. Here's what I see before running any apps:
>
> mark@c2stable ~ $ free
>               total       used       free     shared    buffers     cached
> Mem:      12303060     462120   11840940          0      12412     178712
> -/+ buffers/cache:     270996   12032064
> Swap:     12602976          0   12602976
> mark@c2stable ~ $
>
> It doesn't drop much running Firefox to send this email:
>
> mark@c2stable ~ $ free
>               total       used       free     shared    buffers     cached
> Mem:      12303060     655776   11647284          0      13824     230160
> -/+ buffers/cache:     411792   11891268
> Swap:     12602976          0   12602976
> mark@c2stable ~ $
>
>
> Cheers,
> Mark
>
>    

I don't reboot much.  I been up for a while.  I finally got me a good 
stable kernel build so the next reboot could be a while off.  I could go 
single user for a bit tho.

Dale

:-)  :-)

[gentoo-amd64] Re: Memory usage; 32 bit vs 64 bit.

[Duncan]

Alex Alexander posted on Mon, 03 Jan 2011 21:37:08 +0200 as excerpted:

> On 3 Jan 2011, at 20:34, Dale <rdalek1967@gmail.com> wrote:
> 
>> I recently built me a new 64 bit system.  My old 32 bit system has 2Gbs
>> and my new system has 4Gbs.  I was expecting it to use about the same
>> amount of memory but noticed it uses a good bit more on the new system
>> than the old one.   With just the normal stuff open, I use about 1.5Gbs
>> of ram.  My old system would use a little over half that.   I have the
>> same settings on both.
>> 
>> Is this difference because 64 bit programs use more memory, maybe they
>> are larger than 32 bit programs?  Just curious.  I notice that
>> Seamonkey uses more and KDE's plasma-desktop uses more.  Those are
>> generally the biggest users.
>> 
>> I'm not complaining about the usage, just curious as to why the
>> difference.
>> 
> Are you sure you're checking your free ram correctly? run "free" and
> check the buffers/cache line :)

Linux memory usage is notoriously confusing for the uninitiated and not 
entirely simple to explain or figure out the "real" per-app usage even for 
those who know /something/ about it.

First, to directly answer the question.  64-bit memory usage /will/ be 
somewhat higher, yes, but shouldn't be double.  The reason usage is higher 
is because address pointers are now 64-bit, not 32-bit, so /they/ take 
twice the space.  However, according to the gcc manpage:

       -m32
       -m64
           Generate code for a 32-bit or 64-bit environment.
           The 32-bit environment sets int, long and pointer
           to 32 bits and generates code that runs on any i386
           system.  The 64-bit environment sets int to 32 bits
           and long and pointer to 64 bits and generates code
           for AMD's x86-64 architecture. 

So the common "utility integer" standard C/C++ int types remain 32-bit.  
This actually one of the bigger issues in porting sources from 32-bit to 
64-bit, as for years, lazy 32-bit-only programmers were used to thinking 
of int, long and (memory) pointer as the same size, 32-bits, and being 
able to directly convert between them and use them nearly interchangeably, 
but that's no longer possible on amd64, because pointers and ints are no 
longer the same size.

But the point (not pointer! =:^) we're interested in for purposes of this 
discussion is that the very commonly used "utility integer" known simply 
as "int" remains 32-bit.  Because the 32-bit int is /so/ commonly used, to 
the point that it's the "default" integer type even on 64-bit, with only 
memory pointers and integers requiring 64-bit size getting full 64-bit, 
memory usage doesn't normally double, only increasing by some smaller 
factor, depending on the app and its particular mix of 32-bit int vs 64-
bit memory pointer and 64-bit long integers.

This additional memory usage is one of the negatives of 64-bit, and the 
reason that on archs other than x86, it's common to see 64-bit kernels for 
the ability to address > 4GB at the system level, with a 32-bit user-land 
since few individual apps (with noted exceptions) actually benefit from 
being able to address > 1-4 GB of RAM in a single app.  (Note the 1-4 GB 
range.   This is due to the common user-space/kernel-space split of the 4 
GB address space on 32-bit systems, meaning individual apps may be limited 
to only a gig of usable user-address-space, depending on whether the split 
is 1:3/2:2/3:1 or separate 4GB spaces for user and kernel.  Of course full 
64-bit doesn't have to worry about this.)

x86 is somewhat different in this regard, however, because traditional 32-
bit x86 is known as a "register starved" architecture -- the number of 
available full-CPU-speed registers on 32-bit x86 is comparatively limited, 
forcing code to depend on slower L1 cache (tho that's still way faster 
than L2/L3, which is way faster than main memory, which is way faster than 
typical spinning-disk main storage) where other archs could be using their 
relative abundance of CPU registers.  When it was designing amd64, AMD 
pretty much (I'm not sure if exactly) doubled the number of registers in 
their 64-bit hardware spec as compared to 32-bit (where they kept the same 
limited number of registers for compatibility reasons), with the result 
being that on amd64/x86_64 the speed-boost from access to these additional 
available registers often more than offsets the negative of the 
comparative double-size memory pointers.  The precise balance, whether the 
cost of dealing with double-size memory pointers or the benefit of access 
to all those additional registers wins, depends on the app in question, 
but in general the benefit of the extra registers on amd64/x86_64 as 
opposed to x86_32/ia32 is sufficient that it's far less common to see the 
64-bit kernel, 32-bit userland that is often seen on other archs.

That takes care of the direct answer.  Now to expand on what Alex referred 
to and what I mentioned in my intro as well, the topic of measuring Linux 
memory usage in general.

The uninitiated will often look at "free memory" (the value in the Mem: 
line of the "free" command, run at the command line) on Linux, and wonder 
why it's so small -- why Linux seems to use so much memory.  But, as Alex 
mentioned, that line is rather misleading, again, to the uninitiated.

Linux, like most OSs, considers "empty" memory "wasted" memory.  If the 
memory is available to use, therefore, Linux, as other OSs, will try to 
use it for something, normally for disk cache, mainly, with a bit used for 
other "buffering" as well.  When/if the system needs that memory for other 
stuff (apps), the cache and buffers can be dumped.

The confusion comes not in this, but rather, in the number actually 
exposed as "free" memory, which can be two very different values, either 
the actual "free" (unused=wasted) memory, or the "free for use if 
needed" (including memory used for cache and buffers) memory, depending on 
how the OS chooses to present it.  On Linux, the "free" memory as reported 
by the "free" command on the Mem: line is the first (unused=wasted), while 
that on the -/+ buffers/cache line is the second (free for use if needed).

Swap, of course, can be thrown in as another factor, since within context 
that can be seen as the reverse of disk cache -- app memory swapped out to 
disk as opposed to disk data cached in memory.  Thus free's Swap: line.
It's worth noting here the existence of the Linux kernel's swappiness 
parameter, exposed in the filesystem as /proc/sys/vm/swappiness .  This 
file contains a number 0-100 (attempting to set it > 100 results in an 
error), 60 being the default, indicating the desired balance between 
swapping apps out to retain disk cache and keeping apps in memory thus 
having less room for disk cache.  0 means always prefer keeping apps in 
memory, dumping cache when needed to do so, 100 means always prefer 
dumping apps to swap, retaining cache if at all possible.

As mentioned, the kernel swappiness default is 60, slightly preferring 
cache to apps.  A common recommendation found on the net, however, is to 
lower swappiness to something like 20, preferring with some strength 
retention of apps in memory to retention of cache.

Here, OTOH, I run swappiness=100, because swap is striped across four 
disks, while most of the filesystem is RAID-1 mirrored on the same four 
disks, so swap I/O should be faster than rereading formerly cached data 
back in off disk.  And, at least with my current 6 gigs RAM, with 
PORTAGE_TMPDIR on tmpfs (which is reported in free's cache value) and with 
parallel merging parameters carefully controlled so that even with 
swappiness=100 I only end up a few MB (perhaps a couple hundred) into 
swap, swappiness=100 works very well for me.  I don't notice the bit of 
swapping, and typically when I'm done, I might have 16 or 32 MB swapped 
out, that stays that way until I swapoff -a or reboot, indicating that I 
don't really use that bit of swapped apps much anyway or it'd be swapped 
back in when I did.

If you wish to experiment with swappiness, you can cat it to see the value 
as a normal user, but of course only save/echo a new value to it as root.

When you're done experimenting, if you want to make a permanent change, 
add a line ...

vm.swappiness = 100

... to your /etc/sysctl.conf file.  (Other /proc/sys/* settings can be 
similarly set this way, or of course with a simple echo-redirect line in 
/etc/conf.d/local or the like.  You can google for info on most or all of 
the other files under /proc/sys/, if interested.)

OK, back from the swappiness detour, to memory usage.

What sort of memory usage is reasonable?  Of course that depends on what 
you do with your computer.  =:^)  But, as you know, I'm a KDE user as 
well, and of course a gentoo/amd64 user.  Currently, I have an uptime of a 
week, which was when I last synced and updated both Gentoo and the kernel 
(thus the week uptime, since I rebooted into the new kernel then).  So 
I've not done a full update since I rebooted, tho I did emerge a few new 
packages (phonon-vlc and dependencies, including vlc, I was running phonon-
xine and still have it installed, but decided to try vlc and phonon-vlc) a 
couple days ago.  Of course I'm in KDE (4.5.4) ATM.  With that general 
system state and keeping in mind that I have 6 gigs RAM (the -m tells free 
to report in MB):

$free -m
             total       used       free     shared    buffers     cached
Mem:          5925       3334       2590          0        319       1571
-/+ buffers/cache:       1443       4481
Swap:        20479          0      20479

So ~ 2.5 gigs is entirely unused (empty, effectively wasted, ATM), with 
the ~ 3.25 gigs of used memory split between ~ 1.4 gigs used for apps and 
~ 1.8 gigs of cached and buffer memory, currently used to store data that 
can be dumped to make room for actual apps, if necessary.

Tho in my experience, even the 1.4 gigs of app usage isn't entirely 
required.  It has been awhile ago now, but at one point I was running 1 
gig of total RAM, with no swap.  At that time, app-memory usage seemed to 
run ~ half a gig.  When I upgraded RAM to 8 gigs (I since lost a stick 
that I've not replaced, thus the current 6 gigs), app memory usage 
increased as well, to closer to a gig (IIRC it was about 1.2 gig after a 
week's uptime, back then, to compare apples to apples as they say), 
without changing what I was running or the settings.  So given the memory 
to use, the apps I run apparently use it, up to perhaps a gig and a half.  
But if they're constrained to under a gig, they'll be content with less, 
perhaps half a gig.  I'm not sure of the mechanisms involved there except 
that apps do have access to the memory info as well, and perhaps some of 
them are more liberal with their own caching (in-memory web-page cache for 
browsers, etc) and the like, given memory room to work with.  But there's 
clearly a point at which they have their fill, as at a gig of RAM, apps 
were using half of it (half a gig), while when I upgraded to 8 gig, 8 
times the RAM, app-memory usage only just over doubled.  I suspect 4 gigs 
and 8 gigs would have about the same usage, but below 4 gigs, the apps 
start to be a bit more conservative with their own usage.

That covers overall system memory usage.  But what about individual apps?

Individual app memory usage on Linux is unfortunately a rather complex 
subject.  Top is a useful app for reporting on and controlling (nicing, 
killing, etc) other apps.  Top's manpage has a nice description of the 
various memory related stats and how they relate to each other, so I'll 
refer you to that for some detail I'm omitting here.  Meanwhile, on non-
swapping systems, resident memory (top's RES column) is about as accurate 
a first-order approximation of app memory usage as you'll get, but it's 
only reporting physical memory, so won't include anything swapped out.  
Also, the memory one could expect to free by terminating that app will be 
somewhat less than resident memory, due to libraries and data that may be 
shared between multiple apps.  Top has a SHR (shared) column to report 
potentially shared memory, but doesn't tell you how many other apps (maybe 
none) are actually sharing it.  Some memory reporting apps won't count 
shared memory as belonging to the app at all, others (like top, AFAIK) 
report the full memory shared as belonging to each app, while still others 
try to count how many apps are sharing what bits, and divide the shared 
memory by the number of apps sharing it.  Which way is "right" depends on 
what information you're actually looking for.  If you want the app totals 
to match actual total memory usage, apportioned share reporting is the way 
to go.  If you want to know what quitting the app will actually free, only 
count what's not shared by anything else.  If you want to know how much 
memory an app is actually using, regardless of other apps that may be 
sharing it too, count all the memory it's using, shared or not.

Then there's swapping.  Due to the way Linux works, the data available on 
swapped out memory is limited.  To get all the normal data would require 
swapping all that data back in, rather defeating the purpose of swap, so 
few if any memory usage reporting utils give you much detail about 
anything that's swapped out.  For people with memory enough to do so, a 
swapoff (or simply running without swap at all) force-disables swap, thus 
making full statistics available, but as mentioned above, to a point, many 
apps will use more memory if it's available, conserve if it's not, so 
running without swap on systems that routinely report non-zero swap usage 
doesn't necessarily give a true picture of an app's memory usage with swap 
enabled, either.

Conclusion:  While the output of the free command (and by extension, other 
references to free memory in Linux) may initially seem a bit unintuitive, 
it's straightforward enough, once one understands what's there.  
Unfortunately, the same can't be said about individual application memory 
usage, which remains somewhat difficult to nail down and even more so to 
properly describe, even after one understands the basics.  

FWIW, however, I don't claim to be a programmer or to understand all that 
much beyond the basics.  Should someone believe I'm in error with the 
above, or if they have anything to add or especially if they have a 
reasonably accurate simpler way to describe things, please post!  I love 
to learn, and definitely do NOT believe I've reach my limit in learning in 
this area!

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman

Re: [gentoo-amd64] Re: Memory usage; 32 bit vs 64 bit.

[Volker Armin Hemmann]

swap kills interactivity and is really, really bad on linux. No matter how 
much you stripe it. Swap is a true horror. So setting swappiness to 100 (which 
means: keep caches alive, no matter what and swap the hell out of it) is a 
really bad idea. In my experience it is better to have a very low swappiness 
and let the kernel get the occasional data from disk, than to swap to the same 
disks. Strange eh?

But better to wait 1.5s longer for konqueror to display a directory than to 
have a jerky mouse and input lag because of swap. Remember: the kernel ALWAYS 
swaps out the wrong stuff.
I have swappiness at 60 - and back in the 4GB days at 0. Because swap sucks so 
much.

You are not required to believe that. But just google.

[gentoo-amd64] Re: Memory usage; 32 bit vs 64 bit.

[Duncan]

Volker Armin Hemmann posted on Tue, 04 Jan 2011 08:27:28 +0100 as
excerpted:

> swap kills interactivity and is really, really bad on linux. No matter
> how much you stripe it. Swap is a true horror. So setting swappiness to
> 100 (which means: keep caches alive, no matter what and swap the hell
> out of it) is a really bad idea. In my experience it is better to have a
> very low swappiness and let the kernel get the occasional data from
> disk, than to swap to the same disks. Strange eh?
> 
> But better to wait 1.5s longer for konqueror to display a directory than
> to have a jerky mouse and input lag because of swap. Remember: the
> kernel ALWAYS swaps out the wrong stuff.
> I have swappiness at 60 - and back in the 4GB days at 0. Because swap
> sucks so much.
> 
> You are not required to believe that. But just google.

All I can go on is my experience, which agrees with you (and most of the 
advice on the net) when it's a single-core CPU on single-spindle storage, 
but I've found the experience rather different on multi-core machines 
driving quad-spindle striped swap on mirrored RAID, with enough memory 
swap usage is trivial in the ordinary case.  Once swap usage hits half a 
gig or so, yes, it's noticed, but until then, I literally don't normally 
notice it unless/until I happen to see the usage reported on the system 
monitors.

OTOH, losing portage tree cache or news (nntp) article cache and having to 
fetch the data from disk can be quite noticeable, as it DEFINITELY is the 
first time I access after a reboot (the infamous cold-cache case), the 
biggest reason I tend to leave the system running for weeks at a time, 
rebooting only to load a new test kernel or the like.

(FWIW, I use app-admin/lib_users to track programs using stale/already-
deleted libs after an update, and after quitting kde/X, restart daemons, 
etc, to clear the list if necessary, thus clearing that source of both 
security vulns and so-called anon-memory usage.  I do use the 
portage-2.2.0_alphas with preserve-libs, but use FEATURES=-preserve-libs 
to avoid that source of bugs, so the old libs do normally get deleted.)

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman

Re: [gentoo-amd64] Re: Memory usage; 32 bit vs 64 bit.

[Volker Armin Hemmann]

On Tuesday 04 January 2011 09:12:17 Duncan wrote:
> Volker Armin Hemmann posted on Tue, 04 Jan 2011 08:27:28 +0100 as
> 
> excerpted:
> > swap kills interactivity and is really, really bad on linux. No matter
> > how much you stripe it. Swap is a true horror. So setting swappiness to
> > 100 (which means: keep caches alive, no matter what and swap the hell
> > out of it) is a really bad idea. In my experience it is better to have a
> > very low swappiness and let the kernel get the occasional data from
> > disk, than to swap to the same disks. Strange eh?
> > 
> > But better to wait 1.5s longer for konqueror to display a directory than
> > to have a jerky mouse and input lag because of swap. Remember: the
> > kernel ALWAYS swaps out the wrong stuff.
> > I have swappiness at 60 - and back in the 4GB days at 0. Because swap
> > sucks so much.
> > 
> > You are not required to believe that. But just google.
> 
> All I can go on is my experience, which agrees with you (and most of the
> advice on the net) when it's a single-core CPU on single-spindle storage,
> but I've found the experience rather different on multi-core machines
> driving quad-spindle striped swap on mirrored RAID, with enough memory
> swap usage is trivial in the ordinary case.  Once swap usage hits half a
> gig or so, yes, it's noticed, but until then, I literally don't normally
> notice it unless/until I happen to see the usage reported on the system
> monitors.
> 
> OTOH, losing portage tree cache or news (nntp) article cache and having to
> fetch the data from disk can be quite noticeable, as it DEFINITELY is the
> first time I access after a reboot (the infamous cold-cache case), the
> biggest reason I tend to leave the system running for weeks at a time,
> rebooting only to load a new test kernel or the like.
> 
> (FWIW, I use app-admin/lib_users to track programs using stale/already-
> deleted libs after an update, and after quitting kde/X, restart daemons,
> etc, to clear the list if necessary, thus clearing that source of both
> security vulns and so-called anon-memory usage.  I do use the
> portage-2.2.0_alphas with preserve-libs, but use FEATURES=-preserve-libs
> to avoid that source of bugs, so the old libs do normally get deleted.)

well, I don't waste electricity. I do have / on a ssd and /var with portage on 
a raifd5. And waiting a few seconds longer for emerge -auvDtn world to give 
results does not matter for me.

But a stuck keyboard because of swapping? (swap striped to three disks, 4core 
processor) Inacceptable.

And so swappiness has to stay down. Many days I don't even turn swap on. 
Seeing gcc oom is much less annoying than swap.

It got better over the years but it is still far from being acceptable.

Re: [gentoo-amd64] Re: Memory usage; 32 bit vs 64 bit.

[Enrico Weigelt]

* Volker Armin Hemmann <volkerarmin@googlemail.com> wrote:

> Remember: the kernel ALWAYS swaps out the wrong stuff.

Is there anything userland apps could do to improve it ?
eg. memory pools based on usage patterns ?


cu
-- 
----------------------------------------------------------------------
 Enrico Weigelt, metux IT service -- http://www.metux.de/

 phone:  +49 36207 519931  email: weigelt@metux.de
 mobile: +49 151 27565287  icq:   210169427         skype: nekrad666
----------------------------------------------------------------------
 Embedded-Linux / Portierung / Opensource-QM / Verteilte Systeme
----------------------------------------------------------------------

RES: [gentoo-amd64] Re: Memory usage; 32 bit vs 64 bit.

[Eduardo Schoedler]

On Jan/06/2011, Enrico Weigelt wrote:
> Is there anything userland apps could do to improve it ?
> eg. memory pools based on usage patterns ?

Try tuning vm.swappiness sysctl param.

--
Eduardo Schoedler

Re: [gentoo-amd64] Re: Memory usage; 32 bit vs 64 bit.

[Volker Armin Hemmann]

On Thursday 06 January 2011 21:31:25 Enrico Weigelt wrote:
> * Volker Armin Hemmann <volkerarmin@googlemail.com> wrote:
> > Remember: the kernel ALWAYS swaps out the wrong stuff.
> 
> Is there anything userland apps could do to improve it ?
> eg. memory pools based on usage patterns ?
> 
> 
> cu

sure, they can try to mlock themselves. But that is a nother bucket of 
worms...

Re: [gentoo-amd64] Re: Memory usage; 32 bit vs 64 bit.

[Enrico Weigelt]

* Volker Armin Hemmann <volkerarmin@googlemail.com> wrote:

> > Is there anything userland apps could do to improve it ?
> > eg. memory pools based on usage patterns ?
> 
> sure, they can try to mlock themselves. But that is a nother bucket of 
> worms...

Wouldn't that defeat the purpose of swapping ?

I was thinking about something different: trying to design
applications in a more swap-friendly way, eg. keeping data thats
used together in a relatively closed set of pages. Or separate
heavily used data from seldomly used ones.

For example, an web browser could use an own pool per window.
If some window is minimized, it shouldn't have to be repainted
(in theory ;-)), and everything belonging to it could potentially
be swapped out together, while other (active) windows dont need
data from these pages. OTOH closing an window would free the
whole pool - if it's an anonymous mmap()ed region, the kernel
could simply throw the pages away.


cu
-- 
----------------------------------------------------------------------
 Enrico Weigelt, metux IT service -- http://www.metux.de/

 phone:  +49 36207 519931  email: weigelt@metux.de
 mobile: +49 151 27565287  icq:   210169427         skype: nekrad666
----------------------------------------------------------------------
 Embedded-Linux / Portierung / Opensource-QM / Verteilte Systeme
----------------------------------------------------------------------

Re: [gentoo-amd64] Re: Memory usage; 32 bit vs 64 bit.

[Enrico Weigelt]

* Duncan <1i5t5.duncan@cox.net> wrote:

> $free -m
>              total       used       free     shared    buffers     cached
> Mem:          5925       3334       2590          0        319       1571
> -/+ buffers/cache:       1443       4481
> Swap:        20479          0      20479

Apropos total memory: on my box w/ 4GB, it only shows up 3GB.
On bootup, kernel (built w/ CONFIG_HIGHMEM4G=y) says:

    2119MB HIGHMEM available.
    887MB LOWMEM available.

Who's eating up a whole GB ? BIOS ? GPU ?

> Tho in my experience, even the 1.4 gigs of app usage isn't entirely 
> required.  It has been awhile ago now, but at one point I was running 1 
> gig of total RAM, with no swap.  At that time, app-memory usage seemed to 
> run ~ half a gig.  When I upgraded RAM to 8 gigs (I since lost a stick 
> that I've not replaced, thus the current 6 gigs), app memory usage 
> increased as well, to closer to a gig (IIRC it was about 1.2 gig after a 
> week's uptime, back then, to compare apples to apples as they say), 
> without changing what I was running or the settings. 

That's interesting. Which of the apps now use more memory ?
Any database systems on your box ?

> Individual app memory usage on Linux is unfortunately a rather complex 
> subject.  Top is a useful app for reporting on and controlling (nicing, 
> killing, etc) other apps.  Top's manpage has a nice description of the 
> various memory related stats and how they relate to each other, so I'll 
> refer you to that for some detail I'm omitting here.  Meanwhile, on non-
> swapping systems, resident memory (top's RES column) is about as accurate 
> a first-order approximation of app memory usage as you'll get, but it's 
> only reporting physical memory, so won't include anything swapped out.  
> Also, the memory one could expect to free by terminating that app will be 
> somewhat less than resident memory, due to libraries and data that may be 
> shared between multiple apps.  Top has a SHR (shared) column to report 
> potentially shared memory, but doesn't tell you how many other apps (maybe 
> none) are actually sharing it.  Some memory reporting apps won't count 
> shared memory as belonging to the app at all, others (like top, AFAIK) 
> report the full memory shared as belonging to each app, while still others 
> try to count how many apps are sharing what bits, and divide the shared 
> memory by the number of apps sharing it.  Which way is "right" depends on 
> what information you're actually looking for.  If you want the app totals 
> to match actual total memory usage, apportioned share reporting is the way 
> to go.  If you want to know what quitting the app will actually free, only 
> count what's not shared by anything else.  If you want to know how much 
> memory an app is actually using, regardless of other apps that may be 
> sharing it too, count all the memory it's using, shared or not.

Yep, depending on the actual question you ask, you'll have to look at
different stats. For example, if you're interested in how much memory
usage a single process adds to the system (IOW: how much would be freed
by killing it), you'll have to look through all its mappings and count
off the pages that are also mapped by other process'es.

BTW: does anyone know a tool for that (just too lazy to code it by myself
right now ;-o)

> Then there's swapping.  Due to the way Linux works, the data available on 
> swapped out memory is limited.  To get all the normal data would require 
> swapping all that data back in, rather defeating the purpose of swap, so 
> few if any memory usage reporting utils give you much detail about 
> anything that's swapped out.  For people with memory enough to do so, a 
> swapoff (or simply running without swap at all) force-disables swap, thus 
> making full statistics available, but as mentioned above, to a point, many 
> apps will use more memory if it's available, conserve if it's not, so 
> running without swap on systems that routinely report non-zero swap usage 
> doesn't necessarily give a true picture of an app's memory usage with swap 
> enabled, either.

BTW: does anyone know a sane way how to hand-off this to the kernel ?
So eg. a RDBMS could ask the kernel how much buffers it should use ?
(or somehow use the kernel's buffercache decisions)


cu
-- 
----------------------------------------------------------------------
 Enrico Weigelt, metux IT service -- http://www.metux.de/

 phone:  +49 36207 519931  email: weigelt@metux.de
 mobile: +49 151 27565287  icq:   210169427         skype: nekrad666
----------------------------------------------------------------------
 Embedded-Linux / Portierung / Opensource-QM / Verteilte Systeme
----------------------------------------------------------------------

Re: [gentoo-amd64] Re: Memory usage; 32 bit vs 64 bit.

[Volker Armin Hemmann]

On Thursday 06 January 2011 21:29:20 Enrico Weigelt wrote:
> * Duncan <1i5t5.duncan@cox.net> wrote:
> > $free -m
> > 
> >              total       used       free     shared   
> >              buffers     cached
> > 
> > Mem:          5925       3334       2590          0        319      
> > 1571
> > -/+ buffers/cache:       1443       4481
> > Swap:        20479          0      20479
> 
> Apropos total memory: on my box w/ 4GB, it only shows up 3GB.
> On bootup, kernel (built w/ CONFIG_HIGHMEM4G=y) says:
> 
>     2119MB HIGHMEM available.
>     887MB LOWMEM available.
> 
> Who's eating up a whole GB ? BIOS ? GPU ?

bios

Re: [gentoo-amd64] Re: Memory usage; 32 bit vs 64 bit.

[Enrico Weigelt]

* Volker Armin Hemmann <volkerarmin@googlemail.com> wrote:

> > Apropos total memory: on my box w/ 4GB, it only shows up 3GB.
> > On bootup, kernel (built w/ CONFIG_HIGHMEM4G=y) says:
> > 
> >     2119MB HIGHMEM available.
> >     887MB LOWMEM available.
> > 
> > Who's eating up a whole GB ? BIOS ? GPU ?
> 
> bios

BIOS really eats it all up, or maybe some misconfiguration that
causes memory hidden from the OS ?


cu
-- 
----------------------------------------------------------------------
 Enrico Weigelt, metux IT service -- http://www.metux.de/

 phone:  +49 36207 519931  email: weigelt@metux.de
 mobile: +49 151 27565287  icq:   210169427         skype: nekrad666
----------------------------------------------------------------------
 Embedded-Linux / Portierung / Opensource-QM / Verteilte Systeme
----------------------------------------------------------------------

Re: [gentoo-amd64] Re: Memory usage; 32 bit vs 64 bit.

[Volker Armin Hemmann]

On Thursday 06 January 2011 23:32:01 Enrico Weigelt wrote:
> * Volker Armin Hemmann <volkerarmin@googlemail.com> wrote:
> > > Apropos total memory: on my box w/ 4GB, it only shows up 3GB.
> > > 
> > > On bootup, kernel (built w/ CONFIG_HIGHMEM4G=y) says:
> > >     2119MB HIGHMEM available.
> > >     887MB LOWMEM available.
> > > 
> > > Who's eating up a whole GB ? BIOS ? GPU ?
> > 
> > bios
> 
> BIOS really eats it all up, or maybe some misconfiguration that
> causes memory hidden from the OS ?
> 
> 
> cu

the misconfiguration is done by the bios, mapping pcispace&co into the 3-4gb 
range

[gentoo-amd64] Re: Memory usage; 32 bit vs 64 bit.

[Duncan]

Volker Armin Hemmann posted on Fri, 07 Jan 2011 00:42:01 +0100 as
excerpted:

> On Thursday 06 January 2011 23:32:01 Enrico Weigelt wrote:
>> * Volker Armin Hemmann <volkerarmin@googlemail.com> wrote:
>> > > Apropos total memory: on my box w/ 4GB, it only shows up 3GB.
>> > > 
>> > > On bootup, kernel (built w/ CONFIG_HIGHMEM4G=y) says:
>> > >     2119MB HIGHMEM available. 887MB LOWMEM available.

Just so we're clear here, CONFIG_HIGHMEM4G only applies to 32-bit, as does 
low and high memory since 64-bit is flat memory addressing up into the 
PiBytes (IIRC).  You don't say whether the box is running amd64 (64-bit) 
or x86 (32-bit) (and for that matter, you don't say Gentoo either), but 
the assumption on this list would be Gentoo/amd64, so 64-bit.

But you're obviously running x86 32-bit on that box.  Again, just to make 
it clear as you didn't.

>> > > Who's eating up a whole GB ? BIOS ? GPU ?
>> > 
>> > bios
>> 
>> BIOS really eats it all up, or maybe some misconfiguration that causes
>> memory hidden from the OS ?
> 
> the misconfiguration is done by the bios, mapping pcispace&co into the
> 3-4gb range

Even in 64-bit machines, many legacy 32-bit-only PCI devices can't handle 
IO-address-space configured above the 4-gig 32-bit memory barrier.  As a 
result, there's a "memory hole", usually half a gig to a gig in size, just 
below the 4-gig barrier, that's address space reserved for 32-bit-PCI-IO.  
Of course it's only relatively recently that people began having memory of 
several gigs and thus running into the problem, but what happens when 
someone has > 3 gigs RAM in a machine is that unless the BIOS is equipped 
with memory remapping functionality to map the real memory behind that PCI-
reserved area up above the 4 GB barrier, that IO-region covers the real 
memory, which then cannot be accessed.

Since AFAIK (and I might be wrong) 32-bit MS Windows doesn't have the
to-64-gig PAE capacities that Linux has, there was no pressure for systems 
designed to be sold with it to have that remapping, since I don't believe 
they could make use of it anyway on 32-bit MS.  They were simply limited 
to 3-3.5GB of usable memory.  Of course, machines designed to run 64-bit 
versions of whatever OS could and should have had BIOSs with this 
remapping functionality, but some didn't -- I imagine the BIOS companies 
(Phoenix, Award, AMI...) were charging extra for the feature or something, 
back then, and for all I know, might still charge extra for it.

Even on the ones that have the feature, it's often configurable.  I know 
there are two options related to that in my BIOS (for dual-socket original 
3-digit AMD Opterons, AMD 8xxx chipset, PCI-X, before PCI-E), but they're 
not documented in the print-manual as they were added in a BIOS update 
after the manual was printed (but IIRC the BIOS shipped had the options, 
or at least they were there by the time I upgraded from a gig to 8 gig of 
memory and could actually use them).  I had read about the issue, and 
found the BIOS options that controlled it, but didn't quite understand 
them and more or less simply messed with them until I got the desired 
effect -- the BIOS detecting memory above the 4-GB barrier and Linux 
seeing it as well.  IIRC the one option configures the remapping itself, 
while the other configures how the MMTs see and describe the hole, caching-
wise.

FWIW, any decent mobo manufacturer should either list the feature, or have 
a disclaimer about BIOS mapped memory being limited to 3-3.5 gigs, if you 
lookup the mobo specs on their site.  I imagine systems integrators will 
likewise have the info available, but don't know as I've only ever bought 
two whole systems, my original 486sx25 back in '93 or so, and more 
recently, my netbook.  The rest of the time I simply upgrade in-place.  
But of course many of the buyers really haven't a clue what that language 
is talking about anyway, if they even check the tech-specs to that degree, 
so many end up finding out about it the hard way and never really know why 
it can't see all the memory, only that it doesn't.

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman

Re: [gentoo-amd64] Re: Memory usage; 32 bit vs 64 bit.

[Volker Armin Hemmann]

only with some kinds of win xp are there problems with devices not mapped at 
the end of the 4gb space. Loosing a gig on a 64bit machine is just bad bios 
programming and nothing else.

Re: [gentoo-amd64] Re: Memory usage; 32 bit vs 64 bit.

[Mark Knecht]

On Thu, Jan 6, 2011 at 12:29 PM, Enrico Weigelt <weigelt@metux.de> wrote:
> * Duncan <1i5t5.duncan@cox.net> wrote:
>
>> $free -m
>>              total       used       free     shared    buffers     cached
>> Mem:          5925       3334       2590          0        319       1571
>> -/+ buffers/cache:       1443       4481
>> Swap:        20479          0      20479
>
> Apropos total memory: on my box w/ 4GB, it only shows up 3GB.
> On bootup, kernel (built w/ CONFIG_HIGHMEM4G=y) says:
>
>    2119MB HIGHMEM available.
>    887MB LOWMEM available.
>
> Who's eating up a whole GB ? BIOS ? GPU ?

It sounds suspiciously like video memory to me.

GART could be involved. How much DRAM on your video card?

- Mark

Re: [gentoo-amd64] Memory usage; 32 bit vs 64 bit.

[Rich Freeman]

On Mon, Jan 3, 2011 at 1:34 PM, Dale <rdalek1967@gmail.com> wrote:
> Is this difference because 64 bit programs use more memory, maybe they are
> larger than 32 bit programs?  Just curious.  I notice that Seamonkey uses
> more and KDE's plasma-desktop uses more.  Those are generally the biggest
> users.

How are you measuring memory-use?  If your system has more RAM the
system will tend to cache a lot more stuff - linux doesn't like to let
RAM go to waste.

Probably best to compare individual processes across archs - and not
overall memory use.

[gentoo-amd64] Re: Memory usage; 32 bit vs 64 bit.

[Nikos Chantziaras]

On 01/03/2011 08:34 PM, Dale wrote:
> Hi folks,
>
> I recently built me a new 64 bit system. My old 32 bit system has 2Gbs
> and my new system has 4Gbs. I was expecting it to use about the same
> amount of memory but noticed it uses a good bit more on the new system
> than the old one. With just the normal stuff open, I use about 1.5Gbs of
> ram. My old system would use a little over half that. I have the same
> settings on both.
>
> Is this difference because 64 bit programs use more memory, maybe they
> are larger than 32 bit programs? Just curious. I notice that Seamonkey
> uses more and KDE's plasma-desktop uses more. Those are generally the
> biggest users.
>
> I'm not complaining about the usage, just curious as to why the difference.

There's usually a very small difference (I'd say about 10%).  But 
nothing like you're describing (which is almost 100% if what you're 
saying is true.)

How do you tell how much memory is used?  If you're using KDE, System 
Monitor will give you a good value that's very close to reality.  On my 
system (with a full blown KDE desktop with some plasmoids active and 
currently Firefox + Thunderbird + Amarok running) it says "0.57 GiB of 
5.8 GiB", meaning it uses about 570MB of RAM and I have 6GB total.) 
After a fresh reboot it's about 230MB.

I'm sure you're not using the same configuration (I don't mean USE flags 
and such, but rather how you've configured your application and desktop.)

Re: [gentoo-amd64] Memory usage; 32 bit vs 64 bit.

[Volker Armin Hemmann]

On Monday 03 January 2011 12:34:17 Dale wrote:

> Is this difference because 64 bit programs use more memory, maybe they
> are larger than 32 bit programs? 

yes, but not by much.