[gentoo-user] managing RAM usage

[gentoo-user] managing RAM usage

[Grant]

I'm trying to maximize the number of servers I can run on my machine
without running out of physical RAM.  I noticed something strange when
comparing top and free statistics:

top says each PID associated with a particular server has a RES of
about 100M.  Does that mean each running instance of that server
reserves 100MB?  Since there are 20 of these servers running, this
seems contradictory to the output from free which indicates less than
1GB used -/+ buffers/cache.

Is there any way to monitor RAM usage over time to see how close I'm
getting to using all of my physical RAM?

- Grant

Re: [gentoo-user] managing RAM usage

[Adam Carter]

[-- Attachment #1: Type: text/plain, Size: 530 bytes --]

On Thu, Jun 13, 2013 at 6:14 PM, Grant <emailgrant@gmail.com> wrote:

> I'm trying to maximize the number of servers I can run on my machine
> without running out of physical RAM.  I noticed something strange when
> comparing top and free statistics:
>
> top says each PID associated with a particular server has a RES of
> about 100M.  Does that mean each running instance of that server
> reserves 100MB?


Top man page:
       17. RES  --  Resident Memory Size (KiB)
           The non-swapped physical memory a task has used.

[-- Attachment #2: Type: text/html, Size: 885 bytes --]

Re: [gentoo-user] managing RAM usage

[Alan McKinnon]

On 13/06/2013 10:14, Grant wrote:
> I'm trying to maximize the number of servers I can run on my machine
> without running out of physical RAM.  I noticed something strange when
> comparing top and free statistics:
> 
> top says each PID associated with a particular server has a RES of
> about 100M.  Does that mean each running instance of that server
> reserves 100MB?  Since there are 20 of these servers running, this
> seems contradictory to the output from free which indicates less than
> 1GB used -/+ buffers/cache.

RES does not mean reserved. It means resident.

"man top" and searhc for RES for more info

> 
> Is there any way to monitor RAM usage over time to see how close I'm
> getting to using all of my physical RAM?


Not really if instantaneous values is what you are looking for.

All tools that purport to display "memory used" and "cpu load" values on
an instantaneous basis basically lie through their teeth, and it's a
fallacy to try and interpret such results to gain anything meaningful.

It can't be any other way actually - memory usage can change
dramatically 10,000 times a second, and that is precisely what the
computer is designed to do, and to do it fast, and to do it invisibly.

"using all of my physical RAM" is also a concept that really makes no
sense in any meaningful way. You can't treat it like eg the amount of
water left in a bottle. All of memory is always in use all of the time,
it's always doing *something*. Most of it is under the kernel's control
and is used for caching, and the kernel is free to move things around as
it sees fit. Memory is also shared, you can easily have a situation
where 10 apps each have access to Y megs of RAM and total memory in use
is still only a little bit more than Y

The best you can really do is keep an eye on what free says, and to
graph the trend.




-- 
Alan McKinnon
alan.mckinnon@gmail.com

Re: [gentoo-user] managing RAM usage

[Grant]

>> I'm trying to maximize the number of servers I can run on my machine
>> without running out of physical RAM.  I noticed something strange when
>> comparing top and free statistics:
>>
>> top says each PID associated with a particular server has a RES of
>> about 100M.  Does that mean each running instance of that server
>> reserves 100MB?  Since there are 20 of these servers running, this
>> seems contradictory to the output from free which indicates less than
>> 1GB used -/+ buffers/cache.
>
> RES does not mean reserved. It means resident.
>
> "man top" and searhc for RES for more info

OK but from what I can gather the concept is the same which is that
RES indicates how much RAM the process is using.

>> Is there any way to monitor RAM usage over time to see how close I'm
>> getting to using all of my physical RAM?
>
> Not really if instantaneous values is what you are looking for.
>
> All tools that purport to display "memory used" and "cpu load" values on
> an instantaneous basis basically lie through their teeth, and it's a
> fallacy to try and interpret such results to gain anything meaningful.
>
> It can't be any other way actually - memory usage can change
> dramatically 10,000 times a second, and that is precisely what the
> computer is designed to do, and to do it fast, and to do it invisibly.

Good point.  I suppose we would need the Linux memory system to do
some kind of built-in self-monitoring.

> "using all of my physical RAM" is also a concept that really makes no
> sense in any meaningful way. You can't treat it like eg the amount of
> water left in a bottle. All of memory is always in use all of the time,
> it's always doing *something*. Most of it is under the kernel's control
> and is used for caching, and the kernel is free to move things around as
> it sees fit. Memory is also shared, you can easily have a situation
> where 10 apps each have access to Y megs of RAM and total memory in use
> is still only a little bit more than Y

Does this explain why I can see 20 processes in top with 100MB
resident RAM each and free only indicates 1GB used -/+ buffers/cache?
If so, how can I go about setting apache2 MaxClients and other server
process limits so as to maximize performance and prevent an OOM
condition?

> The best you can really do is keep an eye on what free says, and to
> graph the trend.

Can you recommend an easy way to do that?

- Grant

Re: [gentoo-user] managing RAM usage

[João Matos]

[-- Attachment #1: Type: text/plain, Size: 96 bytes --]

If you extchange apache2 for lighttpd, probabily you will not need to worry
about memory usage.

[-- Attachment #2: Type: text/html, Size: 114 bytes --]

Re: [gentoo-user] managing RAM usage

[Bruce Hill]

On Thu, Jun 13, 2013 at 01:14:39AM -0700, Grant wrote:
> I'm trying to maximize the number of servers I can run on my machine
> without running out of physical RAM.  I noticed something strange when
> comparing top and free statistics:
> 
> top says each PID associated with a particular server has a RES of
> about 100M.  Does that mean each running instance of that server
> reserves 100MB?  Since there are 20 of these servers running, this
> seems contradictory to the output from free which indicates less than
> 1GB used -/+ buffers/cache.
> 
> Is there any way to monitor RAM usage over time to see how close I'm
> getting to using all of my physical RAM?
> 
> - Grant

Check out my friend's information here http://koltsoff.com/pub/ures/ and the
accompanying Meminfo script (linked at bottom of ^ web page).
-- 
Happy Penguin Computers               >')
126 Fenco Drive                       ( \
Tupelo, MS 38801                       ^^
support@happypenguincomputers.com
662-269-2706 662-205-6424
http://happypenguincomputers.com/

A: Because it messes up the order in which people normally read text.                                                                                                                                                          
Q: Why is top-posting such a bad thing?                                                                                                                                                                                        
A: Top-posting.                                                                                                                                                                                                                
Q: What is the most annoying thing in e-mail?

Don't top-post: http://en.wikipedia.org/wiki/Top_post#Top-posting

Re: [gentoo-user] managing RAM usage

[Grant]

>> I'm trying to maximize the number of servers I can run on my machine
>> without running out of physical RAM.  I noticed something strange when
>> comparing top and free statistics:
>>
>> top says each PID associated with a particular server has a RES of
>> about 100M.  Does that mean each running instance of that server
>> reserves 100MB?  Since there are 20 of these servers running, this
>> seems contradictory to the output from free which indicates less than
>> 1GB used -/+ buffers/cache.
>>
>> Is there any way to monitor RAM usage over time to see how close I'm
>> getting to using all of my physical RAM?
>>
>> - Grant
>
> Check out my friend's information here http://koltsoff.com/pub/ures/ and the
> accompanying Meminfo script (linked at bottom of ^ web page).
> --
> Happy Penguin Computers               >')

Thanks for that info.  The answer to this question seems pretty
indefinite.  I'm now logging 'free' to a file once per minute so I can
see how real-world traffic affects overall memory usage.  Still I
would need to see the system with processed forked all the way to
MaxClients to know how close it gets to OOM.

- Grant