[gentoo-user] KDE ridiculous memory usage

[gentoo-user] KDE ridiculous memory usage

[Florian Philipp]

[-- Attachment #1: Type: text/plain, Size: 2321 bytes --]

Hi list!

I have a bit of a problem. I'm on KDE-4.4.5 and it eats memory for
breakfast. Directly after booting, everything is okay but the usage
grows significantly. I wonder whether this is expected behavior.

The following statistics have been taken after 8 days of uptime during
which the system was on standby most of the time during work days and at
night.

free -m
             total  used  free  shared  buffers  cached
Mem:          3754  3588   165       0       57     258
-/+ buffers/cache:  3271   482
Swap:         6142   978  5163

A desktop machine that has 4GB RAM and still needs to swap?!

Excerpt from top:
 VIRT  RES  SHR S %CPU %MEM     TIME+ COMMAND
1094m 484m  10m S    0 12.9  96:43.01 firefox
 932m 471m  15m S    0 12.6   5:10.20 akregator
 384m 303m 2856 S    0  8.1  59:43.43 virtuoso-t
 709m 282m 2936 S    0  7.5   0:40.51 nepomukservices
 839m 146m  15m S    0  3.9   8:37.76 thunderbird-bin
 191m 131m  532 S    0  3.5  12:30.73 dbus-daemon
 902m 105m 5288 S    0  2.8   0:30.16 krunner
 263m 105m 1724 S    0  2.8   2:31.18 squid
 255m  61m 6672 S    7  1.6 305:04.24 X
1106m  55m 7756 S    0  1.5   4:22.73 amarok
 534m  54m  10m S    0  1.5   2:33.94 kopete
 559m  52m 6536 S    0  1.4  56:52.37 nepomukservices
 718m  38m  12m S    4  1.0 143:36.62 plasma-desktop
 295m  33m 2048 S    0  0.9   1:59.32 mysqld
 360m  17m 1856 S    0  0.5   0:07.56 tomboy
 445m  16m 3392 S    0  0.4  38:54.36 nepomukservices
 365m  14m 6356 S    1  0.4  27:38.49 konsole
 438m  11m 4928 S    0  0.3   0:20.12 kded4
 508m  11m 6364 S    0  0.3   0:45.79 kwin

Okay, I'm used to Firefox taking much memory. I'm okay with that since
it's the most heavily used application currently running. But why does
Akregator need that much memory? It doesn't even have any tabs open at
the moment and is just running minimized in the background.

Virtuoso looks like an optional Soprano dependency which in turn is
needed for Nepomuk. Are the default use flags for dev-libs/soprano
suboptimal? What happens if I choose other flags for Soprano?

The rest of the list is a bit suspicious, as well. Especially DBus and
Kopete look like they live way beyond their means (or my means ;) ).

Do other users experience the same?

Thanks in advance!
Florian Philipp


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 262 bytes --]

Re: [gentoo-user] KDE ridiculous memory usage

[Alex Schuster]

Florian Philipp writes:

> I have a bit of a problem. I'm on KDE-4.4.5 and it eats memory for
> breakfast. Directly after booting, everything is okay but the usage
> grows significantly. I wonder whether this is expected behavior.

I was just about to write something about this. I suffered from bad
performance for quite a while now (like mplayer stuttering during
emerges), started a thread in in the gentoo-performance list and got some
advice that made things a little better, but I suspect the effect came
from reduced memory usage only. Since yesterday the problems all seem to
be gone, but again it's not a real solution, as I plugged in another 2GB
of memory, so now I have 6G.

I used to restart kdm once per day in order to free memory. If I did not
do this, KDE4 became nearly unsusabe.

> The following statistics have been taken after 8 days of uptime during
> which the system was on standby most of the time during work days and
> at night.
> 
> free -m
>              total  used  free  shared  buffers  cached
> Mem:          3754  3588   165       0       57     258
> -/+ buffers/cache:  3271   482
> Swap:         6142   978  5163
> 
> A desktop machine that has 4GB RAM and still needs to swap?!

After 1 day of uptime, my system needs even more, but I'm also running
some stuff.

wonko@weird ~ $ free -m
             total       used       free     shared    buffers     cached
Mem:          5721       5618        103          0        112       1108
-/+ buffers/cache:       4397       1323
Swap:         4094         50       4044


> Excerpt from top:
>  VIRT  RES  SHR S %CPU %MEM     TIME+ COMMAND
> 1094m 484m  10m S    0 12.9  96:43.01 firefox
>  932m 471m  15m S    0 12.6   5:10.20 akregator
>  384m 303m 2856 S    0  8.1  59:43.43 virtuoso-t
>  709m 282m 2936 S    0  7.5   0:40.51 nepomukservices
>  839m 146m  15m S    0  3.9   8:37.76 thunderbird-bin
>  191m 131m  532 S    0  3.5  12:30.73 dbus-daemon
>  902m 105m 5288 S    0  2.8   0:30.16 krunner
>  263m 105m 1724 S    0  2.8   2:31.18 squid
>  255m  61m 6672 S    7  1.6 305:04.24 X
> 1106m  55m 7756 S    0  1.5   4:22.73 amarok
>  534m  54m  10m S    0  1.5   2:33.94 kopete
>  559m  52m 6536 S    0  1.4  56:52.37 nepomukservices
>  718m  38m  12m S    4  1.0 143:36.62 plasma-desktop
>  295m  33m 2048 S    0  0.9   1:59.32 mysqld
>  360m  17m 1856 S    0  0.5   0:07.56 tomboy
>  445m  16m 3392 S    0  0.4  38:54.36 nepomukservices
>  365m  14m 6356 S    1  0.4  27:38.49 konsole
>  438m  11m 4928 S    0  0.3   0:20.12 kded4
>  508m  11m 6364 S    0  0.3   0:45.79 kwin

Now this looks different here. I have X with 946M, plasma-desktop with 
505M, that's 15 times the memory you need. Then comes java with 371M (for 
TV-Browser - yes, 371MB just for showing the TV programme!), emerge wants 
272M while emerging openoffice. Chromium also needs much memory, my 33 
tabs want 762M:

 VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND			
1722m 946m  22m R	24 16.5 217:29.77 X 				
1728m 504m  23m S	 0  8.8  61:07.82 plasma-desktop	
2018m 371m 6772 S	 0  6.5   2:45.60 java  			
 379m 272m 1884 S	 0  4.8   4:39.50 emerge			
2632m 127m  11m S	 0  2.2   6:48.31 pica  			
 694m 123m  21m S	 0  2.2  11:28.50 kontact			
1246m 117m  20m S	 0  2.1  27:30.34 amarok			
 757m 101m  86m S	 0  1.8 116:20.68 vmware-vmx		
 946m  94m  10m S	 0  1.7   1:37.32 chrome			
 682m  91m  15m S	 0  1.6   4:22.98 chrome			
 494m  79m  14m S	 0  1.4   0:40.28 kmymoney  		
 929m  61m  13m S	 0  1.1   2:29.96 chrome			
 328m  56m 5084 S	 0  1.0   1:46.09 kio_imap4 		
73712  49m  616 S	 0  0.9   0:10.79 screen	
 921m  48m  30m S	 0  0.8   0:01.62 systemsettings	
 573m  46m  14m S	 0  0.8   1:36.06 dolphin

> Okay, I'm used to Firefox taking much memory. I'm okay with that since
> it's the most heavily used application currently running. But why does
> Akregator need that much memory? It doesn't even have any tabs open at
> the moment and is just running minimized in the background.

Beats me.

> Virtuoso looks like an optional Soprano dependency which in turn is
> needed for Nepomuk. Are the default use flags for dev-libs/soprano
> suboptimal? What happens if I choose other flags for Soprano?

Don't know. But you can just turn off virtuoso in systemsettings-> desktop 
search.
I just turned it on again, and - now I need 271M of swap, and again my 
system becomes unresponsive due to the constant swapping that is going on. 
With 6G!

Which is another problem I think. One question is how KDE4 can need such a 
lot of memory, the other is how the system can become so unresponsive once 
its starts swapping. I used to have larger swap with less RAM, and did not 
have those performance problems. One year ago I usually had 2G tmpfs for 
/var/tmp/portage, nowadays (with 4G) I cannot emerge things while working 
with the system (like, watching videos with mplayer). It feels like as 
soon as RAM is not enough and swapping occurs, the system swaps stuff that 
it will need again immediately.

What stuff do I run? I have 8 activities/desktops, there are some 
screenshots at [1] in case someone is interested. My default session has 4 
konsoles with a total of 8 tabs, 3 dolphins with 5 views, amarok, kontact, 
kmymoney2. TV-Browser (java application which eats a lot of memory), some 
admin tools like gkrellm or diagnostic plasmoids. And a lot of browser 
tabs, around 30. Currently I'm using chromium, that seems to use less 
memory than konqueror. The number of tabs increases with uptime. Is this 
too much? Desktop effects are enabled. Today I started openoffice once, 
played a little quake3. There is a windows VM running with vmplayer, but 
that takes only 50MB. wine uses more, I'm using this today, but normally 
not.

The system is an AMD Athlon 4850e (2 cores, 2500MHz) with 4GB of RAM. 
Everything is on LVM, most partitions are LUKS-encrypted. /var/tmp/portage 
is unencrypted, and at the moment swap is also not encrypted and on my 2nd 
drive. The encryption does not be much of an overhead, when the system 
stutters, top shows a large wa(it) value, and not much CPU usage. swappiness
is set to 10.

Any ideas? I might just get another 2G, and then the problems will be gone,
but I think this would be only a workaround. 6G should be enough already
even when using lots of applications, shouldn't it`?

BTW, I emerged and tried KDE 3.5 a week ago. Cool, things were fast
there. Probably because it needs less memory. But I don't want to go back.

	Wonko

[1] http://archives.gentoo.org/gentoo-performance/msg_e4365aee884ee527dc8fb82d2c725ec4.xml

Re: [gentoo-user] KDE ridiculous memory usage

[Alan McKinnon]

Apparently, though unproven, at 16:45 on Saturday 18 September 2010, Florian 
Philipp did opine thusly:

> Hi list!
> 
> I have a bit of a problem. I'm on KDE-4.4.5 and it eats memory for
> breakfast. Directly after booting, everything is okay but the usage
> grows significantly. I wonder whether this is expected behavior.
> 
> The following statistics have been taken after 8 days of uptime during
> which the system was on standby most of the time during work days and at
> night.
> 
> free -m
>              total  used  free  shared  buffers  cached
> Mem:          3754  3588   165       0       57     258
> -/+ buffers/cache:  3271   482
> Swap:         6142   978  5163
> 
> A desktop machine that has 4GB RAM and still needs to swap?!
> 
> Excerpt from top:
>  VIRT  RES  SHR S %CPU %MEM     TIME+ COMMAND
> 1094m 484m  10m S    0 12.9  96:43.01 firefox
>  932m 471m  15m S    0 12.6   5:10.20 akregator
>  384m 303m 2856 S    0  8.1  59:43.43 virtuoso-t
>  709m 282m 2936 S    0  7.5   0:40.51 nepomukservices
>  839m 146m  15m S    0  3.9   8:37.76 thunderbird-bin
>  191m 131m  532 S    0  3.5  12:30.73 dbus-daemon
>  902m 105m 5288 S    0  2.8   0:30.16 krunner
>  263m 105m 1724 S    0  2.8   2:31.18 squid
>  255m  61m 6672 S    7  1.6 305:04.24 X
> 1106m  55m 7756 S    0  1.5   4:22.73 amarok
>  534m  54m  10m S    0  1.5   2:33.94 kopete
>  559m  52m 6536 S    0  1.4  56:52.37 nepomukservices
>  718m  38m  12m S    4  1.0 143:36.62 plasma-desktop
>  295m  33m 2048 S    0  0.9   1:59.32 mysqld
>  360m  17m 1856 S    0  0.5   0:07.56 tomboy
>  445m  16m 3392 S    0  0.4  38:54.36 nepomukservices
>  365m  14m 6356 S    1  0.4  27:38.49 konsole
>  438m  11m 4928 S    0  0.3   0:20.12 kded4
>  508m  11m 6364 S    0  0.3   0:45.79 kwin

Like I posted in another thread today, the memory columns in top do not mean 
what most people think they mean, nor are they simplistic.

The columns tell you the amount of memory that process can access. This is 
vitally important to understand. Modern memory managers in all OSes have the 
concept of shared code and shared memory. It would be insanely wasteful for 
each process to have it's own copy of all the data in RAM it ever uses. At a 
minimum, every process would need a full copy of glibc loaded into RAM.

Here's what really happens (simplistic version):

An app loads, and links to libraries it needs. They may or may not already be 
in RAM; if nor, they are loaded. Those binary images increase the amount of 
RAM the process may address. The app uses more RAM for it's own purposes (data 
it is using) and after a while lots of that data is still in RAM but no longer 
being used.

When things get tight, the kernel has a good long hard look at memory usage 
and starts chucking bits away that can be dispensed with safely. How much 
control do you, the user, have over this: none whatsoever. Why: because the 
situation is changing millions of times a second and there's no way you can 
keep up.

It's like your heart. You don't actually want to be bothered keeping the damn 
thing pumping consciously. So you let your brain stem do all that heavy 
lifting. With memory, the kernel is your brain stem.

Your numbers above look perfectly normal. Most of that RAM can and will be 
dumped when something else comes along that needs it. The clincher is your 
swap usage. After 8 days you are using only about 12% of total which indicates 
the kernel is quite happily keeping everything under control and still has 
plenty of wiggle room left to keep you humming along nicely.

The only point where this memory scheme goes wrong is when an app has a memory 
leak - it has finished with some data in RAM and does not release it. The 
chances that all your "memory hogs" all have leaks like this are very small. 

Final conclusion: you have nothing to worry about.




> 
> Okay, I'm used to Firefox taking much memory. I'm okay with that since
> it's the most heavily used application currently running. But why does
> Akregator need that much memory? It doesn't even have any tabs open at
> the moment and is just running minimized in the background.
> 
> Virtuoso looks like an optional Soprano dependency which in turn is
> needed for Nepomuk. Are the default use flags for dev-libs/soprano
> suboptimal? What happens if I choose other flags for Soprano?
> 
> The rest of the list is a bit suspicious, as well. Especially DBus and
> Kopete look like they live way beyond their means (or my means ;) ).
> 
> Do other users experience the same?
> 
> Thanks in advance!
> Florian Philipp

-- 
alan dot mckinnon at gmail dot com

[gentoo-user] Re: KDE ridiculous memory usage

[Nikos Chantziaras]

On 09/18/2010 05:45 PM, Florian Philipp wrote:
> Hi list!
>
> I have a bit of a problem. I'm on KDE-4.4.5 and it eats memory for
> breakfast. Directly after booting, everything is okay but the usage
> grows significantly. I wonder whether this is expected behavior.
>
> The following statistics have been taken after 8 days of uptime during
> which the system was on standby most of the time during work days and at
> night.
>
> free -m
>               total  used  free  shared  buffers  cached
> Mem:          3754  3588   165       0       57     258
> -/+ buffers/cache:  3271   482
> Swap:         6142   978  5163

That looks bad.  I suspect it's the semantic desktop thingy that's at 
fault (I guess it's database and indexing service must eat tons of RAM), 
since I have it disabled and this is how it looks here after 5 days uptime:

          total       used       free     shared    buffers     cached
Mem:      5973       3534       2438          0       1056       1685
-/+ buffers/cache:    793       5179
Swap:      917          0        917

(The important value is "-/+ buffers/cache:  793")

This is with KDE 4.5.1 and "semantic-desktop" USE flag disabled.

[gentoo-user] Re: KDE ridiculous memory usage

[Nikos Chantziaras]

On 09/19/2010 11:25 AM, Alan McKinnon wrote:
> Apparently, though unproven, at 16:45 on Saturday 18 September 2010, Florian
> Philipp did opine thusly:
>
>> Hi list!
>>
>> I have a bit of a problem. I'm on KDE-4.4.5 and it eats memory for
>> breakfast. Directly after booting, everything is okay but the usage
>> grows significantly. I wonder whether this is expected behavior.
>>
>> The following statistics have been taken after 8 days of uptime during
>> which the system was on standby most of the time during work days and at
>> night.
>>
>> free -m
>>               total  used  free  shared  buffers  cached
>> Mem:          3754  3588   165       0       57     258
>> -/+ buffers/cache:  3271   482
>> Swap:         6142   978  5163
>> [...]
>
> Like I posted in another thread today, the memory columns in top do not mean
> what most people think they mean, nor are they simplistic.

However, the values reported by "free -m" are somewhat useful and 
indicate that something is very wrong with memory consumption on his system.

Re: [gentoo-user] Re: KDE ridiculous memory usage

[Dale]

Nikos Chantziaras wrote:
> On 09/19/2010 11:25 AM, Alan McKinnon wrote:
>> Apparently, though unproven, at 16:45 on Saturday 18 September 2010, 
>> Florian
>> Philipp did opine thusly:
>>
>>> Hi list!
>>>
>>> I have a bit of a problem. I'm on KDE-4.4.5 and it eats memory for
>>> breakfast. Directly after booting, everything is okay but the usage
>>> grows significantly. I wonder whether this is expected behavior.
>>>
>>> The following statistics have been taken after 8 days of uptime during
>>> which the system was on standby most of the time during work days 
>>> and at
>>> night.
>>>
>>> free -m
>>>                     total  used  free  shared  buffers  cached
>>> Mem:          3754  3588   165       0       57     258
>>> -/+ buffers/cache:  3271   482
>>> Swap:         6142   978  5163
>>> [...]
>>
>> Like I posted in another thread today, the memory columns in top do 
>> not mean
>> what most people think they mean, nor are they simplistic.
>
> However, the values reported by "free -m" are somewhat useful and 
> indicate that something is very wrong with memory consumption on his 
> system.
>

This is my free -m:

root@smoker / # free -m
                     total       used       free     shared    
buffers     cached
Mem:          2024       1934         89          0        380        657
-/+ buffers/cache:      896       1127
Swap:          478              0         478
root@smoker / #

I have less memory installed but if I understand this correctly, I have 
more trouble than he does.  This install is a few years old and my rig 
is several years old.  It's been doing fine so far.  I'm also using the 
same KDE.

Currently running, KDE, Seamonkey and a nice emerge of a video package.  
The compile process is using the most memory at the moment.

Dale

:-)  :-)

[gentoo-user] Re: KDE ridiculous memory usage

[Nikos Chantziaras]

On 09/19/2010 12:15 PM, Dale wrote:
> Nikos Chantziaras wrote:
>> On 09/19/2010 11:25 AM, Alan McKinnon wrote:
>>> Apparently, though unproven, at 16:45 on Saturday 18 September 2010,
>>> Florian
>>> Philipp did opine thusly:
>>>
>>>> Hi list!
>>>>
>>>> I have a bit of a problem. I'm on KDE-4.4.5 and it eats memory for
>>>> breakfast. Directly after booting, everything is okay but the usage
>>>> grows significantly. I wonder whether this is expected behavior.
>>>>
>>>> The following statistics have been taken after 8 days of uptime during
>>>> which the system was on standby most of the time during work days
>>>> and at
>>>> night.
>>>>
>>>> free -m
>>>> total used free shared buffers cached
>>>> Mem: 3754 3588 165 0 57 258
>>>> -/+ buffers/cache: 3271 482
>>>> Swap: 6142 978 5163
>>>> [...]
>>>
>>> Like I posted in another thread today, the memory columns in top do
>>> not mean
>>> what most people think they mean, nor are they simplistic.
>>
>> However, the values reported by "free -m" are somewhat useful and
>> indicate that something is very wrong with memory consumption on his
>> system.
>>
>
> This is my free -m:
>
> root@smoker / # free -m
> total used free shared buffers cached
> Mem: 2024 1934 89 0 380 657
> -/+ buffers/cache: 896 1127
> Swap: 478 0 478
> root@smoker / #
>
> I have less memory installed but if I understand this correctly, I have
> more trouble than he does.

Why?  It reports 896MB usage vs 3271MB in Florian's system.  Looks 
pretty normal to me.

Re: [gentoo-user] Re: KDE ridiculous memory usage

[Dale]

Nikos Chantziaras wrote:
> On 09/19/2010 12:15 PM, Dale wrote:
>> Nikos Chantziaras wrote:
>>> On 09/19/2010 11:25 AM, Alan McKinnon wrote:
>>>> Apparently, though unproven, at 16:45 on Saturday 18 September 2010,
>>>> Florian
>>>> Philipp did opine thusly:
>>>>
>>>>> Hi list!
>>>>>
>>>>> I have a bit of a problem. I'm on KDE-4.4.5 and it eats memory for
>>>>> breakfast. Directly after booting, everything is okay but the usage
>>>>> grows significantly. I wonder whether this is expected behavior.
>>>>>
>>>>> The following statistics have been taken after 8 days of uptime 
>>>>> during
>>>>> which the system was on standby most of the time during work days
>>>>> and at
>>>>> night.
>>>>>
>>>>> free -m
>>>>> total used free shared buffers cached
>>>>> Mem: 3754 3588 165 0 57 258
>>>>> -/+ buffers/cache: 3271 482
>>>>> Swap: 6142 978 5163
>>>>> [...]
>>>>
>>>> Like I posted in another thread today, the memory columns in top do
>>>> not mean
>>>> what most people think they mean, nor are they simplistic.
>>>
>>> However, the values reported by "free -m" are somewhat useful and
>>> indicate that something is very wrong with memory consumption on his
>>> system.
>>>
>>
>> This is my free -m:
>>
>> root@smoker / # free -m
>> total used free shared buffers cached
>> Mem: 2024 1934 89 0 380 657
>> -/+ buffers/cache: 896 1127
>> Swap: 478 0 478
>> root@smoker / #
>>
>> I have less memory installed but if I understand this correctly, I have
>> more trouble than he does.
>
> Why?  It reports 896MB usage vs 3271MB in Florian's system.  Looks 
> pretty normal to me.
>

I THINK I read he was up for about 8 days.  I had just booted up a 
little bit ago.  Looking at the Mem line, I am using almost all of my 
memory already.  I was also keeping in mind that the OP has about double 
the memory that I have.  I'm just not sure what exactly is wrong with 
his either.  It was more of a question than anything.

He is using a lot of swap but that can be adjusted by setting the 
swappiness file with a lower value IF he wants to do that.  I have mine 
set to 20 or so.  I prefer to keep as much in memory as possible but at 
the same time, I don't want to crash if say GIMP gets a little memory 
hungry when I open 300 images all at once.  I did that once.  It took a 
while.  lol

I was always told that Linux uses memory a lot better than most other 
OS's especially M$.  Cache as much as possible and run faster which 
means it will use all the memory at some point.  Mine does that way and 
always has.   Since the kernel handles all this, I'm not sure what the 
OP can do to fix anything unless it is a kernel bug.  Then a upgrade may 
be the sure.  I guess?

Dale

:-)  :-)

Re: [gentoo-user] Re: KDE ridiculous memory usage

[Alan McKinnon]

Apparently, though unproven, at 10:54 on Sunday 19 September 2010, Nikos 
Chantziaras did opine thusly:

> On 09/19/2010 11:25 AM, Alan McKinnon wrote:
> > Apparently, though unproven, at 16:45 on Saturday 18 September 2010,
> > Florian
> > 
> > Philipp did opine thusly:
> >> Hi list!
> >> 
> >> I have a bit of a problem. I'm on KDE-4.4.5 and it eats memory for
> >> breakfast. Directly after booting, everything is okay but the usage
> >> grows significantly. I wonder whether this is expected behavior.
> >> 
> >> The following statistics have been taken after 8 days of uptime during
> >> which the system was on standby most of the time during work days and at
> >> night.
> >> 
> >> free -m
> >> 
> >>               total  used  free  shared  buffers  cached
> >> 
> >> Mem:          3754  3588   165       0       57     258
> >> -/+ buffers/cache:  3271   482
> >> Swap:         6142   978  5163
> >> [...]
> > 
> > Like I posted in another thread today, the memory columns in top do not
> > mean what most people think they mean, nor are they simplistic.
> 
> However, the values reported by "free -m" are somewhat useful and
> indicate that something is very wrong with memory consumption on his
> system.


What specific numbers and what appears to be out of place?



-- 
alan dot mckinnon at gmail dot com

[gentoo-user] Re: KDE ridiculous memory usage

[Nikos Chantziaras]

On 09/19/2010 01:12 PM, Alan McKinnon wrote:
> Apparently, though unproven, at 10:54 on Sunday 19 September 2010, Nikos
> Chantziaras did opine thusly:
>
>> On 09/19/2010 11:25 AM, Alan McKinnon wrote:
>>> Apparently, though unproven, at 16:45 on Saturday 18 September 2010,
>>> Florian
>>>
>>> Philipp did opine thusly:
>>>> Hi list!
>>>>
>>>> I have a bit of a problem. I'm on KDE-4.4.5 and it eats memory for
>>>> breakfast. Directly after booting, everything is okay but the usage
>>>> grows significantly. I wonder whether this is expected behavior.
>>>>
>>>> The following statistics have been taken after 8 days of uptime during
>>>> which the system was on standby most of the time during work days and at
>>>> night.
>>>>
>>>> free -m
>>>>
>>>>                total  used  free  shared  buffers  cached
>>>>
>>>> Mem:          3754  3588   165       0       57     258
>>>> -/+ buffers/cache:  3271   482
>>>> Swap:         6142   978  5163
>>>> [...]
>>>
>>> Like I posted in another thread today, the memory columns in top do not
>>> mean what most people think they mean, nor are they simplistic.
>>
>> However, the values reported by "free -m" are somewhat useful and
>> indicate that something is very wrong with memory consumption on his
>> system.
>
>
> What specific numbers and what appears to be out of place?

This:

-/+ buffers/cache:  3271   482

Re: [gentoo-user] KDE ridiculous memory usage

[Al]

> free -m
>             total  used  free  shared  buffers  cached
> Mem:          3754  3588   165       0       57     258

3588 of 3754 is free, AFAIK not used at all.  Plug off 3500 and sell it.

If your system is slow maybe from managing all that unusued memory. ;-)

Al

Re: [gentoo-user] KDE ridiculous memory usage

[Dale]

Al wrote:
>> free -m
>>              total  used  free  shared  buffers  cached
>> Mem:          3754  3588   165       0       57     258
>>      
> 3588 of 3754 is free, AFAIK not used at all.  Plug off 3500 and sell it.
>
> If your system is slow maybe from managing all that unusued memory. ;-)
>
> Al
>
>    

Actually, the 3588 is what is used.  The 165 is what is free.  Maybe the 
email program you are using is not lining the columns up properly.

Dale

:-)  :-)

Re: [gentoo-user] KDE ridiculous memory usage

[Alan McKinnon]

Apparently, though unproven, at 12:37 on Sunday 19 September 2010, Al did 
opine thusly:

> > free -m
> >             total  used  free  shared  buffers  cached
> > Mem:          3754  3588   165       0       57     258
> 
> 3588 of 3754 is free, AFAIK not used at all.  Plug off 3500 and sell it.
> 
> If your system is slow maybe from managing all that unusued memory. ;-)
> 
> Al


I think someone needs to go study how linux memory management works, and what 
buffers and cache really are


RULE NUMBER ONE OF LINUX MEMORY:

SUPERFICIAL UTILITIES LIKE free WILL *ALWAYS* REPORT ALMOST ALL MEMORY IN USE.
REASON: IT *IS* IN USE.



-- 
alan dot mckinnon at gmail dot com

Re: [gentoo-user] KDE ridiculous memory usage

[Alex Schuster]

Alan McKinnon writes:

> Apparently, though unproven, at 16:45 on Saturday 18 September 2010,
> Florian Philipp did opine thusly:

> > I have a bit of a problem. I'm on KDE-4.4.5 and it eats memory for
> > breakfast. Directly after booting, everything is okay but the usage
> > grows significantly. I wonder whether this is expected behavior.
> > 
> > The following statistics have been taken after 8 days of uptime
> > during which the system was on standby most of the time during work
> > days and at night.
> > 
> > free -m
> > 
> >              total  used  free  shared  buffers  cached
> > 
> > Mem:          3754  3588   165       0       57     258
> > -/+ buffers/cache:  3271   482
> > Swap:         6142   978  5163
> > 
> > A desktop machine that has 4GB RAM and still needs to swap?!

What I forgot to ask: Do you feel the performance becomes bad? Does the 
system feel more responsive again when you restart KDM and log in again?

I don't mind the system growing swap, that's normal, but now, as soon as 
significant swapping starts, the system becomes slow. I don't know why.

> > Excerpt from top:
> >  VIRT  RES  SHR S %CPU %MEM     TIME+ COMMAND
> > 
> > 1094m 484m  10m S    0 12.9  96:43.01 firefox
> > 
> >  932m 471m  15m S    0 12.6   5:10.20 akregator
> >  384m 303m 2856 S    0  8.1  59:43.43 virtuoso-t
> >  709m 282m 2936 S    0  7.5   0:40.51 nepomukservices
> >  839m 146m  15m S    0  3.9   8:37.76 thunderbird-bin
> >  191m 131m  532 S    0  3.5  12:30.73 dbus-daemon
> >  902m 105m 5288 S    0  2.8   0:30.16 krunner
> >  263m 105m 1724 S    0  2.8   2:31.18 squid
> >  255m  61m 6672 S    7  1.6 305:04.24 X
> > 
> > 1106m  55m 7756 S    0  1.5   4:22.73 amarok
> > 
> >  534m  54m  10m S    0  1.5   2:33.94 kopete
> >  559m  52m 6536 S    0  1.4  56:52.37 nepomukservices
> >  718m  38m  12m S    4  1.0 143:36.62 plasma-desktop
> >  295m  33m 2048 S    0  0.9   1:59.32 mysqld
> >  360m  17m 1856 S    0  0.5   0:07.56 tomboy
> >  445m  16m 3392 S    0  0.4  38:54.36 nepomukservices
> >  365m  14m 6356 S    1  0.4  27:38.49 konsole
> >  438m  11m 4928 S    0  0.3   0:20.12 kded4
> >  508m  11m 6364 S    0  0.3   0:45.79 kwin
> 
> Like I posted in another thread today, the memory columns in top do not
> mean what most people think they mean, nor are they simplistic.

You gave the example of Thunderbird using 150M and Firefox 180M, but 
together they would not use 330M because some stuff is shared. Hm, isn't 
this what the SHR column in top is for? In Florian's case, there is 
firefox with 484M in the RES column and thunderbird with 146M, but the SHA 
column gives 10M + 15M, so only 25M of 630M are shared?

	Wonko

Re: [gentoo-user] KDE ridiculous memory usage

[Florian Philipp]

Am 18.09.2010 22:19, schrieb Alex Schuster:
[...]
>
> I used to restart kdm once per day in order to free memory. If I did not
> do this, KDE4 became nearly unsusabe.
>
Yeah, logout - logon seems to resolve my problem temporarily, as well.

> 
> Now this looks different here. I have X with 946M, plasma-desktop with 
> 505M, that's 15 times the memory you need. Then comes java with 371M (for 
> TV-Browser - yes, 371MB just for showing the TV programme!), emerge wants 
> 272M while emerging openoffice. Chromium also needs much memory, my 33 
> tabs want 762M:

Wow, especially X's usage makes me wonder whether this is a kernel bug.

> 
> Which is another problem I think. One question is how KDE4 can need such a 
> lot of memory, the other is how the system can become so unresponsive once 
> its starts swapping. I used to have larger swap with less RAM, and did not 
> have those performance problems. One year ago I usually had 2G tmpfs for 
> /var/tmp/portage, nowadays (with 4G) I cannot emerge things while working 
> with the system (like, watching videos with mplayer). It feels like as 
> soon as RAM is not enough and swapping occurs, the system swaps stuff that 
> it will need again immediately.

Hmm, maybe it is the usage pattern that matters. I guess X (or whatever
gets swapped out in your case) wants to access all the data, maybe for a
cyclic refresh or something, it blocks for some time.

That's the good thing about normal memory leaks: Whatever is leaked, it
is normally not accessed again, anyway.

> 
> The system is an AMD Athlon 4850e (2 cores, 2500MHz) with 4GB of RAM. 
> Everything is on LVM, most partitions are LUKS-encrypted. /var/tmp/portage 
> is unencrypted, and at the moment swap is also not encrypted and on my 2nd 
> drive. The encryption does not be much of an overhead, when the system 
> stutters, top shows a large wa(it) value, and not much CPU usage. swappiness
> is set to 10.
> 

My system is nearly completely on LUKS and LVM. That doesn't seem to be
the problym in my case, either.

> Any ideas? I might just get another 2G, and then the problems will be gone,
> but I think this would be only a workaround. 6G should be enough already
> even when using lots of applications, shouldn't it`?
> 
> BTW, I emerged and tried KDE 3.5 a week ago. Cool, things were fast
> there. Probably because it needs less memory. But I don't want to go back.
> 

The interesting thing is that I have a netbook with a minimal KDE-4 on
it. It doesn't need more than 150M of its 512M memory. Of course it
doesn't have Semantic Desktop and all that but it still works good and
is responsive as hell.

Re: [gentoo-user] KDE ridiculous memory usage

[Florian Philipp]

Am 19.09.2010 13:34, schrieb Alex Schuster:
> Alan McKinnon writes:
>> Like I posted in another thread today, the memory columns in top do not
>> mean what most people think they mean, nor are they simplistic.
> 
> You gave the example of Thunderbird using 150M and Firefox 180M, but 
> together they would not use 330M because some stuff is shared. Hm, isn't 
> this what the SHR column in top is for? In Florian's case, there is 
> firefox with 484M in the RES column and thunderbird with 146M, but the SHA 
> column gives 10M + 15M, so only 25M of 630M are shared?
> 

I thought the SHR column is about shared memory like System-V SHM, mmap
and Pipes when used for inter-process communication. But I could be wrong.

Re: [gentoo-user] KDE ridiculous memory usage

[Alan McKinnon]

Apparently, though unproven, at 13:34 on Sunday 19 September 2010, Alex 
Schuster did opine thusly:

> Alan McKinnon writes:
> > Apparently, though unproven, at 16:45 on Saturday 18 September 2010,
> > 
> > Florian Philipp did opine thusly:
> > > I have a bit of a problem. I'm on KDE-4.4.5 and it eats memory for
> > > breakfast. Directly after booting, everything is okay but the usage
> > > grows significantly. I wonder whether this is expected behavior.
> > > 
> > > The following statistics have been taken after 8 days of uptime
> > > during which the system was on standby most of the time during work
> > > days and at night.
> > > 
> > > free -m
> > > 
> > >              total  used  free  shared  buffers  cached
> > > 
> > > Mem:          3754  3588   165       0       57     258
> > > -/+ buffers/cache:  3271   482
> > > Swap:         6142   978  5163
> > > 
> > > A desktop machine that has 4GB RAM and still needs to swap?!
> 
> What I forgot to ask: Do you feel the performance becomes bad? Does the
> system feel more responsive again when you restart KDM and log in again?
> 
> I don't mind the system growing swap, that's normal, but now, as soon as
> significant swapping starts, the system becomes slow. I don't know why.

It's swapping. It will become slow. Disks are millions of time slower than 
RAM.

> 
> > > Excerpt from top:
> > >  VIRT  RES  SHR S %CPU %MEM     TIME+ COMMAND
> > > 
> > > 1094m 484m  10m S    0 12.9  96:43.01 firefox
> > > 
> > >  932m 471m  15m S    0 12.6   5:10.20 akregator
> > >  384m 303m 2856 S    0  8.1  59:43.43 virtuoso-t
> > >  709m 282m 2936 S    0  7.5   0:40.51 nepomukservices
> > >  839m 146m  15m S    0  3.9   8:37.76 thunderbird-bin
> > >  191m 131m  532 S    0  3.5  12:30.73 dbus-daemon
> > >  902m 105m 5288 S    0  2.8   0:30.16 krunner
> > >  263m 105m 1724 S    0  2.8   2:31.18 squid
> > >  255m  61m 6672 S    7  1.6 305:04.24 X
> > > 
> > > 1106m  55m 7756 S    0  1.5   4:22.73 amarok
> > > 
> > >  534m  54m  10m S    0  1.5   2:33.94 kopete
> > >  559m  52m 6536 S    0  1.4  56:52.37 nepomukservices
> > >  718m  38m  12m S    4  1.0 143:36.62 plasma-desktop
> > >  295m  33m 2048 S    0  0.9   1:59.32 mysqld
> > >  360m  17m 1856 S    0  0.5   0:07.56 tomboy
> > >  445m  16m 3392 S    0  0.4  38:54.36 nepomukservices
> > >  365m  14m 6356 S    1  0.4  27:38.49 konsole
> > >  438m  11m 4928 S    0  0.3   0:20.12 kded4
> > >  508m  11m 6364 S    0  0.3   0:45.79 kwin
> > 
> > Like I posted in another thread today, the memory columns in top do not
> > mean what most people think they mean, nor are they simplistic.
> 
> You gave the example of Thunderbird using 150M and Firefox 180M, but
> together they would not use 330M because some stuff is shared. Hm, isn't
> this what the SHR column in top is for? In Florian's case, there is
> firefox with 484M in the RES column and thunderbird with 146M, but the SHA
> column gives 10M + 15M, so only 25M of 630M are shared?

Yes that's true. I sucked the 150 && 180 numbers out of my ass.

The post was to highlight common problems with reading top output, not to 
diagnose any problem he might be having.



-- 
alan dot mckinnon at gmail dot com

Re: [gentoo-user] KDE ridiculous memory usage

[Florian Philipp]

Am 19.09.2010 10:25, schrieb Alan McKinnon:
[...]
> Like I posted in another thread today, the memory columns in top do not mean 
> what most people think they mean, nor are they simplistic.
> 
> The columns tell you the amount of memory that process can access. This is 
> vitally important to understand. Modern memory managers in all OSes have the 
> concept of shared code and shared memory. It would be insanely wasteful for 
> each process to have it's own copy of all the data in RAM it ever uses. At a 
> minimum, every process would need a full copy of glibc loaded into RAM.
> 
> Here's what really happens (simplistic version):
> 
> An app loads, and links to libraries it needs. They may or may not already be 
> in RAM; if nor, they are loaded. Those binary images increase the amount of 
> RAM the process may address. The app uses more RAM for it's own purposes (data 
> it is using) and after a while lots of that data is still in RAM but no longer 
> being used.
> 
> When things get tight, the kernel has a good long hard look at memory usage 
> and starts chucking bits away that can be dispensed with safely. How much 
> control do you, the user, have over this: none whatsoever. Why: because the 
> situation is changing millions of times a second and there's no way you can 
> keep up.
> 
> It's like your heart. You don't actually want to be bothered keeping the damn 
> thing pumping consciously. So you let your brain stem do all that heavy 
> lifting. With memory, the kernel is your brain stem.
> 
> Your numbers above look perfectly normal. Most of that RAM can and will be 
> dumped when something else comes along that needs it. The clincher is your 
> swap usage. After 8 days you are using only about 12% of total which indicates 
> the kernel is quite happily keeping everything under control and still has 
> plenty of wiggle room left to keep you humming along nicely.
> 
> The only point where this memory scheme goes wrong is when an app has a memory 
> leak - it has finished with some data in RAM and does not release it. The 
> chances that all your "memory hogs" all have leaks like this are very small. 
> 
> Final conclusion: you have nothing to worry about.
> 

I disagree on that last point. While it might be true that some of the
statistics are not correct, I have a feeling that it is not acceptable
or normal that a simple desktop system is not able to free enough memory
to have more that 1/8 of it available for cache.

I mean, my old system had 2 GB RAM and an equivalent Gnome system on it.
It needed swap as well due to Firefox and Eclipse eating memory. But
otherwise its usage was far less than what I see here.

Re: [gentoo-user] KDE ridiculous memory usage

[Volker Armin Hemmann]

On Saturday 18 September 2010, Florian Philipp wrote:
>

> Okay, I'm used to Firefox taking much memory. I'm okay with that since
> it's the most heavily used application currently running. But why does
> Akregator need that much memory? It doesn't even have any tabs open at
> the moment and is just running minimized in the background.

so akregator has a mem leak. Kill and restart it. And before you do: bug 
report with kde.

Re: [gentoo-user] KDE ridiculous memory usage

[Al]

>
> Actually, the 3588 is what is used.  The 165 is what is free.  Maybe the
> email program you are using is not lining the columns up properly.
>

Maybe you are right. Headers out of alignment.

Al

Re: [gentoo-user] KDE ridiculous memory usage

[Yohan Pereira]

ok first of .. i dont run my comp (laptop) for that long, although i am 
planning to start using hibernate. its usually up the whole day though.

secondly im on kde4.5.1 (but i dont remeber having such bad memory problems 
with the version your running). 

Krunner's neopomuk plugin leaks memory, everytime you search for something 
that returns any nepomuk results, krunners memory usage jumps by ~10 mb and 
never decreases. disable it if you have it enabled and restart it. there is a 
bug report here https://bugs.kde.org/show_bug.cgi?id=224287
maybe we should file a new one.

if you still want nepomuk, you can limit virtuoso-t's memory usage in 
nepomuk's kcm module(last tab), it usually abides by those values, however 
once for me it went over that and continued to grow and nepomuk became 
unresponsive (none of the search querries worked) and i had to kill it. ive 
tried to reproduce that bug to no awail. 

i use akregator too and i dont find it such memory hog (maybe its the version i 
use 4.4.6)
8009 yohan     20   0  502m  52m  20m S    0  1.3   0:03.54 akregator

you could try upgrading, i think i found the newer version a bit more snapiper 
(but thats probably psychological).

this is the result of free -m
                   total       used       free     shared    buffers     cached
Mem:          3928       2409       1519          0        767        640
-/+ buffers/cache:       1001       2926
Swap:         6981          0       6981
(i thoought i needed all that swap for hibernating and thats why its so big )
-- 
- Yohan Pereira.