[gentoo-user] kde-apps/kde-l10n-16.04.3:5/5::gentoo conflicting with kde-apps/kdepim-l10n-15.12.3:5/5::gentoo

[gentoo-user] kde-apps/kde-l10n-16.04.3:5/5::gentoo conflicting with kde-apps/kdepim-l10n-15.12.3:5/5::gentoo

[Michael Mol]

[-- Attachment #1: Type: text/plain, Size: 1010 bytes --]

Been getting this in my email every morning for several days now. Rather 
expected it to clear by now, but since it hasn't, and googling doesn't seem to 
indicate anyone has noted the issue...

 * Error: The above package list contains packages which cannot be
 * installed at the same time on the same system.

  (kde-apps/kde-l10n-16.04.3:5/5::gentoo, installed) pulled in by
    >=kde-apps/kde-l10n-16.04.3 required by (kde-apps/kde4-
l10n-16.04.3:4/4::gentoo, installed)
    >=kde-apps/kde-l10n-16.04.3 required by (kde-apps/kde-apps-
meta-16.04.3:5/5::gentoo, installed)

  (kde-apps/kdepim-l10n-15.12.3:5/5::gentoo, ebuild scheduled for merge) 
pulled in by
    >=kde-apps/kdepim-l10n-4.14.3 required by (kde-apps/kdepim-
meta-4.14.11_pre20160211:4/4::gentoo, installed)


Now, it's not clear, if I'd like to continue using both KMail and non-
deprecated kde-apps, what to do here. Just hope that kdepim gets updated to 
qt5 soon? I'd pitch in, but I don't have the time.

-- 
:wq

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 473 bytes --]

Re: [gentoo-user] kde-apps/kde-l10n-16.04.3:5/5::gentoo conflicting with kde-apps/kdepim-l10n-15.12.3:5/5::gentoo

[Alan McKinnon]

On 08/08/2016 17:02, Michael Mol wrote:
> Been getting this in my email every morning for several days now. Rather 
> expected it to clear by now, but since it hasn't, and googling doesn't seem to 
> indicate anyone has noted the issue...
> 
>  * Error: The above package list contains packages which cannot be
>  * installed at the same time on the same system.
> 
>   (kde-apps/kde-l10n-16.04.3:5/5::gentoo, installed) pulled in by
>     >=kde-apps/kde-l10n-16.04.3 required by (kde-apps/kde4-
> l10n-16.04.3:4/4::gentoo, installed)
>     >=kde-apps/kde-l10n-16.04.3 required by (kde-apps/kde-apps-
> meta-16.04.3:5/5::gentoo, installed)
> 
>   (kde-apps/kdepim-l10n-15.12.3:5/5::gentoo, ebuild scheduled for merge) 
> pulled in by
>     >=kde-apps/kdepim-l10n-4.14.3 required by (kde-apps/kdepim-
> meta-4.14.11_pre20160211:4/4::gentoo, installed)
> 
> 
> Now, it's not clear, if I'd like to continue using both KMail and non-
> deprecated kde-apps, what to do here. Just hope that kdepim gets updated to 
> qt5 soon? I'd pitch in, but I don't have the time.
> 


please post the portion of the output/mail that shows the blockers.

-- 
Alan McKinnon
alan.mckinnon@gmail.com

Re: [gentoo-user] kde-apps/kde-l10n-16.04.3:5/5::gentoo conflicting with kde-apps/kdepim-l10n-15.12.3:5/5::gentoo

[Michael Mol]

[-- Attachment #1.1: Type: text/plain, Size: 12315 bytes --]

=2D-nextPart2216931.89McLYzdAM
Content-Transfer-Encoding: 7Bit
Content-Type: text/plain; charset="iso-8859-1"


On Monday, August 08, 2016 06:52:15 PM Alan McKinnon wrote:
> On 08/08/2016 17:02, Michael Mol wrote:
> > Been getting this in my email every morning for several days now. Rather
> > expected it to clear by now, but since it hasn't, and googling doesn't
> > seem to indicate anyone has noted the issue...
> > 
> >  * Error: The above package list contains packages which cannot be
> >  * installed at the same time on the same system.
> >  
> >   (kde-apps/kde-l10n-16.04.3:5/5::gentoo, installed) pulled in by
> >   
> >     >=kde-apps/kde-l10n-16.04.3 required by (kde-apps/kde4-
> > 
> > l10n-16.04.3:4/4::gentoo, installed)
> > 
> >     >=kde-apps/kde-l10n-16.04.3 required by (kde-apps/kde-apps-
> > 
> > meta-16.04.3:5/5::gentoo, installed)
> > 
> >   (kde-apps/kdepim-l10n-15.12.3:5/5::gentoo, ebuild scheduled for merge)
> > 
> > pulled in by
> > 
> >     >=kde-apps/kdepim-l10n-4.14.3 required by (kde-apps/kdepim-
> > 
> > meta-4.14.11_pre20160211:4/4::gentoo, installed)
> > 
> > 
> > Now, it's not clear, if I'd like to continue using both KMail and non-
> > deprecated kde-apps, what to do here. Just hope that kdepim gets updated
> > to
> > qt5 soon? I'd pitch in, but I don't have the time.
> 
> please post the portion of the output/mail that shows the blockers.

[nomerge       ] kde-apps/kde-apps-meta-16.04.3 
[nomerge       ] kde-apps/kdepim-meta-4.14.11_pre20160211 
[ebuild  NS    ]  kde-apps/kdepim-l10n-15.12.3 [4.14.3-r1] USE="-debug -handbook" 
L10N="-ar -bg -bs -ca -ca-valencia -cs -da -de -el -en-GB -eo -es -et -eu -fa -fi -fr -ga -gl -
he -hi -hr -hu -ia -id -is -it -ja -kk -km -ko -lt -lv -mr -nb -nds -nl -nn -pa -pl -pt -pt-BR -ro -ru 
=2Dsk -sl -sr -sv -tr -ug -uk -wa -zh-CN -zh-TW" 
[blocks b      ]   kde-apps/kdepim-l10n:4 ("kde-apps/kdepim-l10n:4" is blocking kde-
apps/kdepim-l10n-15.12.3)
[uninstall     ]    kde-apps/kdepim-l10n-4.14.3-r1 
[blocks B      ] <kde-apps/kdepim-l10n-16.04.3:5 ("<kde-apps/kdepim-l10n-16.04.3:5" is 
blocking kde-apps/kde-l10n-16.04.3)

=2D- 
:wq

=2D-nextPart2216931.89McLYzdAM
Content-Transfer-Encoding: quoted-printable
Content-Type: text/html; charset="iso-8859-1"

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0//EN" "http://www.w3.org/TR/=
REC-html40/strict.dtd">
<html><head><meta name=3D"qrichtext" content=3D"1" /><style type=3D"tex=
t/css">
p, li { white-space: pre-wrap; }
</style></head><body style=3D" font-family:'Sans Serif'; font-size:9pt;=
 font-weight:400; font-style:normal;">
<p style=3D"-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px=
; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0p=
x; ">&nbsp;</p>
<p style=3D"-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px=
; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0p=
x; ">&nbsp;</p>
<p style=3D" margin-top:0px; margin-bottom:0px; margin-left:0px; margin=
=2Dright:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">On =
Monday, August 08, 2016 06:52:15 PM Alan McKinnon wrote:</p>
<p style=3D" margin-top:0px; margin-bottom:0px; margin-left:0px; margin=
=2Dright:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">&gt=
; On 08/08/2016 17:02, Michael Mol wrote:</p>
<p style=3D" margin-top:0px; margin-bottom:0px; margin-left:0px; margin=
=2Dright:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">&gt=
; &gt; Been getting this in my email every morning for several days now=
. Rather</p>
<p style=3D" margin-top:0px; margin-bottom:0px; margin-left:0px; margin=
=2Dright:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">&gt=
; &gt; expected it to clear by now, but since it hasn't, and googling d=
oesn't</p>
<p style=3D" margin-top:0px; margin-bottom:0px; margin-left:0px; margin=
=2Dright:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">&gt=
; &gt; seem to indicate anyone has noted the issue...</p>
<p style=3D" margin-top:0px; margin-bottom:0px; margin-left:0px; margin=
=2Dright:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">&gt=
; &gt; </p>
<p style=3D" margin-top:0px; margin-bottom:0px; margin-left:0px; margin=
=2Dright:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">&gt=
; &gt;  * Error: The above package list contains packages which cannot =
be</p>
<p style=3D" margin-top:0px; margin-bottom:0px; margin-left:0px; margin=
=2Dright:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">&gt=
; &gt;  * installed at the same time on the same system.</p>
<p style=3D" margin-top:0px; margin-bottom:0px; margin-left:0px; margin=
=2Dright:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">&gt=
; &gt;  </p>
<p style=3D" margin-top:0px; margin-bottom:0px; margin-left:0px; margin=
=2Dright:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">&gt=
; &gt;   (kde-apps/kde-l10n-16.04.3:5/5::gentoo, installed) pulled in b=
y</p>
<p style=3D" margin-top:0px; margin-bottom:0px; margin-left:0px; margin=
=2Dright:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">&gt=
; &gt;   </p>
<p style=3D" margin-top:0px; margin-bottom:0px; margin-left:0px; margin=
=2Dright:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">&gt=
; &gt;     &gt;=3Dkde-apps/kde-l10n-16.04.3 required by (kde-apps/kde4-=
</p>
<p style=3D" margin-top:0px; margin-bottom:0px; margin-left:0px; margin=
=2Dright:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">&gt=
; &gt; </p>
<p style=3D" margin-top:0px; margin-bottom:0px; margin-left:0px; margin=
=2Dright:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">&gt=
; &gt; l10n-16.04.3:4/4::gentoo, installed)</p>
<p style=3D" margin-top:0px; margin-bottom:0px; margin-left:0px; margin=
=2Dright:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">&gt=
; &gt; </p>
<p style=3D" margin-top:0px; margin-bottom:0px; margin-left:0px; margin=
=2Dright:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">&gt=
; &gt;     &gt;=3Dkde-apps/kde-l10n-16.04.3 required by (kde-apps/kde-a=
pps-</p>
<p style=3D" margin-top:0px; margin-bottom:0px; margin-left:0px; margin=
=2Dright:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">&gt=
; &gt; </p>
<p style=3D" margin-top:0px; margin-bottom:0px; margin-left:0px; margin=
=2Dright:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">&gt=
; &gt; meta-16.04.3:5/5::gentoo, installed)</p>
<p style=3D" margin-top:0px; margin-bottom:0px; margin-left:0px; margin=
=2Dright:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">&gt=
; &gt; </p>
<p style=3D" margin-top:0px; margin-bottom:0px; margin-left:0px; margin=
=2Dright:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">&gt=
; &gt;   (kde-apps/kdepim-l10n-15.12.3:5/5::gentoo, ebuild scheduled fo=
r merge)</p>
<p style=3D" margin-top:0px; margin-bottom:0px; margin-left:0px; margin=
=2Dright:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">&gt=
; &gt; </p>
<p style=3D" margin-top:0px; margin-bottom:0px; margin-left:0px; margin=
=2Dright:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">&gt=
; &gt; pulled in by</p>
<p style=3D" margin-top:0px; margin-bottom:0px; margin-left:0px; margin=
=2Dright:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">&gt=
; &gt; </p>
<p style=3D" margin-top:0px; margin-bottom:0px; margin-left:0px; margin=
=2Dright:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">&gt=
; &gt;     &gt;=3Dkde-apps/kdepim-l10n-4.14.3 required by (kde-apps/kde=
pim-</p>
<p style=3D" margin-top:0px; margin-bottom:0px; margin-left:0px; margin=
=2Dright:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">&gt=
; &gt; </p>
<p style=3D" margin-top:0px; margin-bottom:0px; margin-left:0px; margin=
=2Dright:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">&gt=
; &gt; meta-4.14.11_pre20160211:4/4::gentoo, installed)</p>
<p style=3D" margin-top:0px; margin-bottom:0px; margin-left:0px; margin=
=2Dright:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">&gt=
; &gt; </p>
<p style=3D" margin-top:0px; margin-bottom:0px; margin-left:0px; margin=
=2Dright:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">&gt=
; &gt; </p>
<p style=3D" margin-top:0px; margin-bottom:0px; margin-left:0px; margin=
=2Dright:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">&gt=
; &gt; Now, it's not clear, if I'd like to continue using both KMail an=
d non-</p>
<p style=3D" margin-top:0px; margin-bottom:0px; margin-left:0px; margin=
=2Dright:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">&gt=
; &gt; deprecated kde-apps, what to do here. Just hope that kdepim gets=
 updated</p>
<p style=3D" margin-top:0px; margin-bottom:0px; margin-left:0px; margin=
=2Dright:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">&gt=
; &gt; to</p>
<p style=3D" margin-top:0px; margin-bottom:0px; margin-left:0px; margin=
=2Dright:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">&gt=
; &gt; qt5 soon? I'd pitch in, but I don't have the time.</p>
<p style=3D" margin-top:0px; margin-bottom:0px; margin-left:0px; margin=
=2Dright:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">&gt=
; </p>
<p style=3D" margin-top:0px; margin-bottom:0px; margin-left:0px; margin=
=2Dright:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">&gt=
; please post the portion of the output/mail that shows the blockers.</=
p>
<p style=3D"-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px=
; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0p=
x; ">&nbsp;</p>
<p style=3D" margin-top:0px; margin-bottom:0px; margin-left:0px; margin=
=2Dright:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><sp=
an style=3D" font-size:12px; color:#eff0f1;">[nomerge=A0=A0=A0=A0=A0=A0=
=A0] kde-apps/kde-apps-meta-16.04.3=A0</span></p>
<p style=3D" margin-top:0px; margin-bottom:0px; margin-left:0px; margin=
=2Dright:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><sp=
an style=3D" font-size:12px; color:#eff0f1;">[nomerge=A0=A0=A0=A0=A0=A0=
=A0] kde-apps/kdepim-meta-4.14.11_pre20160211=A0</span></p>
<p style=3D" margin-top:0px; margin-bottom:0px; margin-left:0px; margin=
=2Dright:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><sp=
an style=3D" font-size:12px; color:#eff0f1;">[ebuild=A0=A0NS=A0=A0=A0=A0=
]=A0=A0kde-apps/kdepim-l10n-15.12.3 [4.14.3-r1] USE=3D&quot;-debug -han=
dbook&quot; L10N=3D&quot;-ar -bg -bs -ca -ca-valencia -cs -da -de -el -=
en-GB -eo -es -et -eu -fa -fi -fr -ga -gl -he -hi -hr -hu -ia -id -is -=
it -ja -kk -km -ko -lt -lv -mr -nb -nds -nl -nn -pa -pl -pt -pt-BR -ro =
=2Dru -sk -sl -sr -sv -tr -ug -uk -wa -zh-CN -zh-TW&quot;=A0</span></p>
<p style=3D" margin-top:0px; margin-bottom:0px; margin-left:0px; margin=
=2Dright:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><sp=
an style=3D" font-size:12px; color:#eff0f1;">[blocks b=A0=A0=A0=A0=A0=A0=
]=A0=A0=A0kde-apps/kdepim-l10n:4 (&quot;kde-apps/kdepim-l10n:4&quot; is=
 blocking kde-apps/kdepim-l10n-15.12.3)</span></p>
<p style=3D" margin-top:0px; margin-bottom:0px; margin-left:0px; margin=
=2Dright:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><sp=
an style=3D" font-size:12px; color:#eff0f1;">[uninstall=A0=A0=A0=A0=A0]=
=A0=A0=A0=A0kde-apps/kdepim-l10n-4.14.3-r1=A0</span></p>
<p style=3D" margin-top:0px; margin-bottom:0px; margin-left:0px; margin=
=2Dright:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><sp=
an style=3D" font-size:12px; color:#eff0f1;">[blocks B=A0=A0=A0=A0=A0=A0=
] &lt;kde-apps/kdepim-l10n-16.04.3:5 (&quot;&lt;kde-apps/kdepim-l10n-16=
.04.3:5&quot; is blocking kde-apps/kde-l10n-16.04.3)</span></p>
<p style=3D"-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px=
; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0p=
x; ">&nbsp;</p>
<p style=3D" margin-top:0px; margin-bottom:0px; margin-left:0px; margin=
=2Dright:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">-- =
</p>
<p style=3D" margin-top:0px; margin-bottom:0px; margin-left:0px; margin=
=2Dright:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">:wq=
</p></body></html>
=2D-nextPart2216931.89McLYzdAM--
This is a multi-part message in MIME format.

[-- Attachment #1.2: Type: text/plain, Size: 2016 bytes --]


On Monday, August 08, 2016 06:52:15 PM Alan McKinnon wrote:
> On 08/08/2016 17:02, Michael Mol wrote:
> > Been getting this in my email every morning for several days now. Rather
> > expected it to clear by now, but since it hasn't, and googling doesn't
> > seem to indicate anyone has noted the issue...
> > 
> >  * Error: The above package list contains packages which cannot be
> >  * installed at the same time on the same system.
> >  
> >   (kde-apps/kde-l10n-16.04.3:5/5::gentoo, installed) pulled in by
> >   
> >     >=kde-apps/kde-l10n-16.04.3 required by (kde-apps/kde4-
> > 
> > l10n-16.04.3:4/4::gentoo, installed)
> > 
> >     >=kde-apps/kde-l10n-16.04.3 required by (kde-apps/kde-apps-
> > 
> > meta-16.04.3:5/5::gentoo, installed)
> > 
> >   (kde-apps/kdepim-l10n-15.12.3:5/5::gentoo, ebuild scheduled for merge)
> > 
> > pulled in by
> > 
> >     >=kde-apps/kdepim-l10n-4.14.3 required by (kde-apps/kdepim-
> > 
> > meta-4.14.11_pre20160211:4/4::gentoo, installed)
> > 
> > 
> > Now, it's not clear, if I'd like to continue using both KMail and non-
> > deprecated kde-apps, what to do here. Just hope that kdepim gets updated
> > to
> > qt5 soon? I'd pitch in, but I don't have the time.
> 
> please post the portion of the output/mail that shows the blockers.

[nomerge       ] kde-apps/kde-apps-meta-16.04.3 
[nomerge       ] kde-apps/kdepim-meta-4.14.11_pre20160211 
[ebuild  NS    ]  kde-apps/kdepim-l10n-15.12.3 [4.14.3-r1] USE="-debug -handbook" 
L10N="-ar -bg -bs -ca -ca-valencia -cs -da -de -el -en-GB -eo -es -et -eu -fa -fi -fr -ga -gl -
he -hi -hr -hu -ia -id -is -it -ja -kk -km -ko -lt -lv -mr -nb -nds -nl -nn -pa -pl -pt -pt-BR -ro -ru 
-sk -sl -sr -sv -tr -ug -uk -wa -zh-CN -zh-TW" 
[blocks b      ]   kde-apps/kdepim-l10n:4 ("kde-apps/kdepim-l10n:4" is blocking kde-
apps/kdepim-l10n-15.12.3)
[uninstall     ]    kde-apps/kdepim-l10n-4.14.3-r1 
[blocks B      ] <kde-apps/kdepim-l10n-16.04.3:5 ("<kde-apps/kdepim-l10n-16.04.3:5" is 
blocking kde-apps/kde-l10n-16.04.3)

-- 
:wq

[-- Attachment #1.3: Type: text/html, Size: 9487 bytes --]

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 473 bytes --]

Re: [gentoo-user] kde-apps/kde-l10n-16.04.3:5/5::gentoo conflicting with kde-apps/kdepim-l10n-15.12.3:5/5::gentoo

[Alan McKinnon]

On 08/08/2016 19:20, Michael Mol wrote:
>  
> 
>  
> 
> On Monday, August 08, 2016 06:52:15 PM Alan McKinnon wrote:
> 
>> On 08/08/2016 17:02, Michael Mol wrote:
> 
>> > Been getting this in my email every morning for several days now. Rather
> 
>> > expected it to clear by now, but since it hasn't, and googling doesn't
> 
>> > seem to indicate anyone has noted the issue...
> 
>> >
> 
>> > * Error: The above package list contains packages which cannot be
> 
>> > * installed at the same time on the same system.
> 
>> >
> 
>> > (kde-apps/kde-l10n-16.04.3:5/5::gentoo, installed) pulled in by
> 
>> >
> 
>> > >=kde-apps/kde-l10n-16.04.3 required by (kde-apps/kde4-
> 
>> >
> 
>> > l10n-16.04.3:4/4::gentoo, installed)
> 
>> >
> 
>> > >=kde-apps/kde-l10n-16.04.3 required by (kde-apps/kde-apps-
> 
>> >
> 
>> > meta-16.04.3:5/5::gentoo, installed)
> 
>> >
> 
>> > (kde-apps/kdepim-l10n-15.12.3:5/5::gentoo, ebuild scheduled for merge)
> 
>> >
> 
>> > pulled in by
> 
>> >
> 
>> > >=kde-apps/kdepim-l10n-4.14.3 required by (kde-apps/kdepim-
> 
>> >
> 
>> > meta-4.14.11_pre20160211:4/4::gentoo, installed)
> 
>> >
> 
>> >
> 
>> > Now, it's not clear, if I'd like to continue using both KMail and non-
> 
>> > deprecated kde-apps, what to do here. Just hope that kdepim gets updated
> 
>> > to
> 
>> > qt5 soon? I'd pitch in, but I don't have the time.
> 
>>
> 
>> please post the portion of the output/mail that shows the blockers.
> 
>  
> 
> [nomerge       ] kde-apps/kde-apps-meta-16.04.3 
> 
> [nomerge       ] kde-apps/kdepim-meta-4.14.11_pre20160211 
> 
> [ebuild  NS    ]  kde-apps/kdepim-l10n-15.12.3 [4.14.3-r1] USE="-debug
> -handbook" L10N="-ar -bg -bs -ca -ca-valencia -cs -da -de -el -en-GB -eo
> -es -et -eu -fa -fi -fr -ga -gl -he -hi -hr -hu -ia -id -is -it -ja -kk
> -km -ko -lt -lv -mr -nb -nds -nl -nn -pa -pl -pt -pt-BR -ro -ru -sk -sl
> -sr -sv -tr -ug -uk -wa -zh-CN -zh-TW" 
> 
> [blocks b      ]   kde-apps/kdepim-l10n:4 ("kde-apps/kdepim-l10n:4" is
> blocking kde-apps/kdepim-l10n-15.12.3)
> 
> [uninstall     ]    kde-apps/kdepim-l10n-4.14.3-r1 
> 
> [blocks B      ] <kde-apps/kdepim-l10n-16.04.3:5
> ("<kde-apps/kdepim-l10n-16.04.3:5" is blocking kde-apps/kde-l10n-16.04.3)
> 
>  
> 
> -- 
> 
> :wq
> 

It wants to pull in kde-apps/kdepim-l10n-15.12.3

Any reason it refuses  kde-apps/kdepim-l10n-16.04.3 other than it's
unstable?


KMail is the lost child of KDE for many months now, I reckon this
situation is just going to get worse and worse. I know for myself my
mail problems ceased the day I dumped KMail4 for claws and/or thunderbird


-- 
Alan McKinnon
alan.mckinnon@gmail.com

Re: [gentoo-user] kde-apps/kde-l10n-16.04.3:5/5::gentoo conflicting with kde-apps/kdepim-l10n-15.12.3:5/5::gentoo

[Peter Humphrey]

On Monday 08 Aug 2016 22:45:09 Alan McKinnon wrote:

> KMail is the lost child of KDE for many months now, I reckon this
> situation is just going to get worse and worse. I know for myself my
> mail problems ceased the day I dumped KMail4 for claws and/or thunderbird

Not wishing to hijack the thread, but have you found a way to convert KMail 
archives to Claws format? Google shows me some dodgy-looking tricks, but is 
there a proper way?

-- 
Rgds
Peter

Re: [gentoo-user] kde-apps/kde-l10n-16.04.3:5/5::gentoo conflicting with kde-apps/kdepim-l10n-15.12.3:5/5::gentoo

[Neil Bothwick]

[-- Attachment #1: Type: text/plain, Size: 423 bytes --]

On Tue, 09 Aug 2016 08:52:29 +0100, Peter Humphrey wrote:

> Not wishing to hijack the thread, but have you found a way to convert
> KMail archives to Claws format? Google shows me some dodgy-looking
> tricks, but is there a proper way?

There's a conversion script on the Claws web site:

http://www.claws-mail.org/tools.php?section=downloads


-- 
Neil Bothwick

Top Oxymorons Number 10: Computer security

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

Re: [gentoo-user] kde-apps/kde-l10n-16.04.3:5/5::gentoo conflicting with kde-apps/kdepim-l10n-15.12.3:5/5::gentoo

[Peter Humphrey]

On Tuesday 09 Aug 2016 09:03:11 Neil Bothwick wrote:
> On Tue, 09 Aug 2016 08:52:29 +0100, Peter Humphrey wrote:
> > Not wishing to hijack the thread, but have you found a way to convert
> > KMail archives to Claws format? Google shows me some dodgy-looking
> > tricks, but is there a proper way?
> 
> There's a conversion script on the Claws web site:
> 
> http://www.claws-mail.org/tools.php?section=downloads

Thanks Neil. I'll definitely give that a go.

-- 
Rgds
Peter

Re: [gentoo-user] kde-apps/kde-l10n-16.04.3:5/5::gentoo conflicting with kde-apps/kdepim-l10n-15.12.3:5/5::gentoo

[Alan McKinnon]

On 09/08/2016 09:52, Peter Humphrey wrote:
> On Monday 08 Aug 2016 22:45:09 Alan McKinnon wrote:
> 
>> KMail is the lost child of KDE for many months now, I reckon this
>> situation is just going to get worse and worse. I know for myself my
>> mail problems ceased the day I dumped KMail4 for claws and/or thunderbird
> 
> Not wishing to hijack the thread, but have you found a way to convert KMail 
> archives to Claws format? Google shows me some dodgy-looking tricks, but is 
> there a proper way?
> 


install a local IMAP server and move your mails to it

-- 
Alan McKinnon
alan.mckinnon@gmail.com

Re: [gentoo-user] kde-apps/kde-l10n-16.04.3:5/5::gentoo conflicting with kde-apps/kdepim-l10n-15.12.3:5/5::gentoo

[Peter Humphrey]

On Tue, 9 Aug 2016 10:50:21 +0200
Alan McKinnon <alan.mckinnon@gmail.com> wrote:

> On 09/08/2016 09:52, Peter Humphrey wrote:
> > On Monday 08 Aug 2016 22:45:09 Alan McKinnon wrote:
> >   
> >> KMail is the lost child of KDE for many months now, I reckon this
> >> situation is just going to get worse and worse. I know for myself my
> >> mail problems ceased the day I dumped KMail4 for claws and/or
> >> thunderbird  
> > 
> > Not wishing to hijack the thread, but have you found a way to convert
> > KMail archives to Claws format? Google shows me some dodgy-looking
> > tricks, but is there a proper way?
> >   
> 
> 
> install a local IMAP server and move your mails to it
> 

Interesting idea. But the script Neil mentioned worked quickly, easily and
apparently flawlessly. I'm now writing this in Claws.

Re: [gentoo-user] kde-apps/kde-l10n-16.04.3:5/5::gentoo conflicting with kde-apps/kdepim-l10n-15.12.3:5/5::gentoo

[Michael Mol]

[-- Attachment #1: Type: text/plain, Size: 2374 bytes --]

On Monday, August 08, 2016 10:45:09 PM Alan McKinnon wrote:
> On 08/08/2016 19:20, Michael Mol wrote:
> > On Monday, August 08, 2016 06:52:15 PM Alan McKinnon wrote:
> >> On 08/08/2016 17:02, Michael Mol wrote:

[snip]

> > 
> > [nomerge       ] kde-apps/kde-apps-meta-16.04.3
> > 
> > [nomerge       ] kde-apps/kdepim-meta-4.14.11_pre20160211
> > 
> > [ebuild  NS    ]  kde-apps/kdepim-l10n-15.12.3 [4.14.3-r1] USE="-debug
> > -handbook" L10N="-ar -bg -bs -ca -ca-valencia -cs -da -de -el -en-GB -eo
> > -es -et -eu -fa -fi -fr -ga -gl -he -hi -hr -hu -ia -id -is -it -ja -kk
> > -km -ko -lt -lv -mr -nb -nds -nl -nn -pa -pl -pt -pt-BR -ro -ru -sk -sl
> > -sr -sv -tr -ug -uk -wa -zh-CN -zh-TW"
> > 
> > [blocks b      ]   kde-apps/kdepim-l10n:4 ("kde-apps/kdepim-l10n:4" is
> > blocking kde-apps/kdepim-l10n-15.12.3)
> > 
> > [uninstall     ]    kde-apps/kdepim-l10n-4.14.3-r1
> > 
> > [blocks B      ] <kde-apps/kdepim-l10n-16.04.3:5
> > ("<kde-apps/kdepim-l10n-16.04.3:5" is blocking kde-apps/kde-l10n-16.04.3)
> 
> It wants to pull in kde-apps/kdepim-l10n-15.12.3
> 
> Any reason it refuses  kde-apps/kdepim-l10n-16.04.3 other than it's
> unstable?

Good catch. I thought I had most of kde-apps unmasked for unstable to keep 
with the rolling. Missed that one.

> 
> 
> KMail is the lost child of KDE for many months now, I reckon this
> situation is just going to get worse and worse. I know for myself my
> mail problems ceased the day I dumped KMail4 for claws and/or thunderbird

That's really, really sad.

I used Thunderbird for years, but I eventually had to stop when it would, 
averaging once a month (though sometimes not for a couple months, sometimes a 
couple times a week) explode in memory consumption and drive the entire system 
unresponsively into swap.

I've tried claws from time to time due to other annoyances with Thunderbird, 
but I kept switching back. Not because I liked Tbird, but (IIRC) because of 
stability issues I had with claws.

Even with the bugs it has, Kontact and Akonadi has been the most reliable mail 
client I've used in the last year. When it gives me problems, I know why, and 
I can address it. (Running a heavily tuned MySQLd instance behind Akonadi, for 
example...)

I wish someone would pay me to fix this stuff; I'd be able to spend the time on 
it.

-- 
:wq

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 473 bytes --]

Re: [gentoo-user] kde-apps/kde-l10n-16.04.3:5/5::gentoo conflicting with kde-apps/kdepim-l10n-15.12.3:5/5::gentoo

[J. Roeleveld]

On August 9, 2016 4:13:31 PM GMT+02:00, james <garftd@verizon.net> wrote:
>On 08/09/2016 07:42 AM, Michael Mol wrote:
>> On Monday, August 08, 2016 10:45:09 PM Alan McKinnon wrote:
>>> On 08/08/2016 19:20, Michael Mol wrote:
>>>> On Monday, August 08, 2016 06:52:15 PM Alan McKinnon wrote:
>>>>> On 08/08/2016 17:02, Michael Mol wrote:
>
>>>> snip <<<
>>> KMail is the lost child of KDE for many months now, I reckon this
>>> situation is just going to get worse and worse. I know for myself my
>>> mail problems ceased the day I dumped KMail4 for claws and/or
>thunderbird
>>
>> That's really, really sad.
>>
>> I used Thunderbird for years, but I eventually had to stop when it
>would,
>> averaging once a month (though sometimes not for a couple months,
>sometimes a
>> couple times a week) explode in memory consumption and drive the
>entire system
>> unresponsively into swap.
>>
>> I've tried claws from time to time due to other annoyances with
>Thunderbird,
>> but I kept switching back. Not because I liked Tbird, but (IIRC)
>because of
>> stability issues I had with claws.
>>
>> Even with the bugs it has, Kontact and Akonadi has been the most
>reliable mail
>> client I've used in the last year. When it gives me problems, I know
>why, and
>> I can address it. (Running a heavily tuned MySQLd instance behind
>Akonadi, for
>> example...)
>>
>> I wish someone would pay me to fix this stuff; I'd be able to spend
>the time on
>> it.
>
>Perhaps an experiment. Locate some folks that know about how to promote
>
>'crowd funding'. The propose a project like this, targeted at business 
>and user, to all pitch in. In fact, quite a few beloved open source 
>projects could benefit, if the idea of crowd funding took hold
>on open source soft. Perhaps one of the foundations deeply involved in 
>the open source movement would get behind the idea?
>
>KDE is very popular, so the concept or something similar might just
>have 
>legs, even if it only funds a series of grad-students or young 
>programmers to maintain good FOSS projects?
>
>AS a side note, I put 32G of ram on my system and still at times it is 
>laggy with little processor load and htop shows little <30% ram usage.
>What tools do you use to track down mem. management issues?
>
>Any specific kernel tweaks?
>
>
>hth,
>James
>
>
>
>hth,
>James

Try iotop....

-- 
Sent from my Android device with K-9 Mail. Please excuse my brevity.

Re: [gentoo-user] kde-apps/kde-l10n-16.04.3:5/5::gentoo conflicting with kde-apps/kdepim-l10n-15.12.3:5/5::gentoo

[james]

On 08/09/2016 07:42 AM, Michael Mol wrote:
> On Monday, August 08, 2016 10:45:09 PM Alan McKinnon wrote:
>> On 08/08/2016 19:20, Michael Mol wrote:
>>> On Monday, August 08, 2016 06:52:15 PM Alan McKinnon wrote:
>>>> On 08/08/2016 17:02, Michael Mol wrote:

>>> snip <<<
>> KMail is the lost child of KDE for many months now, I reckon this
>> situation is just going to get worse and worse. I know for myself my
>> mail problems ceased the day I dumped KMail4 for claws and/or thunderbird
>
> That's really, really sad.
>
> I used Thunderbird for years, but I eventually had to stop when it would,
> averaging once a month (though sometimes not for a couple months, sometimes a
> couple times a week) explode in memory consumption and drive the entire system
> unresponsively into swap.
>
> I've tried claws from time to time due to other annoyances with Thunderbird,
> but I kept switching back. Not because I liked Tbird, but (IIRC) because of
> stability issues I had with claws.
>
> Even with the bugs it has, Kontact and Akonadi has been the most reliable mail
> client I've used in the last year. When it gives me problems, I know why, and
> I can address it. (Running a heavily tuned MySQLd instance behind Akonadi, for
> example...)
>
> I wish someone would pay me to fix this stuff; I'd be able to spend the time on
> it.

Perhaps an experiment. Locate some folks that know about how to promote 
'crowd funding'. The propose a project like this, targeted at business 
and user, to all pitch in. In fact, quite a few beloved open source 
projects could benefit, if the idea of crowd funding took hold
on open source soft. Perhaps one of the foundations deeply involved in 
the open source movement would get behind the idea?

KDE is very popular, so the concept or something similar might just have 
legs, even if it only funds a series of grad-students or young 
programmers to maintain good FOSS projects?

AS a side note, I put 32G of ram on my system and still at times it is 
laggy with little processor load and htop shows little <30% ram usage.
What tools do you use to track down mem. management issues?

Any specific kernel tweaks?


hth,
James



hth,
James

Re: [gentoo-user] kde-apps/kde-l10n-16.04.3:5/5::gentoo conflicting with kde-apps/kdepim-l10n-15.12.3:5/5::gentoo

[Michael Mol]

[-- Attachment #1.1: Type: text/plain, Size: 24669 bytes --]

=2D-nextPart2985939.38cYH2hHAJ
Content-Transfer-Encoding: 7Bit
Content-Type: text/plain; charset="us-ascii"

On Tuesday, August 09, 2016 09:13:31 AM james wrote:
> On 08/09/2016 07:42 AM, Michael Mol wrote:
> > On Monday, August 08, 2016 10:45:09 PM Alan McKinnon wrote:
> >> On 08/08/2016 19:20, Michael Mol wrote:
> >>> On Monday, August 08, 2016 06:52:15 PM Alan McKinnon wrote:
> >>>> On 08/08/2016 17:02, Michael Mol wrote:
> >>> snip <<<
> >> 
> >> KMail is the lost child of KDE for many months now, I reckon this
> >> situation is just going to get worse and worse. I know for myself my
> >> mail problems ceased the day I dumped KMail4 for claws and/or thunderbird
> > 
> > That's really, really sad.
> > 
> > I used Thunderbird for years, but I eventually had to stop when it would,
> > averaging once a month (though sometimes not for a couple months,
> > sometimes a couple times a week) explode in memory consumption and drive
> > the entire system unresponsively into swap.
> > 
> > I've tried claws from time to time due to other annoyances with
> > Thunderbird, but I kept switching back. Not because I liked Tbird, but
> > (IIRC) because of stability issues I had with claws.
> > 
> > Even with the bugs it has, Kontact and Akonadi has been the most reliable
> > mail client I've used in the last year. When it gives me problems, I know
> > why, and I can address it. (Running a heavily tuned MySQLd instance
> > behind Akonadi, for example...)
> > 
> > I wish someone would pay me to fix this stuff; I'd be able to spend the
> > time on it.
> 
> Perhaps an experiment. Locate some folks that know about how to promote
> 'crowd funding'. The propose a project like this, targeted at business
> and user, to all pitch in. In fact, quite a few beloved open source
> projects could benefit, if the idea of crowd funding took hold
> on open source soft. Perhaps one of the foundations deeply involved in
> the open source movement would get behind the idea?
> 
> KDE is very popular, so the concept or something similar might just have
> legs, even if it only funds a series of grad-students or young
> programmers to maintain good FOSS projects?

A wonderful thought. I rather expect KDE is already doing this, but if not, they ought to. (I'm 
sure someone who commits code to KDE reads this list...)

Certainly wouldn't cover someone like me who has a family to support, but still.

> 
> AS a side note, I put 32G of ram on my system and still at times it is
> laggy with little processor load and htop shows little <30% ram usage.
> What tools do you use to track down mem. management issues?

I use Zabbix extensively at work, and have the Zabbix agent on my workstation reporting 
back various supported metrics. There's a great deal you can use (and--my favorite--
abuse) Zabbix for, especially once you understand how it thinks.

> 
> Any specific kernel tweaks?

Most of my tweaks for KDE revolved around tuning mysqld itself. But for sysctls improving 
workstation responsiveness as it relates to memory interactions with I/O, these are my go-
tos:

vm.*dirty*_background_bytes = 1048576
vm.*dirty*_bytes = 10485760
vm.*swap*piness = 0

vm.dirty_background_bytes ensures that any data (i.e. from mmap or fwrite, not from 
swapping) waiting to be written to disk *starts* getting written to disk once you've got at 
least the configured amount (1MB) of data waiting. (If you've got a disk controller with 
battery-backed or flash-backed write cache, you might consider increasing this to some 
significant fraction of your write cache. I.e. if you've got a 1GB FBWC with 768MB of that 
dedicated to write cache, you might set this to 512MB or so. Depending on your workload. 
I/O tuning is for those of us who enjoy the dark arts.)

vm.dirty_bytes says that once you've got the configured amount (10MB) of data waiting to 
be disk, then no more asynchronous I/O is permitted until you have no more data waiting; 
all outstanding writes must be finished first. (My rule of thumb is to have this between 2-10 
times the value of vm.dirty_background_bytes. Though I'm really trying to avoid it being 
high enough that it could take more than 50ms to transfer to disk; that way, any stalls that 
do happen are almost imperceptible.)

You want vm.dirty_background_bytes to be high enough that your hardware doesn't spend 
its time powered on if it doesn't have to be, and so that your hardware can transfer data in 
large, efficient, streamable chunks.

You want vm.dirty_bytes enough higher than your first number so that your hardware has 
enough time to spin up and transfer data before you put the hammer down and say, "all 
right, nobody else gets to queue writes until all the waiting data has reached disk."

You want vm.dirty_bytes *low* enough that when you *do* have to put that hammer down, 
it doesn't interfere with your perceptions of a responsive system. (And in a server context, 
you want it low enough that things can't time out--or be pushed into timing out--waiting for 
it. Call your user attention a matter of timing out expecting things to respond to you, and 
the same principle applies...)

Now, vm.swappiness? That's a weighting factor for how quickly the kernel should try 
moving memory to swap to be able to speedily respond to new allocations. Me, I prefer 
the kernel to not preemptively move lesser-used data to swap, because that's going to be 
a few hundred megabytes worth of data all associated with one application, and it'll be a 
real drag when I switch back to the application I haven't used for half an hour. So I set 
vm.swappiness to 0, to tell the kernel to only move data to swap if it has no other 
alternative while trying to satisfy a new memory allocation request.

=2D- 
:wq

=2D-nextPart2985939.38cYH2hHAJ
Content-Transfer-Encoding: quoted-printable
Content-Type: text/html; charset="us-ascii"

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0//EN" "http://www.w3.org/TR/=
REC-html40/strict.dtd">
<html><head><meta name=3D"qrichtext" content=3D"1" /><style type=3D"tex=
t/css">
p, li { white-space: pre-wrap; }
</style></head><body style=3D" font-family:'Sans Serif'; font-size:9pt;=
 font-weight:400; font-style:normal;">
<p style=3D" margin-top:0px; margin-bottom:0px; margin-left:0px; margin=
=2Dright:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">On =
Tuesday, August 09, 2016 09:13:31 AM james wrote:</p>
<p style=3D" margin-top:0px; margin-bottom:0px; margin-left:0px; margin=
=2Dright:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">&gt=
; On 08/09/2016 07:42 AM, Michael Mol wrote:</p>
<p style=3D" margin-top:0px; margin-bottom:0px; margin-left:0px; margin=
=2Dright:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">&gt=
; &gt; On Monday, August 08, 2016 10:45:09 PM Alan McKinnon wrote:</p>
<p style=3D" margin-top:0px; margin-bottom:0px; margin-left:0px; margin=
=2Dright:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">&gt=
; &gt;&gt; On 08/08/2016 19:20, Michael Mol wrote:</p>
<p style=3D" margin-top:0px; margin-bottom:0px; margin-left:0px; margin=
=2Dright:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">&gt=
; &gt;&gt;&gt; On Monday, August 08, 2016 06:52:15 PM Alan McKinnon wro=
te:</p>
<p style=3D" margin-top:0px; margin-bottom:0px; margin-left:0px; margin=
=2Dright:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">&gt=
; &gt;&gt;&gt;&gt; On 08/08/2016 17:02, Michael Mol wrote:</p>
<p style=3D" margin-top:0px; margin-bottom:0px; margin-left:0px; margin=
=2Dright:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">&gt=
; &gt;&gt;&gt; snip &lt;&lt;&lt;</p>
<p style=3D" margin-top:0px; margin-bottom:0px; margin-left:0px; margin=
=2Dright:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">&gt=
; &gt;&gt; </p>
<p style=3D" margin-top:0px; margin-bottom:0px; margin-left:0px; margin=
=2Dright:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">&gt=
; &gt;&gt; KMail is the lost child of KDE for many months now, I reckon=
 this</p>
<p style=3D" margin-top:0px; margin-bottom:0px; margin-left:0px; margin=
=2Dright:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">&gt=
; &gt;&gt; situation is just going to get worse and worse. I know for m=
yself my</p>
<p style=3D" margin-top:0px; margin-bottom:0px; margin-left:0px; margin=
=2Dright:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">&gt=
; &gt;&gt; mail problems ceased the day I dumped KMail4 for claws and/o=
r thunderbird</p>
<p style=3D" margin-top:0px; margin-bottom:0px; margin-left:0px; margin=
=2Dright:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">&gt=
; &gt; </p>
<p style=3D" margin-top:0px; margin-bottom:0px; margin-left:0px; margin=
=2Dright:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">&gt=
; &gt; That's really, really sad.</p>
<p style=3D" margin-top:0px; margin-bottom:0px; margin-left:0px; margin=
=2Dright:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">&gt=
; &gt; </p>
<p style=3D" margin-top:0px; margin-bottom:0px; margin-left:0px; margin=
=2Dright:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">&gt=
; &gt; I used Thunderbird for years, but I eventually had to stop when =
it would,</p>
<p style=3D" margin-top:0px; margin-bottom:0px; margin-left:0px; margin=
=2Dright:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">&gt=
; &gt; averaging once a month (though sometimes not for a couple months=
,</p>
<p style=3D" margin-top:0px; margin-bottom:0px; margin-left:0px; margin=
=2Dright:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">&gt=
; &gt; sometimes a couple times a week) explode in memory consumption a=
nd drive</p>
<p style=3D" margin-top:0px; margin-bottom:0px; margin-left:0px; margin=
=2Dright:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">&gt=
; &gt; the entire system unresponsively into swap.</p>
<p style=3D" margin-top:0px; margin-bottom:0px; margin-left:0px; margin=
=2Dright:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">&gt=
; &gt; </p>
<p style=3D" margin-top:0px; margin-bottom:0px; margin-left:0px; margin=
=2Dright:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">&gt=
; &gt; I've tried claws from time to time due to other annoyances with<=
/p>
<p style=3D" margin-top:0px; margin-bottom:0px; margin-left:0px; margin=
=2Dright:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">&gt=
; &gt; Thunderbird, but I kept switching back. Not because I liked Tbir=
d, but</p>
<p style=3D" margin-top:0px; margin-bottom:0px; margin-left:0px; margin=
=2Dright:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">&gt=
; &gt; (IIRC) because of stability issues I had with claws.</p>
<p style=3D" margin-top:0px; margin-bottom:0px; margin-left:0px; margin=
=2Dright:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">&gt=
; &gt; </p>
<p style=3D" margin-top:0px; margin-bottom:0px; margin-left:0px; margin=
=2Dright:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">&gt=
; &gt; Even with the bugs it has, Kontact and Akonadi has been the most=
 reliable</p>
<p style=3D" margin-top:0px; margin-bottom:0px; margin-left:0px; margin=
=2Dright:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">&gt=
; &gt; mail client I've used in the last year. When it gives me problem=
s, I know</p>
<p style=3D" margin-top:0px; margin-bottom:0px; margin-left:0px; margin=
=2Dright:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">&gt=
; &gt; why, and I can address it. (Running a heavily tuned MySQLd insta=
nce</p>
<p style=3D" margin-top:0px; margin-bottom:0px; margin-left:0px; margin=
=2Dright:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">&gt=
; &gt; behind Akonadi, for example...)</p>
<p style=3D" margin-top:0px; margin-bottom:0px; margin-left:0px; margin=
=2Dright:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">&gt=
; &gt; </p>
<p style=3D" margin-top:0px; margin-bottom:0px; margin-left:0px; margin=
=2Dright:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">&gt=
; &gt; I wish someone would pay me to fix this stuff; I'd be able to sp=
end the</p>
<p style=3D" margin-top:0px; margin-bottom:0px; margin-left:0px; margin=
=2Dright:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">&gt=
; &gt; time on it.</p>
<p style=3D" margin-top:0px; margin-bottom:0px; margin-left:0px; margin=
=2Dright:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">&gt=
; </p>
<p style=3D" margin-top:0px; margin-bottom:0px; margin-left:0px; margin=
=2Dright:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">&gt=
; Perhaps an experiment. Locate some folks that know about how to promo=
te</p>
<p style=3D" margin-top:0px; margin-bottom:0px; margin-left:0px; margin=
=2Dright:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">&gt=
; 'crowd funding'. The propose a project like this, targeted at busines=
s</p>
<p style=3D" margin-top:0px; margin-bottom:0px; margin-left:0px; margin=
=2Dright:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">&gt=
; and user, to all pitch in. In fact, quite a few beloved open source</=
p>
<p style=3D" margin-top:0px; margin-bottom:0px; margin-left:0px; margin=
=2Dright:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">&gt=
; projects could benefit, if the idea of crowd funding took hold</p>
<p style=3D" margin-top:0px; margin-bottom:0px; margin-left:0px; margin=
=2Dright:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">&gt=
; on open source soft. Perhaps one of the foundations deeply involved i=
n</p>
<p style=3D" margin-top:0px; margin-bottom:0px; margin-left:0px; margin=
=2Dright:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">&gt=
; the open source movement would get behind the idea?</p>
<p style=3D" margin-top:0px; margin-bottom:0px; margin-left:0px; margin=
=2Dright:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">&gt=
; </p>
<p style=3D" margin-top:0px; margin-bottom:0px; margin-left:0px; margin=
=2Dright:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">&gt=
; KDE is very popular, so the concept or something similar might just h=
ave</p>
<p style=3D" margin-top:0px; margin-bottom:0px; margin-left:0px; margin=
=2Dright:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">&gt=
; legs, even if it only funds a series of grad-students or young</p>
<p style=3D" margin-top:0px; margin-bottom:0px; margin-left:0px; margin=
=2Dright:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">&gt=
; programmers to maintain good FOSS projects?</p>
<p style=3D"-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px=
; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0p=
x; ">&nbsp;</p>
<p style=3D" margin-top:0px; margin-bottom:0px; margin-left:0px; margin=
=2Dright:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">A w=
onderful thought. I rather expect KDE is already doing this, but if not=
, they ought to. (I'm sure someone who commits code to KDE reads this l=
ist...)</p>
<p style=3D"-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px=
; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0p=
x; ">&nbsp;</p>
<p style=3D" margin-top:0px; margin-bottom:0px; margin-left:0px; margin=
=2Dright:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">Cer=
tainly wouldn't cover someone like me who has a family to support, but =
still.</p>
<p style=3D"-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px=
; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0p=
x; ">&nbsp;</p>
<p style=3D" margin-top:0px; margin-bottom:0px; margin-left:0px; margin=
=2Dright:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">&gt=
; </p>
<p style=3D" margin-top:0px; margin-bottom:0px; margin-left:0px; margin=
=2Dright:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">&gt=
; AS a side note, I put 32G of ram on my system and still at times it i=
s</p>
<p style=3D" margin-top:0px; margin-bottom:0px; margin-left:0px; margin=
=2Dright:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">&gt=
; laggy with little processor load and htop shows little &lt;30% ram us=
age.</p>
<p style=3D" margin-top:0px; margin-bottom:0px; margin-left:0px; margin=
=2Dright:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">&gt=
; What tools do you use to track down mem. management issues?</p>
<p style=3D"-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px=
; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0p=
x; ">&nbsp;</p>
<p style=3D" margin-top:0px; margin-bottom:0px; margin-left:0px; margin=
=2Dright:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">I u=
se Zabbix extensively at work, and have the Zabbix agent on my workstat=
ion reporting back various supported metrics. There's a great deal you =
can use (and--my favorite--abuse) Zabbix for, especially once you under=
stand how it thinks.</p>
<p style=3D"-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px=
; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0p=
x; ">&nbsp;</p>
<p style=3D" margin-top:0px; margin-bottom:0px; margin-left:0px; margin=
=2Dright:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">&gt=
; </p>
<p style=3D" margin-top:0px; margin-bottom:0px; margin-left:0px; margin=
=2Dright:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">&gt=
; Any specific kernel tweaks?</p>
<p style=3D"-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px=
; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0p=
x; ">&nbsp;</p>
<p style=3D" margin-top:0px; margin-bottom:0px; margin-left:0px; margin=
=2Dright:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">Mos=
t of my tweaks for KDE revolved around tuning mysqld itself. But for sy=
sctls improving workstation responsiveness as it relates to memory inte=
ractions with I/O, these are my go-tos:</p>
<p style=3D"-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px=
; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0p=
x; ">&nbsp;</p>
<p style=3D" margin-top:0px; margin-bottom:0px; margin-left:0px; margin=
=2Dright:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><sp=
an style=3D" font-family:'monospace'; color:#000000; background-color:#=
ffffff;">vm.</span><span style=3D" font-family:'monospace'; font-weight=
:600; color:#ff5454; background-color:#ffffff;">dirty</span><span style=
=3D" font-family:'monospace'; color:#000000; background-color:#ffffff;"=
>_background_bytes =3D 1048576</span><span style=3D" font-family:'monos=
pace';"><br /></span><span style=3D" font-family:'monospace'; color:#00=
0000; background-color:#ffffff;">vm.</span><span style=3D" font-family:=
'monospace'; font-weight:600; color:#ff5454; background-color:#ffffff;"=
>dirty</span><span style=3D" font-family:'monospace'; color:#000000; ba=
ckground-color:#ffffff;">_bytes =3D 10485760</span><span style=3D" font=
=2Dfamily:'monospace';"><br /></span><span style=3D" font-family:'monospa=
ce'; color:#000000; background-color:#ffffff;">vm.</span><span style=3D=
" font-family:'monospace'; font-weight:600; color:#ff5454; background-c=
olor:#ffffff;">swap</span><span style=3D" font-family:'monospace'; colo=
r:#000000; background-color:#ffffff;">piness =3D 0</span><span style=3D=
" font-family:'monospace';"><br /></span></p>
<p style=3D" margin-top:0px; margin-bottom:0px; margin-left:0px; margin=
=2Dright:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><sp=
an style=3D" font-family:'monospace';">vm.dirty_background_bytes ensure=
s that any data (i.e. from mmap or fwrite, not from swapping) waiting t=
o be written to disk *starts* getting written to disk once you've got a=
t least the configured amount (1MB) of data waiting. (If you've got a d=
isk controller with battery-backed or flash-backed write cache, you mig=
ht consider increasing this to some significant fraction of your write =
cache. I.e. if you've got a 1GB FBWC with 768MB of that dedicated to wr=
ite cache, you might set this to 512MB or so. Depending on your workloa=
d. I/O tuning is for those of us who enjoy the dark arts.)</span></p>
<p style=3D"-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px=
; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0p=
x; ">&nbsp;</p>
<p style=3D" margin-top:0px; margin-bottom:0px; margin-left:0px; margin=
=2Dright:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><sp=
an style=3D" font-family:'monospace';">vm.dirty_bytes says that once yo=
u've got the configured amount (10MB) of data waiting to be disk, then =
no more asynchronous I/O is permitted until you have no more data waiti=
ng; all outstanding writes must be finished first. (My rule of thumb is=
 to have this between 2-10 times the value of vm.dirty_background_bytes=
. Though I'm really trying to avoid it being high enough that it could =
take more than 50ms to transfer to disk; that way, any stalls that do h=
appen are almost imperceptible.)</span></p>
<p style=3D"-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px=
; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0p=
x; ">&nbsp;</p>
<p style=3D" margin-top:0px; margin-bottom:0px; margin-left:0px; margin=
=2Dright:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><sp=
an style=3D" font-family:'monospace';">You want vm.dirty_background_byt=
es to be high enough that your hardware doesn't spend its time powered =
on if it doesn't have to be, and so that your hardware can transfer dat=
a in large, efficient, streamable chunks.</span></p>
<p style=3D"-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px=
; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0p=
x; ">&nbsp;</p>
<p style=3D" margin-top:0px; margin-bottom:0px; margin-left:0px; margin=
=2Dright:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><sp=
an style=3D" font-family:'monospace';">You want vm.dirty_bytes enough h=
igher than your first number so that your hardware has enough time to s=
pin up and transfer data before you put the hammer down and say, &quot;=
all right, nobody else gets to queue writes until all the waiting data =
has reached disk.&quot;</span></p>
<p style=3D"-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px=
; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0p=
x; ">&nbsp;</p>
<p style=3D" margin-top:0px; margin-bottom:0px; margin-left:0px; margin=
=2Dright:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><sp=
an style=3D" font-family:'monospace';">You want vm.dirty_bytes *low* en=
ough that when you *do* have to put that hammer down, it doesn't interf=
ere with your perceptions of a responsive system. (And in a server cont=
ext, you want it low enough that things can't time out--or be pushed in=
to timing out--waiting for it. Call your user attention a matter of tim=
ing out expecting things to respond to you, and the same principle appl=
ies...)</span></p>
<p style=3D"-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px=
; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0p=
x; ">&nbsp;</p>
<p style=3D" margin-top:0px; margin-bottom:0px; margin-left:0px; margin=
=2Dright:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><sp=
an style=3D" font-family:'monospace';">Now, vm.swappiness? That's a wei=
ghting factor for how quickly the kernel should try moving memory to sw=
ap to be able to speedily respond to new allocations. Me, I prefer the =
kernel to not preemptively move lesser-used data to swap, because that'=
s going to be a few hundred megabytes worth of data all associated with=
 one application, and it'll be a real drag when I switch back to the ap=
plication I haven't used for half an hour. So I set vm.swappiness to 0,=
 to tell the kernel to only move data to swap if it has no other altern=
ative while trying to satisfy a new memory allocation request.</span></=
p>
<p style=3D"-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px=
; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0p=
x; ">&nbsp;</p>
<p style=3D" margin-top:0px; margin-bottom:0px; margin-left:0px; margin=
=2Dright:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">-- =
</p>
<p style=3D" margin-top:0px; margin-bottom:0px; margin-left:0px; margin=
=2Dright:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">:wq=
</p></body></html>
=2D-nextPart2985939.38cYH2hHAJ--
This is a multi-part message in MIME format.

[-- Attachment #1.2: Type: text/plain, Size: 5630 bytes --]

On Tuesday, August 09, 2016 09:13:31 AM james wrote:
> On 08/09/2016 07:42 AM, Michael Mol wrote:
> > On Monday, August 08, 2016 10:45:09 PM Alan McKinnon wrote:
> >> On 08/08/2016 19:20, Michael Mol wrote:
> >>> On Monday, August 08, 2016 06:52:15 PM Alan McKinnon wrote:
> >>>> On 08/08/2016 17:02, Michael Mol wrote:
> >>> snip <<<
> >> 
> >> KMail is the lost child of KDE for many months now, I reckon this
> >> situation is just going to get worse and worse. I know for myself my
> >> mail problems ceased the day I dumped KMail4 for claws and/or thunderbird
> > 
> > That's really, really sad.
> > 
> > I used Thunderbird for years, but I eventually had to stop when it would,
> > averaging once a month (though sometimes not for a couple months,
> > sometimes a couple times a week) explode in memory consumption and drive
> > the entire system unresponsively into swap.
> > 
> > I've tried claws from time to time due to other annoyances with
> > Thunderbird, but I kept switching back. Not because I liked Tbird, but
> > (IIRC) because of stability issues I had with claws.
> > 
> > Even with the bugs it has, Kontact and Akonadi has been the most reliable
> > mail client I've used in the last year. When it gives me problems, I know
> > why, and I can address it. (Running a heavily tuned MySQLd instance
> > behind Akonadi, for example...)
> > 
> > I wish someone would pay me to fix this stuff; I'd be able to spend the
> > time on it.
> 
> Perhaps an experiment. Locate some folks that know about how to promote
> 'crowd funding'. The propose a project like this, targeted at business
> and user, to all pitch in. In fact, quite a few beloved open source
> projects could benefit, if the idea of crowd funding took hold
> on open source soft. Perhaps one of the foundations deeply involved in
> the open source movement would get behind the idea?
> 
> KDE is very popular, so the concept or something similar might just have
> legs, even if it only funds a series of grad-students or young
> programmers to maintain good FOSS projects?

A wonderful thought. I rather expect KDE is already doing this, but if not, they ought to. (I'm 
sure someone who commits code to KDE reads this list...)

Certainly wouldn't cover someone like me who has a family to support, but still.

> 
> AS a side note, I put 32G of ram on my system and still at times it is
> laggy with little processor load and htop shows little <30% ram usage.
> What tools do you use to track down mem. management issues?

I use Zabbix extensively at work, and have the Zabbix agent on my workstation reporting 
back various supported metrics. There's a great deal you can use (and--my favorite--
abuse) Zabbix for, especially once you understand how it thinks.

> 
> Any specific kernel tweaks?

Most of my tweaks for KDE revolved around tuning mysqld itself. But for sysctls improving 
workstation responsiveness as it relates to memory interactions with I/O, these are my go-
tos:

vm.*dirty*_background_bytes = 1048576
vm.*dirty*_bytes = 10485760
vm.*swap*piness = 0

vm.dirty_background_bytes ensures that any data (i.e. from mmap or fwrite, not from 
swapping) waiting to be written to disk *starts* getting written to disk once you've got at 
least the configured amount (1MB) of data waiting. (If you've got a disk controller with 
battery-backed or flash-backed write cache, you might consider increasing this to some 
significant fraction of your write cache. I.e. if you've got a 1GB FBWC with 768MB of that 
dedicated to write cache, you might set this to 512MB or so. Depending on your workload. 
I/O tuning is for those of us who enjoy the dark arts.)

vm.dirty_bytes says that once you've got the configured amount (10MB) of data waiting to 
be disk, then no more asynchronous I/O is permitted until you have no more data waiting; 
all outstanding writes must be finished first. (My rule of thumb is to have this between 2-10 
times the value of vm.dirty_background_bytes. Though I'm really trying to avoid it being 
high enough that it could take more than 50ms to transfer to disk; that way, any stalls that 
do happen are almost imperceptible.)

You want vm.dirty_background_bytes to be high enough that your hardware doesn't spend 
its time powered on if it doesn't have to be, and so that your hardware can transfer data in 
large, efficient, streamable chunks.

You want vm.dirty_bytes enough higher than your first number so that your hardware has 
enough time to spin up and transfer data before you put the hammer down and say, "all 
right, nobody else gets to queue writes until all the waiting data has reached disk."

You want vm.dirty_bytes *low* enough that when you *do* have to put that hammer down, 
it doesn't interfere with your perceptions of a responsive system. (And in a server context, 
you want it low enough that things can't time out--or be pushed into timing out--waiting for 
it. Call your user attention a matter of timing out expecting things to respond to you, and 
the same principle applies...)

Now, vm.swappiness? That's a weighting factor for how quickly the kernel should try 
moving memory to swap to be able to speedily respond to new allocations. Me, I prefer 
the kernel to not preemptively move lesser-used data to swap, because that's going to be 
a few hundred megabytes worth of data all associated with one application, and it'll be a 
real drag when I switch back to the application I haven't used for half an hour. So I set 
vm.swappiness to 0, to tell the kernel to only move data to swap if it has no other 
alternative while trying to satisfy a new memory allocation request.

-- 
:wq

[-- Attachment #1.3: Type: text/html, Size: 18028 bytes --]

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 473 bytes --]

Re: [gentoo-user] kde-apps/kde-l10n-16.04.3:5/5::gentoo conflicting with kde-apps/kdepim-l10n-15.12.3:5/5::gentoo

[Daniel Frey]

On 08/09/2016 05:42 AM, Michael Mol wrote:
> I used Thunderbird for years, but I eventually had to stop when it would, 
> averaging once a month (though sometimes not for a couple months, sometimes a 
> couple times a week) explode in memory consumption and drive the entire system 
> unresponsively into swap.
> 

I've been using thunderbird exclusively on my PC and haven't seen this
particular issue. When was the last time you tried it?

I've probably got between 8k and 10k messages in it right now. Memory
consumption is 3.3% of 8GB and I do see every 30 seconds or so
thunderbird wakes up and does something for a few seconds, using 8-10%
of CPU while it does. But I've never noticed it actually doing anything
(like slowing the system to a crawl.)

Dan

Re: [gentoo-user] kde-apps/kde-l10n-16.04.3:5/5::gentoo conflicting with kde-apps/kdepim-l10n-15.12.3:5/5::gentoo

[james]

On 08/09/2016 09:06 AM, J. Roeleveld wrote:
> On August 9, 2016 4:13:31 PM GMT+02:00, james <garftd@verizon.net> wrote:

>> AS a side note, I put 32G of ram on my system and still at times it is
>> laggy with little processor load and htop shows little <30% ram usage.
>> What tools do you use to track down mem. management issues?
>>
>> Any specific kernel tweaks?

> Try iotop....

OK, so the gentoo wikis says the only kernel modes for iotop are


General setup
-> CPU/Task time and stats accounting
    [*] Enable extended accounting over taskstats
    [*] Enable per-task storage I/O accounting


Do you add any others? Do you build a specific kernel for these sorts of 
low level account and debug codes to work, as reading about, some
can cause noticable performance degredations. So do you have a special 
kernel to track these problem and then return to a production kernel,
or leave them in all the time?

curiously,
James

Re: [gentoo-user] kde-apps/kde-l10n-16.04.3:5/5::gentoo conflicting with kde-apps/kdepim-l10n-15.12.3:5/5::gentoo

[james]

On 08/09/2016 09:17 AM, Michael Mol wrote:
> On Tuesday, August 09, 2016 09:13:31 AM james wrote:
>
>> On 08/09/2016 07:42 AM, Michael Mol wrote:
>
>> > On Monday, August 08, 2016 10:45:09 PM Alan McKinnon wrote:
>
>> >> On 08/08/2016 19:20, Michael Mol wrote:
>
>> >>> On Monday, August 08, 2016 06:52:15 PM Alan McKinnon wrote:
>
>> >>>> On 08/08/2016 17:02, Michael Mol wrote:
>
>> >>> snip <<<
>
>> >>
>
>> >> KMail is the lost child of KDE for many months now, I reckon this
>
>> >> situation is just going to get worse and worse. I know for myself my
>
>> >> mail problems ceased the day I dumped KMail4 for claws and/or
> thunderbird
>
>> >
>
>> > That's really, really sad.
>
>> >
>
>> > I used Thunderbird for years, but I eventually had to stop when it
> would,
>
>> > averaging once a month (though sometimes not for a couple months,
>
>> > sometimes a couple times a week) explode in memory consumption and drive
>
>> > the entire system unresponsively into swap.
>
>> >
>
>> > I've tried claws from time to time due to other annoyances with
>
>> > Thunderbird, but I kept switching back. Not because I liked Tbird, but
>
>> > (IIRC) because of stability issues I had with claws.
>
>> >
>
>> > Even with the bugs it has, Kontact and Akonadi has been the most
> reliable
>
>> > mail client I've used in the last year. When it gives me problems, I
> know
>
>> > why, and I can address it. (Running a heavily tuned MySQLd instance
>
>> > behind Akonadi, for example...)
>
>> >
>
>> > I wish someone would pay me to fix this stuff; I'd be able to spend the
>
>> > time on it.
>
>>
>
>> Perhaps an experiment. Locate some folks that know about how to promote
>
>> 'crowd funding'. The propose a project like this, targeted at business
>
>> and user, to all pitch in. In fact, quite a few beloved open source
>
>> projects could benefit, if the idea of crowd funding took hold
>
>> on open source soft. Perhaps one of the foundations deeply involved in
>
>> the open source movement would get behind the idea?
>
>>
>
>> KDE is very popular, so the concept or something similar might just have
>
>> legs, even if it only funds a series of grad-students or young
>
>> programmers to maintain good FOSS projects?
>
>
>
> A wonderful thought. I rather expect KDE is already doing this, but if
> not, they ought to. (I'm sure someone who commits code to KDE reads this
> list...)
>
>
>
> Certainly wouldn't cover someone like me who has a family to support,
> but still.
>
>
>
>>
>
>> AS a side note, I put 32G of ram on my system and still at times it is
>
>> laggy with little processor load and htop shows little <30% ram usage.
>
>> What tools do you use to track down mem. management issues?
>
>
>
> I use Zabbix extensively at work, and have the Zabbix agent on my
> workstation reporting back various supported metrics. There's a great
> deal you can use (and--my favorite--abuse) Zabbix for, especially once
> you understand how it thinks.

Congradualtions! Of the net-analyzer crowd, you've manage to find one I 
have not spent time with........
>
>
>
>>
>
>> Any specific kernel tweaks?
>
>
>
> Most of my tweaks for KDE revolved around tuning mysqld itself. But for
> sysctls improving workstation responsiveness as it relates to memory
> interactions with I/O, these are my go-tos:
>
>
>
> vm.dirty_background_bytes = 1048576
> vm.dirty_bytes = 10485760
> vm.swappiness = 0

Mine are::
cat dirty_bytes
0
cat dirty_background_bytes
0
cat swappiness
60


>
> vm.dirty_background_bytes ensures that any data (i.e. from mmap or
> fwrite, not from swapping) waiting to be written to disk *starts*
> getting written to disk once you've got at least the configured amount
> (1MB) of data waiting. (If you've got a disk controller with
> battery-backed or flash-backed write cache, you might consider
> increasing this to some significant fraction of your write cache. I.e.
> if you've got a 1GB FBWC with 768MB of that dedicated to write cache,
> you might set this to 512MB or so. Depending on your workload. I/O
> tuning is for those of us who enjoy the dark arts.)
>
>
>
> vm.dirty_bytes says that once you've got the configured amount (10MB) of
> data waiting to be disk, then no more asynchronous I/O is permitted
> until you have no more data waiting; all outstanding writes must be
> finished first. (My rule of thumb is to have this between 2-10 times the
> value of vm.dirty_background_bytes. Though I'm really trying to avoid it
> being high enough that it could take more than 50ms to transfer to disk;
> that way, any stalls that do happen are almost imperceptible.)
>
>
>
> You want vm.dirty_background_bytes to be high enough that your hardware
> doesn't spend its time powered on if it doesn't have to be, and so that
> your hardware can transfer data in large, efficient, streamable chunks.
>
>
>
> You want vm.dirty_bytes enough higher than your first number so that
> your hardware has enough time to spin up and transfer data before you
> put the hammer down and say, "all right, nobody else gets to queue
> writes until all the waiting data has reached disk."
>
>
>
> You want vm.dirty_bytes *low* enough that when you *do* have to put that
> hammer down, it doesn't interfere with your perceptions of a responsive
> system. (And in a server context, you want it low enough that things
> can't time out--or be pushed into timing out--waiting for it. Call your
> user attention a matter of timing out expecting things to respond to
> you, and the same principle applies...)
>
>
>
> Now, vm.swappiness? That's a weighting factor for how quickly the kernel
> should try moving memory to swap to be able to speedily respond to new
> allocations. Me, I prefer the kernel to not preemptively move
> lesser-used data to swap, because that's going to be a few hundred
> megabytes worth of data all associated with one application, and it'll
> be a real drag when I switch back to the application I haven't used for
> half an hour. So I set vm.swappiness to 0, to tell the kernel to only
> move data to swap if it has no other alternative while trying to satisfy
> a new memory allocation request.


OK, OK, OK. I need to read a bit about these. Any references or docs or 
is the result of parsing out what is the least painful for a 
workstation? I do not run any heavy databases on my workstation; they
are only there to hack on them. I test db centric stuff on domain 
servers, sometimes with limited resources. I run lxde and I'm moving to 
lxqt for workstations and humanoid (terminal) IO.


Do you set these differently for servers?

Nodes in a cluster?

I use OpenRC, just so you know. I also have a motherboard with IOMMU 
that is currently has questionable settings in the kernel config file. I 
cannot find consensus if/how IOMMU that affects IO with the Sata HD 
devices versus mm mapped peripherals.... in the context of 4.x kernel 
options. I'm trying very hard here to avoid a deep dive on these issues, 
so trendy strategies are most welcome, as workstation and cluster node 
optimizations are all I'm really working on atm.


THANKS (as always)!

James

Re: [gentoo-user] kde-apps/kde-l10n-16.04.3:5/5::gentoo conflicting with kde-apps/kdepim-l10n-15.12.3:5/5::gentoo

[Michael Mol]

[-- Attachment #1: Type: text/plain, Size: 7367 bytes --]

On Tuesday, August 09, 2016 01:23:57 PM james wrote:
> On 08/09/2016 09:17 AM, Michael Mol wrote:
> > On Tuesday, August 09, 2016 09:13:31 AM james wrote:
> >> On 08/09/2016 07:42 AM, Michael Mol wrote:
> >> > On Monday, August 08, 2016 10:45:09 PM Alan McKinnon wrote:
> >> >> On 08/08/2016 19:20, Michael Mol wrote:
> >> >>> On Monday, August 08, 2016 06:52:15 PM Alan McKinnon wrote:
> >> >>>> On 08/08/2016 17:02, Michael Mol wrote:

> > I use Zabbix extensively at work, and have the Zabbix agent on my
> > workstation reporting back various supported metrics. There's a great
> > deal you can use (and--my favorite--abuse) Zabbix for, especially once
> > you understand how it thinks.
> 
> Congradualtions! Of the net-analyzer crowd, you've manage to find one I
> have not spent time with........

Oh, man, are you in for a treat. I recently had a conversation with a guy I 
happened to sit next to while traveling about how, were I in his position, I'd 
improve his cash crop and hydroponics operations (he periodically tests soil 
and sunlight properties) continually using a combination of cheap, custom 
probes and SBCs, feeding the data into Zabbix for monitoring and trend 
analysis / prediction. Zabbix will do time-series graphing and analysis of 
arbitrary input data; it may have been designed for watching interface 
counters, but there's no reason it need be limited to that...

> 
> >> Any specific kernel tweaks?
> > 
> > Most of my tweaks for KDE revolved around tuning mysqld itself. But for
> > sysctls improving workstation responsiveness as it relates to memory
> > interactions with I/O, these are my go-tos:
> > 
> > 
> > 
> > vm.dirty_background_bytes = 1048576
> > vm.dirty_bytes = 10485760
> > vm.swappiness = 0
> 
> Mine are::
> cat dirty_bytes
> 0
> cat dirty_background_bytes
> 0

So, that means you have vm.dirty_bytes_ratio and vm.dirty_background_ratio 
set, instead. I forget what those default to, but I think 
dirty_bacgkround_ratio defaults to something like 10, which means *10%* of 
your memory may get used for buffering disk I/O before it starts writing data 
to disk. dirty_bytes_ratio will necessarily be higher, which means that if 
you're performing seriously write-intensive activities on a system with 32GiB 
of RAM, you may find yourself with a system that will halt until it finishes 
flushing 3+GiB of data to disk.

> cat swappiness
> 60

Yeah, you want that set to lower than that.

> 
> > vm.dirty_background_bytes ensures that any data (i.e. from mmap or
> > fwrite, not from swapping) waiting to be written to disk *starts*
> > getting written to disk once you've got at least the configured amount
> > (1MB) of data waiting. (If you've got a disk controller with
> > battery-backed or flash-backed write cache, you might consider
> > increasing this to some significant fraction of your write cache. I.e.
> > if you've got a 1GB FBWC with 768MB of that dedicated to write cache,
> > you might set this to 512MB or so. Depending on your workload. I/O
> > tuning is for those of us who enjoy the dark arts.)
> > 
> > 
> > 
> > vm.dirty_bytes says that once you've got the configured amount (10MB) of
> > data waiting to be disk, then no more asynchronous I/O is permitted
> > until you have no more data waiting; all outstanding writes must be
> > finished first. (My rule of thumb is to have this between 2-10 times the
> > value of vm.dirty_background_bytes. Though I'm really trying to avoid it
> > being high enough that it could take more than 50ms to transfer to disk;
> > that way, any stalls that do happen are almost imperceptible.)
> > 
> > 
> > 
> > You want vm.dirty_background_bytes to be high enough that your hardware
> > doesn't spend its time powered on if it doesn't have to be, and so that
> > your hardware can transfer data in large, efficient, streamable chunks.
> > 
> > 
> > 
> > You want vm.dirty_bytes enough higher than your first number so that
> > your hardware has enough time to spin up and transfer data before you
> > put the hammer down and say, "all right, nobody else gets to queue
> > writes until all the waiting data has reached disk."
> > 
> > 
> > 
> > You want vm.dirty_bytes *low* enough that when you *do* have to put that
> > hammer down, it doesn't interfere with your perceptions of a responsive
> > system. (And in a server context, you want it low enough that things
> > can't time out--or be pushed into timing out--waiting for it. Call your
> > user attention a matter of timing out expecting things to respond to
> > you, and the same principle applies...)
> > 
> > 
> > 
> > Now, vm.swappiness? That's a weighting factor for how quickly the kernel
> > should try moving memory to swap to be able to speedily respond to new
> > allocations. Me, I prefer the kernel to not preemptively move
> > lesser-used data to swap, because that's going to be a few hundred
> > megabytes worth of data all associated with one application, and it'll
> > be a real drag when I switch back to the application I haven't used for
> > half an hour. So I set vm.swappiness to 0, to tell the kernel to only
> > move data to swap if it has no other alternative while trying to satisfy
> > a new memory allocation request.
> 
> OK, OK, OK. I need to read a bit about these. Any references or docs or
> is the result of parsing out what is the least painful for a
> workstation? I do not run any heavy databases on my workstation; they
> are only there to hack on them. I test db centric stuff on domain
> servers, sometimes with limited resources. I run lxde and I'm moving to
> lxqt for workstations and humanoid (terminal) IO.

https://www.kernel.org/doc/Documentation/sysctl/vm.txt
https://lonesysadmin.net/2013/12/22/better-linux-disk-caching-performance-vm-dirty_ratio/

> 
> 
> Do you set these differently for servers?

On my servers, I keep these values similar, because I'd rather have a little 
bit lower throughput than risk a catastrophic cascade failure stemming from an 
I/O stall.

> 
> Nodes in a cluster?

Same story.

The exception is my storage cluster, which has dirty_bytes much higher, as 
it's very solidly battery backed, so I can use its oodles of memory as a write 
cache, giving its kernel time to reorder writes and flush data to disk 
efficiently, and letting clients very rapidly return from write requests.

> 
> I use OpenRC, just so you know. I also have a motherboard with IOMMU
> that is currently has questionable settings in the kernel config file. I
> cannot find consensus if/how IOMMU that affects IO with the Sata HD
> devices versus mm mapped peripherals.... in the context of 4.x kernel
> options. I'm trying very hard here to avoid a deep dive on these issues,
> so trendy strategies are most welcome, as workstation and cluster node
> optimizations are all I'm really working on atm.

Honestly, I'd suggest you deep dive. An image once, with clarity, will last 
you a lot longer than ongoing fuzzy and trendy images from people whose 
hardware and workflow is likely to be different from yours.

The settings I provided should be absolutely fine for most use cases. Only 
exception would be mobile devices with spinning rust, but those are getting 
rarer and rarer...

-- 
:wq

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 473 bytes --]

Re: [gentoo-user] kde-apps/kde-l10n-16.04.3:5/5::gentoo conflicting with kde-apps/kdepim-l10n-15.12.3:5/5::gentoo

[Alan McKinnon]

On 09/08/2016 18:09, Daniel Frey wrote:
> On 08/09/2016 05:42 AM, Michael Mol wrote:
>> I used Thunderbird for years, but I eventually had to stop when it would, 
>> averaging once a month (though sometimes not for a couple months, sometimes a 
>> couple times a week) explode in memory consumption and drive the entire system 
>> unresponsively into swap.
>>
> 
> I've been using thunderbird exclusively on my PC and haven't seen this
> particular issue. When was the last time you tried it?
> 
> I've probably got between 8k and 10k messages in it right now. Memory
> consumption is 3.3% of 8GB and I do see every 30 seconds or so
> thunderbird wakes up and does something for a few seconds, using 8-10%
> of CPU while it does. But I've never noticed it actually doing anything
> (like slowing the system to a crawl.)
> 
> Dan
> 
> 


Every time I've seen Thunderbird stuuter and stall, it's been network
related. Usually I'm trying to access a large IMAP store remotely (that
tends to stall all IMAP clients to some degree depending on how well the
system deals with blocking).

-- 
Alan McKinnon
alan.mckinnon@gmail.com

Re: [gentoo-user] kde-apps/kde-l10n-16.04.3:5/5::gentoo conflicting with kde-apps/kdepim-l10n-15.12.3:5/5::gentoo

[Michael Mol]

[-- Attachment #1: Type: text/plain, Size: 1392 bytes --]

On Tuesday, August 09, 2016 09:09:53 AM Daniel Frey wrote:
> On 08/09/2016 05:42 AM, Michael Mol wrote:
> > I used Thunderbird for years, but I eventually had to stop when it would,
> > averaging once a month (though sometimes not for a couple months,
> > sometimes a couple times a week) explode in memory consumption and drive
> > the entire system unresponsively into swap.
> 
> I've been using thunderbird exclusively on my PC and haven't seen this
> particular issue. When was the last time you tried it?

I think I gave up on Thunderbird around February or March? Dunno. It was 
earlier this year.

> 
> I've probably got between 8k and 10k messages in it right now. Memory
> consumption is 3.3% of 8GB and I do see every 30 seconds or so
> thunderbird wakes up and does something for a few seconds, using 8-10%
> of CPU while it does. But I've never noticed it actually doing anything
> (like slowing the system to a crawl.)

I've got a few hundred thousand messages. Not interested in asking the thing 
for an exact count, as that takes a while. ;)

Thing is, I'd go for weeks, just fine, only 700MB or so of memory consumed. 
Then, abruptly, its memory consumed would climb to fill all 8GB of my physical 
memory. And if it happened over night, it'd be to about 1.4GB of swap before 
the Zabbix agent stopped sending telemetry to my collector...

-- 
:wq

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 473 bytes --]

Re: [gentoo-user] kde-apps/kde-l10n-16.04.3:5/5::gentoo conflicting with kde-apps/kdepim-l10n-15.12.3:5/5::gentoo

[james]

On 08/09/2016 01:41 PM, Michael Mol wrote:
> On Tuesday, August 09, 2016 01:23:57 PM james wrote:
>> On 08/09/2016 09:17 AM, Michael Mol wrote:
>>> On Tuesday, August 09, 2016 09:13:31 AM james wrote:
>>>> On 08/09/2016 07:42 AM, Michael Mol wrote:
>>>>> On Monday, August 08, 2016 10:45:09 PM Alan McKinnon wrote:
>>>>>> On 08/08/2016 19:20, Michael Mol wrote:
>>>>>>> On Monday, August 08, 2016 06:52:15 PM Alan McKinnon wrote:
>>>>>>>> On 08/08/2016 17:02, Michael Mol wrote:
>
>>> I use Zabbix extensively at work, and have the Zabbix agent on my
>>> workstation reporting back various supported metrics. There's a great
>>> deal you can use (and--my favorite--abuse) Zabbix for, especially once
>>> you understand how it thinks.
>>
>> Congradualtions! Of the net-analyzer crowd, you've manage to find one I
>> have not spent time with........
>
> Oh, man, are you in for a treat. I recently had a conversation with a guy I
> happened to sit next to while traveling about how, were I in his position, I'd
> improve his cash crop and hydroponics operations (he periodically tests soil
> and sunlight properties) continually using a combination of cheap, custom
> probes and SBCs, feeding the data into Zabbix for monitoring and trend
> analysis / prediction. Zabbix will do time-series graphing and analysis of
> arbitrary input data; it may have been designed for watching interface
> counters, but there's no reason it need be limited to that...

Not sure of your tendencies, but yea, I tend to be more hardware and EE 
oriented, than CS. Yep, I spent too many years with time-sequenced data 
(turds) to not be totally excited about what we can now do with 
clusters, analog (16 bit+) IO  and enough processors and memory to keep
a simulation going and in RT(color).  You sure know how to instigate an 
itch.....

Besides, as I transcend  retirement, I'm looking for greener  pastures
and methodologies to enhance da(tm) dream state  ......
(thx)


>>>> Any specific kernel tweaks?
>>>
>>> Most of my tweaks for KDE revolved around tuning mysqld itself. But for
>>> sysctls improving workstation responsiveness as it relates to memory
>>> interactions with I/O, these are my go-tos:
>>>
>>>
>>>
>>> vm.dirty_background_bytes = 1048576
>>> vm.dirty_bytes = 10485760
>>> vm.swappiness = 0
>>
>> Mine are::
>> cat dirty_bytes
>> 0
>> cat dirty_background_bytes
>> 0
>
> So, that means you have vm.dirty_bytes_ratio and vm.dirty_background_ratio
> set, instead. I forget what those default to, but I think
> dirty_bacgkround_ratio defaults to something like 10, which means *10%* of
> your memory may get used for buffering disk I/O before it starts writing data
> to disk. dirty_bytes_ratio will necessarily be higher, which means that if
> you're performing seriously write-intensive activities on a system with 32GiB
> of RAM, you may find yourself with a system that will halt until it finishes
> flushing 3+GiB of data to disk.
>
>> cat swappiness
>> 60
>
> Yeah, you want that set to lower than that.
>
>>
>>> vm.dirty_background_bytes ensures that any data (i.e. from mmap or
>>> fwrite, not from swapping) waiting to be written to disk *starts*
>>> getting written to disk once you've got at least the configured amount
>>> (1MB) of data waiting. (If you've got a disk controller with
>>> battery-backed or flash-backed write cache, you might consider
>>> increasing this to some significant fraction of your write cache. I.e.
>>> if you've got a 1GB FBWC with 768MB of that dedicated to write cache,
>>> you might set this to 512MB or so. Depending on your workload. I/O
>>> tuning is for those of us who enjoy the dark arts.)
>>>
>>>
>>> vm.dirty_bytes says that once you've got the configured amount (10MB) of
>>> data waiting to be disk, then no more asynchronous I/O is permitted
>>> until you have no more data waiting; all outstanding writes must be
>>> finished first. (My rule of thumb is to have this between 2-10 times the
>>> value of vm.dirty_background_bytes. Though I'm really trying to avoid it
>>> being high enough that it could take more than 50ms to transfer to disk;
>>> that way, any stalls that do happen are almost imperceptible.)
>>>
>>>
>>>
>>> You want vm.dirty_background_bytes to be high enough that your hardware
>>> doesn't spend its time powered on if it doesn't have to be, and so that
>>> your hardware can transfer data in large, efficient, streamable chunks.
>>>
>>>
>>>
>>> You want vm.dirty_bytes enough higher than your first number so that
>>> your hardware has enough time to spin up and transfer data before you
>>> put the hammer down and say, "all right, nobody else gets to queue
>>> writes until all the waiting data has reached disk."
>>>
>>> You want vm.dirty_bytes *low* enough that when you *do* have to put that
>>> hammer down, it doesn't interfere with your perceptions of a responsive
>>> system. (And in a server context, you want it low enough that things
>>> can't time out--or be pushed into timing out--waiting for it. Call your
>>> user attention a matter of timing out expecting things to respond to
>>> you, and the same principle applies...)
>>>
>>> Now, vm.swappiness? That's a weighting factor for how quickly the kernel
>>> should try moving memory to swap to be able to speedily respond to new
>>> allocations. Me, I prefer the kernel to not preemptively move
>>> lesser-used data to swap, because that's going to be a few hundred
>>> megabytes worth of data all associated with one application, and it'll
>>> be a real drag when I switch back to the application I haven't used for
>>> half an hour. So I set vm.swappiness to 0, to tell the kernel to only
>>> move data to swap if it has no other alternative while trying to satisfy
>>> a new memory allocation request.
>>
>> OK, OK, OK. I need to read a bit about these. Any references or docs or
>> is the result of parsing out what is the least painful for a
>> workstation? I do not run any heavy databases on my workstation; they
>> are only there to hack on them. I test db centric stuff on domain
>> servers, sometimes with limited resources. I run lxde and I'm moving to
>> lxqt for workstations and humanoid (terminal) IO.
>
> https://www.kernel.org/doc/Documentation/sysctl/vm.txt
> https://lonesysadmin.net/2013/12/22/better-linux-disk-caching-performance-vm-dirty_ratio/

Excellent docs, thx.

>> Do you set these differently for servers?
>
> On my servers, I keep these values similar, because I'd rather have a little
> bit lower throughput than risk a catastrophic cascade failure stemming from an
> I/O stall.
>
>>
>> Nodes in a cluster?
>
> Same story.
>
> The exception is my storage cluster, which has dirty_bytes much higher, as
> it's very solidly battery backed, so I can use its oodles of memory as a write
> cache, giving its kernel time to reorder writes and flush data to disk
> efficiently, and letting clients very rapidly return from write requests.

Are these TSdB (time series data) by chance?

OK, so have your systematically experimented with these parameter 
settings, collected and correlated the data, domain (needs) specific ?

As unikernels collide with my work on building up  minimized and 
optimized linux clusters, my pathway forward is to use several small 
clusters, where the codes/frameworks can be changed, even the 
tweaked-tuned kernels and DFS and note the performance differences for 
very specific domain solutions. My examples are quite similar to that 
aforementioned  flight sim above, but the ordinary and uncommon 
workloads of regular admin (dev/ops) work is only a different domain.

Ideas on automating the exploration of these settings 
(scripts/traces/keystores) are keenly of interest to me, just so you know.



>> I use OpenRC, just so you know. I also have a motherboard with IOMMU
>> that is currently has questionable settings in the kernel config file. I
>> cannot find consensus if/how IOMMU that affects IO with the Sata HD
>> devices versus mm mapped peripherals.... in the context of 4.x kernel
>> options. I'm trying very hard here to avoid a deep dive on these issues,
>> so trendy strategies are most welcome, as workstation and cluster node
>> optimizations are all I'm really working on atm.
>
> Honestly, I'd suggest you deep dive. An image once, with clarity, will last
> you a lot longer than ongoing fuzzy and trendy images from people whose
> hardware and workflow is likely to be different from yours.
>
> The settings I provided should be absolutely fine for most use cases. Only
> exception would be mobile devices with spinning rust, but those are getting
> rarer and rarer...

I did a quick test with games-arcade/xgalaga. It's an old, quirky game 
with sporadic lag variations. On a workstation with 32G ram and (8) 4GHz 
64bit cores, very lightly loaded, there is no reason for in game lag. 
Your previous settings made it much better and quicker the vast majority 
of the time; but not optimal (always responsive). Experiences tell me if 
I can tweak a system so that that game stays responsive whilst the 
application(s) mix is concurrently running then the  quick 
test+parameter settings is reasonably well behaved. So thats becomes a 
baseline for further automated tests and fine tuning for a system under 
study.


Perhaps Zabbix +TSdB can get me further down the pathway.  Time 
sequenced and analyzed data is over kill for this (xgalaga) test, but 
those coalesced test-vectors  will be most useful for me as I seek a 
gentoo centric pathway for low latency clusters (on bare metal).

TIA,


James

Re: [gentoo-user] kde-apps/kde-l10n-16.04.3:5/5::gentoo conflicting with kde-apps/kdepim-l10n-15.12.3:5/5::gentoo

[Michael Mol]

[-- Attachment #1: Type: text/plain, Size: 6412 bytes --]

On Tuesday, August 09, 2016 05:22:22 PM james wrote:
> On 08/09/2016 01:41 PM, Michael Mol wrote:
> > On Tuesday, August 09, 2016 01:23:57 PM james wrote:

> > The exception is my storage cluster, which has dirty_bytes much higher, as
> > it's very solidly battery backed, so I can use its oodles of memory as a
> > write cache, giving its kernel time to reorder writes and flush data to
> > disk efficiently, and letting clients very rapidly return from write
> > requests.
> Are these TSdB (time series data) by chance?

No; my TS data is stored in a MySQL VM whose storage is host-local.

> 
> OK, so have your systematically experimented with these parameter
> settings, collected and correlated the data, domain (needs) specific ?

Not with these particular settings; what they *do* is fairly straightforward, 
so establishing configuration constraints is a function of knowing the capacity 
and behavior of the underlying hardware; there's little need to guess.

For hypothetical example, let's say you're using a single spinning rust disk 
with an enabled write cache of 64MiB. (Common enough, although you should 
ensure the write cache is disabled if you find yourself at risk of poweroff. You 
should be able to script that with nut, or even acpid, though.) That means the 
disk could queue up 64MiB of data to be be written, and efficiently reorder 
writes to flush them to disk faster. So, in that circumstance, perhaps you'd 
set dirty_background_bytes to 64MiB, so that the kernel will try to feed it a 
full cache's worth of data at once, giving the drive a chance to optimize its 
write ordering.

For another hypothetical example, let's say you're using a parity RAID array 
with three data disks and two parity disks, with a strip length of 1MiB. Now, 
with parity RAID, if you modify a small bit of data, when that data gets 
committed to disk, the parity bits need to get updated as well. That means 
that small write requires first reading the relevant portions of all three data 
disks, holding them in memory, adjusting the portion you wrote to, calculating 
the parity, and writing the result out to all five disks. But if you make a 
*large* write that replaces all of the data in the stripe (so, a well-placed 
3MiB write, in this case), you don't have to read the disks to find out what 
data was already there, and can simply write out your data and parity. In this 
case, perhaps you want to set dirty_background_bytes to 3MiB (or some multiple 
thereof), so that the kernel doesn't try flushing data to disk until it has a 
full stripe's worth of material, and can forgo a time-consuming initial read.

For a final hypothetical example, consider SSDs. SSDs share one interesting 
thing in common with parity RAID arrays...they have an optimum write size 
that's a lot larger than 4KiB. When you write a small amount of data to an 
SSD, it has to read an entire block of NAND flash, modify it in its own RAM, 
and write that entire block back out to NAND flash. (All of this happens 
internally to the SSD.) So, for efficiency, you want to give the SSD an entire 
block's worth of data to write at a time, if you can. So you might set 
dirty_background_bytes to the size of the SSD's block, because the fewer the 
write cycles, the longer it will last. (Different model SSDs will have different 
block sizes, ranging anywhere from 512KiB to 8MiB, currently.)

> 
> As unikernels collide with my work on building up  minimized and
> optimized linux clusters, my pathway forward is to use several small
> clusters, where the codes/frameworks can be changed, even the
> tweaked-tuned kernels and DFS and note the performance differences for
> very specific domain solutions. My examples are quite similar to that
> aforementioned  flight sim above, but the ordinary and uncommon
> workloads of regular admin (dev/ops) work is only a different domain.
> 
> Ideas on automating the exploration of these settings
> (scripts/traces/keystores) are keenly of interest to me, just so you know.

I think I missed some context, despite rereading what was already discussed.

> 
> >> I use OpenRC, just so you know. I also have a motherboard with IOMMU
> >> that is currently has questionable settings in the kernel config file. I
> >> cannot find consensus if/how IOMMU that affects IO with the Sata HD
> >> devices versus mm mapped peripherals.... in the context of 4.x kernel
> >> options. I'm trying very hard here to avoid a deep dive on these issues,
> >> so trendy strategies are most welcome, as workstation and cluster node
> >> optimizations are all I'm really working on atm.
> > 
> > Honestly, I'd suggest you deep dive. An image once, with clarity, will
> > last
> > you a lot longer than ongoing fuzzy and trendy images from people whose
> > hardware and workflow is likely to be different from yours.
> > 
> > The settings I provided should be absolutely fine for most use cases. Only
> > exception would be mobile devices with spinning rust, but those are
> > getting
> > rarer and rarer...
> 
> I did a quick test with games-arcade/xgalaga. It's an old, quirky game
> with sporadic lag variations. On a workstation with 32G ram and (8) 4GHz
> 64bit cores, very lightly loaded, there is no reason for in game lag.
> Your previous settings made it much better and quicker the vast majority
> of the time; but not optimal (always responsive). Experiences tell me if
> I can tweak a system so that that game stays responsive whilst the
> application(s) mix is concurrently running then the  quick
> test+parameter settings is reasonably well behaved. So thats becomes a
> baseline for further automated tests and fine tuning for a system under
> study.

What kind of storage are you running on? What filesystem? If you're still 
hitting swap, are you using a swap file or a swap partition?

> 
> 
> Perhaps Zabbix +TSdB can get me further down the pathway.  Time
> sequenced and analyzed data is over kill for this (xgalaga) test, but
> those coalesced test-vectors  will be most useful for me as I seek a
> gentoo centric pathway for low latency clusters (on bare metal).

If you're looking to avoid Zabbix interfering with your performance, you'll 
want the Zabbix server and web interface on a machine separate from the 
machines you're trying to optimize.

-- 
:wq

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 473 bytes --]

Re: [gentoo-user] kde-apps/kde-l10n-16.04.3:5/5::gentoo conflicting with kde-apps/kdepim-l10n-15.12.3:5/5::gentoo

[james]

On 08/10/2016 07:45 AM, Michael Mol wrote:
> On Tuesday, August 09, 2016 05:22:22 PM james wrote:
>> On 08/09/2016 01:41 PM, Michael Mol wrote:
>>> On Tuesday, August 09, 2016 01:23:57 PM james wrote:
>
>>> The exception is my storage cluster, which has dirty_bytes much higher, as
>>> it's very solidly battery backed, so I can use its oodles of memory as a
>>> write cache, giving its kernel time to reorder writes and flush data to
>>> disk efficiently, and letting clients very rapidly return from write
>>> requests.
>> Are these TSdB (time series data) by chance?
>
> No; my TS data is stored in a MySQL VM whose storage is host-local.
>
>>
>> OK, so have your systematically experimented with these parameter
>> settings, collected and correlated the data, domain (needs) specific ?
>
> Not with these particular settings; what they *do* is fairly straightforward,
> so establishing configuration constraints is a function of knowing the capacity
> and behavior of the underlying hardware; there's little need to guess.
>
> For hypothetical example, let's say you're using a single spinning rust disk
> with an enabled write cache of 64MiB. (Common enough, although you should
> ensure the write cache is disabled if you find yourself at risk of poweroff. You
> should be able to script that with nut, or even acpid, though.) That means the
> disk could queue up 64MiB of data to be be written, and efficiently reorder
> writes to flush them to disk faster. So, in that circumstance, perhaps you'd
> set dirty_background_bytes to 64MiB, so that the kernel will try to feed it a
> full cache's worth of data at once, giving the drive a chance to optimize its
> write ordering.
>
> For another hypothetical example, let's say you're using a parity RAID array
> with three data disks and two parity disks, with a strip length of 1MiB. Now,
> with parity RAID, if you modify a small bit of data, when that data gets
> committed to disk, the parity bits need to get updated as well. That means
> that small write requires first reading the relevant portions of all three data
> disks, holding them in memory, adjusting the portion you wrote to, calculating
> the parity, and writing the result out to all five disks. But if you make a
> *large* write that replaces all of the data in the stripe (so, a well-placed
> 3MiB write, in this case), you don't have to read the disks to find out what
> data was already there, and can simply write out your data and parity. In this
> case, perhaps you want to set dirty_background_bytes to 3MiB (or some multiple
> thereof), so that the kernel doesn't try flushing data to disk until it has a
> full stripe's worth of material, and can forgo a time-consuming initial read.
>
> For a final hypothetical example, consider SSDs. SSDs share one interesting
> thing in common with parity RAID arrays...they have an optimum write size
> that's a lot larger than 4KiB. When you write a small amount of data to an
> SSD, it has to read an entire block of NAND flash, modify it in its own RAM,
> and write that entire block back out to NAND flash. (All of this happens
> internally to the SSD.) So, for efficiency, you want to give the SSD an entire
> block's worth of data to write at a time, if you can. So you might set
> dirty_background_bytes to the size of the SSD's block, because the fewer the
> write cycles, the longer it will last. (Different model SSDs will have different
> block sizes, ranging anywhere from 512KiB to 8MiB, currently.)


Ok, after reading some of the docs and postings, several time, I see how 
to focus in on the exact hardware on a specific system. The nice thing 
about clusters is they are largely identical systems, or groups of 
identical systems, in quantity so that helps with scaling issues.... 
testing specific hardware, individually, should lead to near-optimal 
default settings so they can bee deployed as cluster nodes, later.


>> As unikernels collide with my work on building up  minimized and
>> optimized linux clusters, my pathway forward is to use several small
>> clusters, where the codes/frameworks can be changed, even the
>> tweaked-tuned kernels and DFS and note the performance differences for
>> very specific domain solutions. My examples are quite similar to that
>> aforementioned  flight sim above, but the ordinary and uncommon
>> workloads of regular admin (dev/ops) work is only a different domain.
>>
>> Ideas on automating the exploration of these settings
>> (scripts/traces/keystores) are keenly of interest to me, just so you know.
>
> I think I missed some context, despite rereading what was already discussed.

Yea, I was thinking out loud here. just ignore this...

>>>> I use OpenRC, just so you know. I also have a motherboard with IOMMU
>>>> that is currently has questionable settings in the kernel config file. I
>>>> cannot find consensus if/how IOMMU that affects IO with the Sata HD
>>>> devices versus mm mapped peripherals.... in the context of 4.x kernel
>>>> options. I'm trying very hard here to avoid a deep dive on these issues,
>>>> so trendy strategies are most welcome, as workstation and cluster node
>>>> optimizations are all I'm really working on atm.
>>>
>>> Honestly, I'd suggest you deep dive. An image once, with clarity, will
>>> last
>>> you a lot longer than ongoing fuzzy and trendy images from people whose
>>> hardware and workflow is likely to be different from yours.
>>>
>>> The settings I provided should be absolutely fine for most use cases. Only
>>> exception would be mobile devices with spinning rust, but those are
>>> getting
>>> rarer and rarer...
>>
>> I did a quick test with games-arcade/xgalaga. It's an old, quirky game
>> with sporadic lag variations. On a workstation with 32G ram and (8) 4GHz
>> 64bit cores, very lightly loaded, there is no reason for in game lag.
>> Your previous settings made it much better and quicker the vast majority
>> of the time; but not optimal (always responsive). Experiences tell me if
>> I can tweak a system so that that game stays responsive whilst the
>> application(s) mix is concurrently running then the  quick
>> test+parameter settings is reasonably well behaved. So thats becomes a
>> baseline for further automated tests and fine tuning for a system under
>> study.
>
> What kind of storage are you running on? What filesystem? If you're still
> hitting swap, are you using a swap file or a swap partition?

The system I mostly referenced, rarely hits swap in days of uptime. It's 
the keyboard latency, while playing the game, that I try to tune away, 
while other codes are running. I try very hard to keep codes from 
swapping out, cause ultimately I'm most interested in clusters that keep 
everything running (in memory). AkA ultimate utilization of Apache-Spark 
and other "in-memory" techniques.


Combined codes running simultaneously never hits the HD (no swappiness) 
but still there is keyboard lag. Not that it is actually affecting the 
running codes to any appreciable degree, but it is a test I run so that 
the cluster nodes will benefit from still being (low latency) quickly 
attentive to interactions with the cluster master processes, regardless 
of workloads on the nodes. Sure its  not totally accurate, but so far 
this semantical approach, is pretty darn close. It's not part of this 
conversation (on VM etc) but ultimately getting this right solves one of 
the biggest problems for building any cluster; that is workload 
invocation, shedding and management to optimize resource utilization, 
regardless of the orchestration(s) used to manage the nodes. Swapping to 
disc is verbotim, in my (ultimate) goals and target scenarios.

No worries, you have given me enough info and ideas to move forward with 
testing and tuning. I'm going to evolve these  into more precisely 
controlled and monitored experiments, noting exact hardware differences; 
that should complete the tuning of the Memory Management tasks, within 
acceptable confine  . Then automate it for later checking on cluster 
test runs with various hardware setups. Eventually these test will be 
extended to a variety of  memory and storage hardware, once the 
techniques are automated. No worries, I now have enough ideas and 
details (thanks to you) to move forward.


>> Perhaps Zabbix +TSdB can get me further down the pathway.  Time
>> sequenced and analyzed data is over kill for this (xgalaga) test, but
>> those coalesced test-vectors  will be most useful for me as I seek a
>> gentoo centric pathway for low latency clusters (on bare metal).
>
> If you're looking to avoid Zabbix interfering with your performance, you'll
> want the Zabbix server and web interface on a machine separate from the
> machines you're trying to optimize.

agreed.

Thanks Mike,
James

Re: [gentoo-user] kde-apps/kde-l10n-16.04.3:5/5::gentoo conflicting with kde-apps/kdepim-l10n-15.12.3:5/5::gentoo

[Michael Mol]

[-- Attachment #1: Type: text/plain, Size: 4182 bytes --]

On Wednesday, August 10, 2016 10:13:29 AM james wrote:
> On 08/10/2016 07:45 AM, Michael Mol wrote:
> > On Tuesday, August 09, 2016 05:22:22 PM james wrote:

> >> 
> >> I did a quick test with games-arcade/xgalaga. It's an old, quirky game
> >> with sporadic lag variations. On a workstation with 32G ram and (8) 4GHz
> >> 64bit cores, very lightly loaded, there is no reason for in game lag.
> >> Your previous settings made it much better and quicker the vast majority
> >> of the time; but not optimal (always responsive). Experiences tell me if
> >> I can tweak a system so that that game stays responsive whilst the
> >> application(s) mix is concurrently running then the  quick
> >> test+parameter settings is reasonably well behaved. So thats becomes a
> >> baseline for further automated tests and fine tuning for a system under
> >> study.
> > 
> > What kind of storage are you running on? What filesystem? If you're still
> > hitting swap, are you using a swap file or a swap partition?
> 
> The system I mostly referenced, rarely hits swap in days of uptime. It's
> the keyboard latency, while playing the game, that I try to tune away,
> while other codes are running. I try very hard to keep codes from
> swapping out, cause ultimately I'm most interested in clusters that keep
> everything running (in memory). AkA ultimate utilization of Apache-Spark
> and other "in-memory" techniques.

Gotcha. dirty_bytes and dirty_background_bytes won't apply to anything that 
doesn't call mmap() with a file backing or perform some other file I/O. If 
you're not doing those things, they should have little to no impact.

Ideal values for dirty_bytes and dirty_background_bytes will depend heavily on 
the nature of your underlying storage. Dozens of other things might be tweaked 
depending on what filesystem you're using. Which is why I was asking about 
those things.

> 
> 
> Combined codes running simultaneously never hits the HD (no swappiness)
> but still there is keyboard lag.

Where are you measuring this lag? How much lag are we talking about?

> Not that it is actually affecting the
> running codes to any appreciable degree, but it is a test I run so that
> the cluster nodes will benefit from still being (low latency) quickly
> attentive to interactions with the cluster master processes, regardless
> of workloads on the nodes. Sure its  not totally accurate, but so far
> this semantical approach, is pretty darn close. It's not part of this
> conversation (on VM etc) but ultimately getting this right solves one of
> the biggest problems for building any cluster; that is workload
> invocation, shedding and management to optimize resource utilization,
> regardless of the orchestration(s) used to manage the nodes. Swapping to
> disc is verbotim, in my (ultimate) goals and target scenarios.
> 
> No worries, you have given me enough info and ideas to move forward with
> testing and tuning. I'm going to evolve these  into more precisely
> controlled and monitored experiments, noting exact hardware differences;
> that should complete the tuning of the Memory Management tasks, within
> acceptable confine  . Then automate it for later checking on cluster
> test runs with various hardware setups. Eventually these test will be
> extended to a variety of  memory and storage hardware, once the
> techniques are automated. No worries, I now have enough ideas and
> details (thanks to you) to move forward.

You've got me curious, now you're going to go run off and play with your 
thought problems and not share! Tease!

> 
> >> Perhaps Zabbix +TSdB can get me further down the pathway.  Time
> >> sequenced and analyzed data is over kill for this (xgalaga) test, but
> >> those coalesced test-vectors  will be most useful for me as I seek a
> >> gentoo centric pathway for low latency clusters (on bare metal).
> > 
> > If you're looking to avoid Zabbix interfering with your performance,
> > you'll
> > want the Zabbix server and web interface on a machine separate from the
> > machines you're trying to optimize.
> 
> agreed.
> 
> Thanks Mike,
> James

np
-- 
:wq

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 473 bytes --]

Re: [gentoo-user] kde-apps/kde-l10n-16.04.3:5/5::gentoo conflicting with kde-apps/kdepim-l10n-15.12.3:5/5::gentoo

[james]

On 08/10/2016 10:20 AM, Michael Mol wrote:
> On Wednesday, August 10, 2016 10:13:29 AM james wrote:
>> On 08/10/2016 07:45 AM, Michael Mol wrote:
>>> On Tuesday, August 09, 2016 05:22:22 PM james wrote:
>
>>>>
>>>> I did a quick test with games-arcade/xgalaga. It's an old, quirky game
>>>> with sporadic lag variations. On a workstation with 32G ram and (8) 4GHz
>>>> 64bit cores, very lightly loaded, there is no reason for in game lag.
>>>> Your previous settings made it much better and quicker the vast majority
>>>> of the time; but not optimal (always responsive). Experiences tell me if
>>>> I can tweak a system so that that game stays responsive whilst the
>>>> application(s) mix is concurrently running then the  quick
>>>> test+parameter settings is reasonably well behaved. So thats becomes a
>>>> baseline for further automated tests and fine tuning for a system under
>>>> study.
>>>
>>> What kind of storage are you running on? What filesystem? If you're still
>>> hitting swap, are you using a swap file or a swap partition?
>>
>> The system I mostly referenced, rarely hits swap in days of uptime. It's
>> the keyboard latency, while playing the game, that I try to tune away,
>> while other codes are running. I try very hard to keep codes from
>> swapping out, cause ultimately I'm most interested in clusters that keep
>> everything running (in memory). AkA ultimate utilization of Apache-Spark
>> and other "in-memory" techniques.
>
> Gotcha. dirty_bytes and dirty_background_bytes won't apply to anything that
> doesn't call mmap() with a file backing or perform some other file I/O. If
> you're not doing those things, they should have little to no impact.

Background needed:: I'm one of those (idealists?) that deeply believes 
the holy grail of computing will soon emerge (nice pun huh). That is
that clusters, local clusters will run all workloads that multicore 
systems currently do. So a bunch of old crap can become a beautiful
computational system, whilst I sit back and sip exotic beverages and 
enjoy my day; video training to go to the gym and dominate the young 
studs on the court.... New hardware (aka new computers and cosmetic 
surgery) will do the rest.

So an incredible variety of memory, storage and file systems will 
ultimately need to be tested. I try to stay simple and focused (believe 
it  or not). Initially the thought is to run a primitive desktop, like
lxde or lxqt and use those under utilized resources as 
node-computational contributors, whist still remaining responsive at the 
keyboard (xgalaga is a quick and dirty test for this). So, you now have 
a wonderful cover story is the boss catches you noodling around with 
swords and sorcery to, you can tell'm you looking for subtle latency 
issues...... The game speeds up and slows down, with zero swapping, due 
to my I suspect mostly as VM issues and MM issues.
An 8 core never goes above 0.2 on the load and only rarely saturates one 
core, for a transient instance. Even if xgalaga is a single thread game, 
it does not explain this transient keyboard lag. I'm open to other forms 
of quick at-the-keyboard graphical tests as a quick and dirty 
measurement of overall system attentiveness to pending addtional 
input/workload demands. After that is happy, with a given set of running 
codes (test-vectors) I can get a very quick feedback of performance this 
way.

For deeper studies, I like trace-cmd/Ftrace/KernelShark, but those are 
like zabbix on utilization and analytical studies. I use xgalaga as a 
quick and dirty; but am surely open to new codes for that sort of quick 
and easy feedback.



> Ideal values for dirty_bytes and dirty_background_bytes will depend heavily on
> the nature of your underlying storage. Dozens of other things might be tweaked
> depending on what filesystem you're using. Which is why I was asking about
> those things.

A myriad of combinations exist. So picking some common combinations, 
will allow for others to test my work, when it is package up for sharing 
and testing. For me eventually automating a collection of 'test vectors' 
is what's important, not the first few test-vectors themselvs. Then the 
pathway forward for other collections of running processes can become 
yet another collection of 'test vectors'. No limit on these collectives. 
Eventually a customer will step forward and define the collective of 
'test vectors', so I do hope to work with/for one of the more 
progressive vendors, eventually, in these efforts. Certainly sharing the 
work, openly, is far more important to me. For now, I start with things 
I like, know and have some familiarity with; no magic on these choices.


>> Combined codes running simultaneously never hits the HD (no swappiness)
>> but still there is keyboard lag.
>
> Where are you measuring this lag? How much lag are we talking about?

Remember, I'm an EE and complex fluids computational kind of guy, so I 
have no problem drudging down the sparse or full matrix types of 
mentally inebriating adventuresome calculations, like computational 
chemistry. But, since this approach is not yet ready for those sorts of 
things, I keep things simple; for now. What I want, is an automated 
installation semantic, where folks can download images and run them on 
their small clusters) on a weekly basis and keep solving the same 
test-vector collectives over and over. Tweaks and ideas are in the newly 
released images, a group of gentoo-users test things out. But
an automated, quick and simple gentoo system, flies against what most 
folks believe in this community (dammit, I have to respect, so I work on 
my one scripts I have lifted from others)  {wink wink; nudge nudge}.
As you already know....


>> Not that it is actually affecting the
>> running codes to any appreciable degree, but it is a test I run so that
>> the cluster nodes will benefit from still being (low latency) quickly
>> attentive to interactions with the cluster master processes, regardless
>> of workloads on the nodes. Sure its  not totally accurate, but so far
>> this semantical approach, is pretty darn close. It's not part of this
>> conversation (on VM etc) but ultimately getting this right solves one of
>> the biggest problems for building any cluster; that is workload
>> invocation, shedding and management to optimize resource utilization,
>> regardless of the orchestration(s) used to manage the nodes. Swapping to
>> disc is verbotim, in my (ultimate) goals and target scenarios.
>>
>> No worries, you have given me enough info and ideas to move forward with
>> testing and tuning. I'm going to evolve these  into more precisely
>> controlled and monitored experiments, noting exact hardware differences;
>> that should complete the tuning of the Memory Management tasks, within
>> acceptable confine  . Then automate it for later checking on cluster
>> test runs with various hardware setups. Eventually these test will be
>> extended to a variety of  memory and storage hardware, once the
>> techniques are automated. No worries, I now have enough ideas and
>> details (thanks to you) to move forward.
>
> You've got me curious, now you're going to go run off and play with your
> thought problems and not share! Tease!

Dude, I share too much. If you had not gone of vacation (from 
gentoo-user) you'd know this. Since I am way too mentally handicapped to 
do all of this on my own, (and too old and wise to even try) I routinely 
seek guidance and help. I read quite a lot, to remind me of the mistakes 
from previous distributed parallel computational attempts; and that 
reading also saddens me a bit to see so many malformed cluster ideas. Oh 
well, failure is the most important lesson technical folks learn. Most 
often ideas just bounces off the wall right back at me, but I have 
learned to duck (most of the time). YOU and anyone else are most welcome 
to join my efforts; we all shall benefit from robust, local clusters, as 
masters of gentoo (or poezer of gentoo, just like me). <end philosophy>

So while we are at it, scripts or stage-4 images that can be rapidly 
booted up on a given small hardware cluster, are keen to my approach.
Memory management, is probably the most challenging aspect of building 
and robustly (efficient resource utilization) managing these clusters
or outsourced clusters (clouds in vendor speak).  I Use the same cluster 
setup, to test a myriad of different problem-solution sets on the 
identical hardware, but only change the software, including file 
systems: both DFS (cephfs/orangefs/openAFS/Beefs) and the local fs (xfs, 
ext4,) and well as hybrids like btrfs and special file systsems like 
bcache. On top of Openstack, Hadoop, Mesos, old Beowulf (with a fast DFS 
replacing NFS) and others.

Once domain specific problems are moved to a cluster and that solution
set is near-optimal, after robustly testing many codes, in a CI fashion
outlined above, it becomes a stage-4 canned solution for somebody to run
on their hardware. If they need more hardware resouces, within a 
specific interval, THEN outsource those resource needs to the Cloud 
Vendors. Expecting a cloud vendor to be a champion of your Domain 
Specific need, is a roadmap to chapter 11 or 13, for that corporation.
I suspect that once AWS and Google and MS and IBM learn what the NSA 
already knows, there will be a feeding frenzy on aquisitions of old 
technology companies. That's ultimately where the action is in clusters.

All of this 'smoke and mirrors' marketing centric on social networks is 
just that; smoke and mirros. Why do I say this? Simple; there already is 
enough processing power to solve those problems and needs with current 
Snoracle style solutions and the by the bloated on wall-street.

Now HPC, dude, that's the sweet edge of clustering. There are numerous
gargantuan issues in that sphere and a few, like DESHAW are getting RICH 
off of clusters. He, a single Stanford professor, mastered computational 
chemistry, and locked his expertise into ASIC chips.
Now he is conquering wallstreet. Domain Specific solutions are where the 
action is in clusters. It not that there's not money in the social 
networking spheres, those are locked up by the 'cost barrier to entry'
semantics. OK, I digress. But the important thing is local clusters, 
taht can be rapidly build and torn down and reconfigured, with a few 
simple keystrokes, are the future of clusters. A given small to mid 
sized company better learn how to build their own clusters, or they be 
in the welfare line, like several other billion folks are.


CoreOS and unikernels are really quite similar to my approach to 
clusters. A variety of Problem-solutions sets (aka test vectors) on 
identical hardware will light the pathway for Domain Specific cluster 
solutions. Mine will be a node cluster on amd64, for now.

So, I'm not sitting on some Stanford level of skills or knowledge base 
(think amplabs). I have decades of experiences in mostly unfulfilled 
promises for ubiquitous distributed processing, and only narrowly (very 
tightly) focuses success stories. Still, I am a believer in that the 
current crop of linux clusters will become an Utopia computation engine 
system that works from the most modest of needs like  mundane admin 
taskloads to the most demanding, time-sequence RT simulations of some of 
the grand challenges in computational dynamics and similar areas.

But, after several years of research, I mostly see kids trying the same 
crap we tried decades ago, with a new 'fancy-pants' programming 
language:: (hence the prediction that the current cluster kids are being 
manipulated by the VC firms and deep pocketed folks toward certain 
failure), whilst they pay off their debts. Same story, different overlord.


I am conflicted as to whether this is intentional or just a repeat of 
tards leading the blind and innocent off the cliff. That is most of the 
vendor centric cluster (marketers call these clouds), developing new 
codes are clueless. That said, surely those corps with large collections 
of existing software can migrate those critical codes to the cloud and 
only offer new versions of that software, with a (cloud centric) 
internet-needed license. Think Azure/MS, IBM etc etc. But that sort of 
position, will just allow competitors to eat away larger chunks of their 
market share. (But I really don't care about his part of the Cloud 
illusion. I'm a hard core hardware type who already knows that the 
future of clusters is mostly local, with local control. The cloud will 
become a secondary or tertiary market for cpu cycles and garbage 
collection (think social networking databases). Sure folks will put 
their websites on commercial clouds, but that is already just a natural 
evolution of Co-location of server and not some breakthrough is technology.




Down this pathway, the developments in the latest version of Clang, gcc, 
etc etc, and EEs making the resources of the GPU (including DDR5+) into 
a transparent computational resource for the routine compilers. rDMA is 
going to change (everything). Ram will finally not be the bottleneck, as 
FPGA and GPU resources can be configured, dynamically, as either highly 
specialize processors or highly specialized memory (look at CAMs, or 
Content Addressible Memory for a teaser). Router vendors have been 
making billions of dollars by adding CAMs to otherwise mundane 
processing systsems.

No more of those ancient (intel) parallel compilers and shit like 
that.... Plus and avalance of re-configurable memory types; mostly 
transparent to folks that use "emerge" for custom compiling. Then there 
is a hardened kernel. Few in the cluster world even know such things 
exist; more sadly why they are necessary and when they are necessary.
Keep puffing on that buntu hoka pipe, brah_heim.....


The flip side to this is that a lot of Vendors think that bloated linux 
operating systems, on top of non-tuned, non-stripped insecure linux 
kernels is going to be commercially viable. If you build your house on 
turds, when it starts to rain, there is a funky smell in the air, before 
it washes away.  Bloated buntu, debian or RHEL are turds and are not 
going to work compared to stripped, minimal linux systems. That's where 
Docker, just "bitch-slapped" their competition by moving to subsume 
Apline linux.....

Your postings and clarity on VM, has helped me focus, immensely. It is 
the current need in my work. Have I shared enough for you, today?

Any other questions, or ideas are most welcome, publically or privately.
I could be wrong about all of this, but, my fourth generational stab at
ubiquitous (distributed || parallel) processing experiences tell me I'm 
not wrong but have the right idea. I do lack current skills in so many 
areas, that my work is impeded.

Without the gentoo community, I could not posses such visions of 
future-present greatness; nor share it with others.




>>>> Perhaps Zabbix +TSdB can get me further down the pathway.  Time
>>>> sequenced and analyzed data is over kill for this (xgalaga) test, but
>>>> those coalesced test-vectors  will be most useful for me as I seek a
>>>> gentoo centric pathway for low latency clusters (on bare metal).
>>>
>>> If you're looking to avoid Zabbix interfering with your performance,
>>> you'll
>>> want the Zabbix server and web interface on a machine separate from the
>>> machines you're trying to optimize.
>>
>> agreed.
>>
>> Thanks Mike,
>> James
>
> np


Clusters will end up on people's wrist watches, in the trunks of their 
autos and at their homes:: So they control their computational needs and 
security, sooner  rather than later. I think the next president will 
mandate the opening of the OS to many vendors and open source for Cell 
phones, Apps and such. The current monopolies are excessively more 
powerful than the old 'robber barrons' and that fact is well recognized 
by lost of deeper thinkers. It's braned under globalization, but, it's 
demise is just under the horizon, imho.

True, ubiquitous clusters will be a result of hard work on compilers 
that take sequential problems and break them down into pieces and 
reassemble them into a form that can leverage parallel techniques. gcc 5 
and 6 and Clang are moving, rapidly in this direction. GPU vendors 
understand the importance of SIMD and MIMD processing for 'systolic' 
algorithms and such approaches to massive distributed processing. AMD 
(Radeon) understands that this power can most effectively be used, if it 
is cheap and open sourced. Nvidia, no so quick to follow (or lead) down 
this open source path, imho. Intel purchasing a FPGA company and 
licensing GPU technologies from many others, tells me the hardware 
vendors are preparing for a revolution. A direct sales channel to the 
commoners will be their greatest path to rediculous profitability. Why? 
Simple, the smaller the core (competitive team) that exists, the more 
excessive processing resources that will be purchased and purchased 
closer to the retail price.

When hardware vendors partner with a few sofware companies, the margins 
on hardware get squeezed. Besides the hackers of the work, are finding 
any critical barriers to codes and publishing it so all have fair access 
to the latest codes (one way or another). The NSA and such entities are 
not going to stop this, because all of this software espionage, 
justifies governments taxing the snot out of citizens to fight those 
evil hackers. It's a far superior business model for DoD
types like intel and google, than the cold ware ever though about being.
The average tax-payer is too stupid to realize social network, with an 
Onior approach, is just feeding data-sets via google, linkedin, facebook
etc, directly to the NSA and other Nation State actors. WE get jobs
and pay taxes. They set the rules and manage the data.

Problem is, eventually, the commoners will have sufficent clusters, 
solar panels water wells or sources and  green house and tell da_main
to stick his taxes on imports. Fine that works, then everybody gets
a 3D printer and we, the commoners are self sufficient.

The simple fact is that is a great business model for EVERYONE, 
including the elites, so what are we waiting on?  A stupid old man like 
me? Naw, not at Gentoo, buntu, sure, RHEL definately, but not gentoo, 
brah. WE are the solution to everything!

</>

James