[gentoo-user] I want to play movies without hangs

[gentoo-user] I want to play movies without hangs

[Alex Schuster]

Hi there!

Strange things are going on here.

I've written here in the past about my performance problems. My dual-core
had trouble playing movies without stuttering when there was I/O. It was
mainly swapping that caused this, and 8 G were not enough for me running
KDE4.

Then my hardware broke, and I got new one, except for the system hard
drive and the PSU. It's an AMD FX-4100 quad-core with 3.6 GHz, 16 G of
RAM. Running gentoo-sources-3.2.1 as kernel. But it seems playing movies
got even worse!

The videos do not need to have high quality. When I do this, I get
interruptions, sometimes for more than a whole second:

# dd if=/dev/zero of=/tmp/argh bs=10M count=1000

My whole system is encrypted, but the same happens with unencrypted
partitions. All are on LVM. When I write to another drive, there is no
effect. Throughput is around 50-60 MB/s.

Any ideas where to look? I think I'll create a completely fresh
kernel .config with genkernel, maybe my own .config has some weird
problem. But I tried similar things in the past already, getting a kernel
from a live cd, to no effect.

I put cache = 10240 into .mplayer/config to get 10 MB of video cached,
but I see no effect.

Playing music with Amarok is no problem. 

My SATA drives are in AHCI mode, here's some dmesg info about that:

ahci 0000:00:11.0: version 3.0
ahci 0000:00:11.0: PCI INT A -> GSI 22 (level, low) -> IRQ 22
ahci 0000:00:11.0: AHCI 0001.0100 32 slots 4 ports 3 Gbps 0xf impl SATA
mode
ahci 0000:00:11.0: flags: 64bit ncq sntf ilck led clo pmp pio slum part
ccc sxs 
scsi0 : ahci
scsi1 : ahci
scsi2 : ahci
scsi3 : ahci
ata1: SATA max UDMA/133 abar m1024@0xff70b000 port 0xff70b100 irq 22
ata2: SATA max UDMA/133 abar m1024@0xff70b000 port 0xff70b180 irq 22
ata3: SATA max UDMA/133 abar m1024@0xff70b000 port 0xff70b200 irq 22
ata4: SATA max UDMA/133 abar m1024@0xff70b000 port 0xff70b280 irq 22
ahci 0000:02:00.0: PCI INT A -> GSI 19 (level, low) -> IRQ 19
ahci 0000:02:00.0: irq 43 for MSI/MSI-X
ahci: SSS flag set, parallel bus scan disabled
ahci 0000:02:00.0: AHCI 0001.0200 32 slots 2 ports 6 Gbps 0x3 impl SATA
mode
ahci 0000:02:00.0: flags: 64bit ncq sntf stag led clo pmp pio slum part
ccc sxs 
ahci 0000:02:00.0: setting latency timer to 64
scsi4 : ahci
scsi5 : ahci
ata5: SATA max UDMA/133 abar m512@0xff600000 port 0xff600100 irq 43
ata6: SATA max UDMA/133 abar m512@0xff600000 port 0xff600180 irq 43
pata_atiixp 0000:00:14.1: PCI INT A -> GSI 16 (level, low) -> IRQ 16
scsi6 : pata_atiixp
scsi7 : pata_atiixp
ata7: PATA max UDMA/100 cmd 0x1f0 ctl 0x3f6 bmdma 0xf000 irq 14
ata8: PATA max UDMA/100 cmd 0x170 ctl 0x376 bmdma 0xf008 irq 15

(ata7/8 is the additional PATA controller, seen with the pata_atiixp
driver. I have one drive there, but it is not being used.)

When my new girl-friend comes over and we want to watch a movie, and it
stutters... she will ask why I don't simply use Windows to get
better performance, her five year old PC would do this just fine. Wat do
I tell her? WHAT DO I TELL HER??


And then there's what happened yesterday. A world update was going on,
with libreoffice, firefox, wine and thunderbird emerging in parallel, all
big packages. I have the PORTAGE_TMPDIR on a 5GB tmpfs, only libreoffice
is being compiled on disk. Suddenly, my system became very unresponsive,
the mouse had disappeared, the KDE widgets did not update, and xosview
showed a load of 23. All 4 cores were at 100%, the type of usage was
io-wait. How can I find out in such a case which processes are waiting
for I/O? top showed nothing. The Ctrl-Esc task viewer of KDE showed some
processes being 'inactive on hard drive', does this men those are the
waiting tasks? They varied, they were mostly Akonadi stuff. I stopped
akonadi, and after a while the load dropped. But this may be a
coincidence.

After all had calmed down, I had 2G of swap in use. 16G total RAM, all
being used of course, but only 8G being needed according to the -/+
buffers/cache line in free -m, the other 8G are cache. Does my Linux
somehow prefer to have this much cache, even if tmpfs stuff gets put into
swap? I have vm.swappiness = 0 in /etc/sysctl.conf.

Is there a command to show me what processes the memory in swap belongs
to?

	Wonko

Re: [gentoo-user] I want to play movies without hangs

[Mark Knecht]

On Thu, Feb 16, 2012 at 7:29 AM, Alex Schuster <wonko@wonkology.org> wrote:
> Hi there!
>

Hi back at ya.

> How can I find out in such a case which processes are waiting
> for I/O? top showed nothing.

iotop is your friend.

I'll write more when I get some time to think

HTH,
Mark

Re: [gentoo-user] I want to play movies without hangs

[Paul Hartman]

On Thu, Feb 16, 2012 at 9:29 AM, Alex Schuster <wonko@wonkology.org> wrote:
> I've written here in the past about my performance problems. My dual-core
> had trouble playing movies without stuttering when there was I/O. It was
> mainly swapping that caused this, and 8 G were not enough for me running
> KDE4.
>
> Then my hardware broke, and I got new one, except for the system hard
> drive and the PSU. It's an AMD FX-4100 quad-core with 3.6 GHz, 16 G of
> RAM. Running gentoo-sources-3.2.1 as kernel. But it seems playing movies
> got even worse!

You don't mention anything about video card or video driver setup.
That's the first thing I would suspect.

What video card? What drivers? Are you using hardware accelerated
movie playback?

Re: [gentoo-user] I want to play movies without hangs

[Alex Schuster]

Paul Hartman writes:

> On Thu, Feb 16, 2012 at 9:29 AM, Alex Schuster <wonko@wonkology.org>
> wrote:

> > Then my hardware broke, and I got new one, except for the system hard
> > drive and the PSU. It's an AMD FX-4100 quad-core with 3.6 GHz, 16 G of
> > RAM. Running gentoo-sources-3.2.1 as kernel. But it seems playing
> > movies got even worse!
> 
> You don't mention anything about video card or video driver setup.
> That's the first thing I would suspect.
> 
> What video card? What drivers? Are you using hardware accelerated
> movie playback?

Sorry. Radeon HD 4250 onboard graphics, using the open source radeon
driver. Hardware acceleration is working fine. As I wrote, it doesn't
matter which quality the videos are. There is not much CPU being used at
all, around 5% to 20%, so this is not the bottleneck.

	Wonko

Re: [gentoo-user] I want to play movies without hangs

[Paul Hartman]

On Thu, Feb 16, 2012 at 10:49 AM, Alex Schuster <wonko@wonkology.org> wrote:
> Paul Hartman writes:
>
>> On Thu, Feb 16, 2012 at 9:29 AM, Alex Schuster <wonko@wonkology.org>
>> wrote:
>
>> > Then my hardware broke, and I got new one, except for the system hard
>> > drive and the PSU. It's an AMD FX-4100 quad-core with 3.6 GHz, 16 G of
>> > RAM. Running gentoo-sources-3.2.1 as kernel. But it seems playing
>> > movies got even worse!
>>
>> You don't mention anything about video card or video driver setup.
>> That's the first thing I would suspect.
>>
>> What video card? What drivers? Are you using hardware accelerated
>> movie playback?
>
> Sorry. Radeon HD 4250 onboard graphics, using the open source radeon
> driver. Hardware acceleration is working fine. As I wrote, it doesn't
> matter which quality the videos are. There is not much CPU being used at
> all, around 5% to 20%, so this is not the bottleneck.
>
>        Wonko
>

I wonder if you copy the movie to /dev/shm first (so disk I/O is not
an issue) does it still have problems? At least this can potentially
eliminate disk I/O as the cause if something else weird is going on.
:)

For the problem of massive amounts of RAM consumed, that's strange.
Are you compiling debug symbols? That can make the RAM usage (in
linking especially) explode...

Re: [gentoo-user] I want to play movies without hangs

[Alex Schuster]

Mark Knecht writes:

> On Thu, Feb 16, 2012 at 7:29 AM, Alex Schuster <wonko@wonkology.org>
> wrote:

> > How can I find out in such a case which processes are waiting
> > for I/O? top showed nothing.
> 
> iotop is your friend.

I had called it, but didn't spot the problem there. I don't remember
exactly what the output was, I had expected to see some process show a
large value in the IO column, but that was not the case. I THINK! Various
processes appeared, mostly Akonadi stuff.

I should have logged this, I don't remember this s well. I have caught
some illness, and had fever, which did not help my memory. There were
some kworker processes listed on top, but I don't remember whether in
iotop or in top.

Hmm. Now I just started Akonadi again, and get a lot I/O in iotop, and
xosview again shows much iowait CPU activity for a while. But that calmed
down after a minute.

> I'll write more when I get some time to think

Thanks :)

	Wonko

Re: [gentoo-user] I want to play movies without hangs

[Mark Knecht]

On Thu, Feb 16, 2012 at 9:34 AM, Alex Schuster <wonko@wonkology.org> wrote:
> Mark Knecht writes:
>
>> On Thu, Feb 16, 2012 at 7:29 AM, Alex Schuster <wonko@wonkology.org>
>> wrote:
>
>> > How can I find out in such a case which processes are waiting
>> > for I/O? top showed nothing.
>>
>> iotop is your friend.
>
> I had called it, but didn't spot the problem there. I don't remember
> exactly what the output was, I had expected to see some process show a
> large value in the IO column, but that was not the case. I THINK! Various
> processes appeared, mostly Akonadi stuff.
>
> I should have logged this, I don't remember this s well. I have caught
> some illness, and had fever, which did not help my memory. There were
> some kworker processes listed on top, but I don't remember whether in
> iotop or in top.
>
> Hmm. Now I just started Akonadi again, and get a lot I/O in iotop, and
> xosview again shows much iowait CPU activity for a while. But that calmed
> down after a minute.
>
>> I'll write more when I get some time to think
>
> Thanks :)
>
>        Wonko
>

Sorry. I was rushing then as now. If you start iotop and then hit the
'o' key it will show only processes actually doing io. If you're hang
is truly an iowait then my experience is that it should at least
identify what process is having the problem.

HTH,
Mark

Re: [gentoo-user] I want to play movies without hangs

[Walter Dnes]

On Thu, Feb 16, 2012 at 04:29:48PM +0100, Alex Schuster wrote

> Then my hardware broke, and I got new one...

  I had ***EXACTLY THE SAME PROBLEM ON A FRESH INSTALL***.  In My case
it was a 4+ year old Dell with onboard Intel GPU that was having
problems playing NHL Gamecenter Live streams at the slowest speed.  I
solved the problem and sped up everything by doing...
1) emerge system
2) emerge world
3) rebuild the kernel and reboot

  A fresh install will have the stage 3 binaries built with
lowest-common-denominator x86 or amd64 code (depending if you chose 32
or 64 bit install).  This is necessary in order to allow the install
code to run on all CPUs with the target platform.  The downside is that
you lose all the optimisations that make Gentoo scream.  Rebuilding the
install as described above builds optimized (i.e. faster) binaries.  My
CFLAGS line in /etc/make.conf is...

CFLAGS="-O2 -march=native -mfpmath=sse -fomit-frame-pointer -pipe"
CXXFLAGS="${CFLAGS}"

  Before rebuilding your system, go over your USE flags to make sure
you've got the maximum optimization.  To find out what your CPU
supports, execute the command

grep flags /proc/cpuinfo | head -1

  This will define the limits what your system can support.  For
instance, mplayer can use the following flags...

waltdnes@d530 ~ $ emerge -pv mplayer

These are the packages that would be merged, in order:

Calculating dependencies... done!
[ebuild   R    ] media-video/mplayer-1.0_rc4_p20110322-r1  USE="X a52 alsa ass dga encode gif jpeg mmx mmxext mng mp3 opengl png quicktime real rtmp sse sse2 ssse3 theora truetype win32codecs x264 xv xvid xvmc -3dnow -3dnowext -aalib (-altivec) -amr (-aqua) -bidi -bindist -bl -bluray -bs2b -cddb -cdio -cdparanoia -cpudetection -custom-cpuopts -debug -dirac -directfb -doc -dts -dv -dvb -dvd -dvdnav (-dxr3) -enca (-esd) -faac -faad -fbcon -ftp -ggi -gsm -iconv -ipv6 -jack -joystick -jpeg2k -ladspa -libcaca -libmpeg2 -lirc -live -lzo -mad -md5sum -mpg123 -nas -network -nut -openal -osdmenu -oss -pnm -pulseaudio -pvr -radio -rar -rtc -samba -schroedinger -sdl -shm -speex -tga -toolame -tremor -twolame -unicode -v4l -vdpau -vidix -vorbis -vpx -xanim -xinerama -xscreensaver -zoran" VIDEO_CARDS="-mga -s3virge -tdfx -vesa" 0 kB

  Your CPU will obviously support a different set of USE flags than
mine.  Check the files /usr/portage/profiles/use.desc for a list of
global flags and /usr/portage/profiles/use.local.desc for
package-specific flags.

-- 
Walter Dnes <waltdnes@waltdnes.org>

Re: [gentoo-user] I want to play movies without hangs

[Alex Schuster]

Paul Hartman writes:

> I wonder if you copy the movie to /dev/shm first (so disk I/O is not
> an issue) does it still have problems? At least this can potentially
> eliminate disk I/O as the cause if something else weird is going on.
> :)

Yes, this helps. As does copying the movie to another partition than that
on which I to the I/O with my dd if=/dev/zero of= command. If I dd to
this partition, tough, it happens again.

> For the problem of massive amounts of RAM consumed, that's strange.

It's been so for years... but with 16 G of RAM it's no longer an issue.
Well, unless this weird problem happened wth parallel emerges on tmpfs. It
doesn't happen every time though.

> Are you compiling debug symbols? That can make the RAM usage (in
> linking especially) explode...

No. I sometimes enable it, but only when I hunt a bug and want to produce
better bug reports.
And even if the emerge would need very much memory, shouldn't this be
taken from the 8 G of caches being used, instead of starting to swap?

	Wonko

Re: [gentoo-user] I want to play movies without hangs

[Alex Schuster]

Walter Dnes writes:

> On Thu, Feb 16, 2012 at 04:29:48PM +0100, Alex Schuster wrote
> 
> > Then my hardware broke, and I got new one...
> 
>   I had ***EXACTLY THE SAME PROBLEM ON A FRESH INSTALL***.  In My case
> it was a 4+ year old Dell with onboard Intel GPU that was having
> problems playing NHL Gamecenter Live streams at the slowest speed.  I
> solved the problem and sped up everything by doing...
> 1) emerge system
> 2) emerge world
> 3) rebuild the kernel and reboot

Good idea, Walter! But not in my case. The system had been set up long
ago, and I did an emerge -e @world in the past already.

>   A fresh install will have the stage 3 binaries built with
> lowest-common-denominator x86 or amd64 code (depending if you chose 32
> or 64 bit install).  This is necessary in order to allow the install
> code to run on all CPUs with the target platform.  The downside is that
> you lose all the optimisations that make Gentoo scream.  Rebuilding the
> install as described above builds optimized (i.e. faster) binaries.  My
> CFLAGS line in /etc/make.conf is...
> 
> CFLAGS="-O2 -march=native -mfpmath=sse -fomit-frame-pointer -pipe"
> CXXFLAGS="${CFLAGS}"

I had those, when I upgraded the hardware:
CFLAGS="-march=k8-sse3 -mfpmath=sse -O2 -pipe"
Or something very silimar.

But I also just did an emerge -e @world on the new system, using more
sophisticated CFLAGS. I got them by doing like suggested on [*], using
what -march=natve would do. And adding support for this graphite
stuff. They are:
CFLAGS="-pipe -march=amdfam10 -O2 \
        -floop-interchange -floop-strip-mine -floop-block \
        -msse -msse2 -msse3 -msse4 -msse4.1 -msse4.2 -m3dnow \
        -mcx16 -msahf -maes -mpclmul -mpopcnt -mabm -mlwp -mavx \
        --param l1-cache-size=16 --param l1-cache-line-size=64 \
        --param l2-cache-size=2048"

>   Before rebuilding your system, go over your USE flags to make sure
> you've got the maximum optimization.  To find out what your CPU
> supports, execute the command
> 
> grep flags /proc/cpuinfo | head -1
> 
>   This will define the limits what your system can support.  For
> instance, mplayer can use the following flags...
> 
> waltdnes@d530 ~ $ emerge -pv mplayer
> 
> These are the packages that would be merged, in order:
> 
> Calculating dependencies... done!
> [ebuild   R    ] media-video/mplayer-1.0_rc4_p20110322-r1  USE="X a52
> alsa ass dga encode gif jpeg mmx mmxext mng mp3 opengl png quicktime
> real rtmp sse sse2 ssse3 theora truetype win32codecs x264 xv xvid xvmc
> -3dnow -3dnowext -aalib (-altivec) -amr (-aqua) -bidi -bindist -bl
> -bluray -bs2b -cddb -cdio -cdparanoia -cpudetection -custom-cpuopts
> -debug -dirac -directfb -doc -dts -dv -dvb -dvd -dvdnav (-dxr3) -enca
> (-esd) -faac -faad -fbcon -ftp -ggi -gsm -iconv -ipv6 -jack -joystick
> -jpeg2k -ladspa -libcaca -libmpeg2 -lirc -live -lzo -mad -md5sum
> -mpg123 -nas -network -nut -openal -osdmenu -oss -pnm -pulseaudio -pvr
> -radio -rar -rtc -samba -schroedinger -sdl -shm -speex -tga -toolame
> -tremor -twolame -unicode -v4l -vdpau -vidix -vorbis -vpx -xanim
> -xinerama -xscreensaver -zoran" VIDEO_CARDS="-mga -s3virge -tdfx -vesa"
> 0 kB
> 
>   Your CPU will obviously support a different set of USE flags than
> mine.  Check the files /usr/portage/profiles/use.desc for a list of
> global flags and /usr/portage/profiles/use.local.desc for
> package-specific flags.

These are my USE flags for mplayer, they should be fine:
[ebuild   R    ] media-video/mplayer-1.0_rc4_p20120213  USE="3dnow
3dnowext X a52 aalib alsa ass cdio dga directfb dts dv dvb dvd dvdnav
enca encode faad fbcon ftp ggi gif iconv ipv6 jack jpeg jpeg2k live mad
mmx mmxext mng mp3 nas network openal opengl osdmenu oss png pnm
quicktime rar real rtc samba sdl shm speex sse sse2 ssse3 theora toolame
tremor truetype twolame unicode vorbis x264 xinerama xscreensaver xv xvid
(-altivec) (-aqua) -bidi -bindist -bl -bluray -bs2b -cddb -cdparanoia
-cpudetection -debug -doc (-dxr3) (-esd) -faac -gsm -joystick -ladspa
-libcaca -libmpeg2 -lirc -lzo -md5sum -nut -pulseaudio -pvr -radio -rtmp
-tga -v4l -vdpau (-vidix) (-win32codecs) -xanim -xvmc -zoran"
VIDEO_CARDS="-mga -s3virge -tdfx" 0 kB

Now I'm bulding a new kernel, using genkernel, and without providing a
custom made .config. Just in case I have some weird setting somewhere
(debug output for SCSI stuff or something like that).

[later...]

So I did. Argh. I thought genkernel was smart enough to generate a
working kernel from scratch, if no existing .config would be given. But
the initramfs could not open my encrypted root partition, until I compiled
XTS and AES directly into the kernel, not only as modules. Genkernel did
not include modules for my NIC, somewhat annoying because I had to wait
several minutes for mysql to start, until I could open a root shell. KDM
was already running at that time, but I only saw a blank screen, because
the radeon stuff was not compiled with KMS. There's also something
wrong with my hardware clock. And iotop does not work, the kernel is
missing CONFIG_TASKSTATS, CONFIG_TASK_DELAY_ACCT
and CONFIG_TASK_IO_ACCOUNTING.

But now I have KDE running again. As soon as I do my dd command, mplayer
stutters, sometimes hanging for seconds. So it's probably not a flaw in
my kernel .config. Too bad, I hoped that was the problem.

I'm out of ideas now. This is really annoying, my system is fast, mplayer
is using around 20% only, the system is mostly idle, but when there is
I/O, videos do not run smoothly. And I do not want to copy every movie I
want to watch to another drive first.

BTW, when I do the dd if=/dev/zero of=/home/argh bs=10M count=1000
command, and interrupt with Ctrl-C, it takes some seconds until it stops.
Shouldn't this happen immediately, unless I specify a very large block
size? Writing speed is okay, 100 MB/s.

Time to go to sleep now.

[*] http://en.gentoo-wiki.com/wiki/Safe_Cflags#-march.3Dnative

	Wonko

Re: [gentoo-user] I want to play movies without hangs

[James Broadhead]

On 18 February 2012 05:45, Alex Schuster <wonko@wonkology.org> wrote:
> Walter Dnes writes:
>
>> On Thu, Feb 16, 2012 at 04:29:48PM +0100, Alex Schuster wrote
>>
>> > Then my hardware broke, and I got new one...
>>
>>   I had ***EXACTLY THE SAME PROBLEM ON A FRESH INSTALL***.  In My case
>> it was a 4+ year old Dell with onboard Intel GPU that was having
>> problems playing NHL Gamecenter Live streams at the slowest speed.  I
>> solved the problem and sped up everything by doing...
>> 1) emerge system
>> 2) emerge world
>> 3) rebuild the kernel and reboot
>
> Good idea, Walter! But not in my case. The system had been set up long
> ago, and I did an emerge -e @world in the past already.
>
>>   A fresh install will have the stage 3 binaries built with
>> lowest-common-denominator x86 or amd64 code (depending if you chose 32
>> or 64 bit install).  This is necessary in order to allow the install
>> code to run on all CPUs with the target platform.  The downside is that
>> you lose all the optimisations that make Gentoo scream.  Rebuilding the
>> install as described above builds optimized (i.e. faster) binaries.  My
>> CFLAGS line in /etc/make.conf is...
>>
>> CFLAGS="-O2 -march=native -mfpmath=sse -fomit-frame-pointer -pipe"
>> CXXFLAGS="${CFLAGS}"
>
> I had those, when I upgraded the hardware:
> CFLAGS="-march=k8-sse3 -mfpmath=sse -O2 -pipe"
> Or something very silimar.
>
> But I also just did an emerge -e @world on the new system, using more
> sophisticated CFLAGS. I got them by doing like suggested on [*], using
> what -march=natve would do. And adding support for this graphite
> stuff. They are:
> CFLAGS="-pipe -march=amdfam10 -O2 \
>        -floop-interchange -floop-strip-mine -floop-block \
>        -msse -msse2 -msse3 -msse4 -msse4.1 -msse4.2 -m3dnow \
>        -mcx16 -msahf -maes -mpclmul -mpopcnt -mabm -mlwp -mavx \
>        --param l1-cache-size=16 --param l1-cache-line-size=64 \
>        --param l2-cache-size=2048"
>
>>   Before rebuilding your system, go over your USE flags to make sure
>> you've got the maximum optimization.  To find out what your CPU
>> supports, execute the command
>>
>> grep flags /proc/cpuinfo | head -1
>>
>>   This will define the limits what your system can support.  For
>> instance, mplayer can use the following flags...
>>
>> waltdnes@d530 ~ $ emerge -pv mplayer
>>
>> These are the packages that would be merged, in order:
>>
>> Calculating dependencies... done!
>> [ebuild   R    ] media-video/mplayer-1.0_rc4_p20110322-r1  USE="X a52
>> alsa ass dga encode gif jpeg mmx mmxext mng mp3 opengl png quicktime
>> real rtmp sse sse2 ssse3 theora truetype win32codecs x264 xv xvid xvmc
>> -3dnow -3dnowext -aalib (-altivec) -amr (-aqua) -bidi -bindist -bl
>> -bluray -bs2b -cddb -cdio -cdparanoia -cpudetection -custom-cpuopts
>> -debug -dirac -directfb -doc -dts -dv -dvb -dvd -dvdnav (-dxr3) -enca
>> (-esd) -faac -faad -fbcon -ftp -ggi -gsm -iconv -ipv6 -jack -joystick
>> -jpeg2k -ladspa -libcaca -libmpeg2 -lirc -live -lzo -mad -md5sum
>> -mpg123 -nas -network -nut -openal -osdmenu -oss -pnm -pulseaudio -pvr
>> -radio -rar -rtc -samba -schroedinger -sdl -shm -speex -tga -toolame
>> -tremor -twolame -unicode -v4l -vdpau -vidix -vorbis -vpx -xanim
>> -xinerama -xscreensaver -zoran" VIDEO_CARDS="-mga -s3virge -tdfx -vesa"
>> 0 kB
>>
>>   Your CPU will obviously support a different set of USE flags than
>> mine.  Check the files /usr/portage/profiles/use.desc for a list of
>> global flags and /usr/portage/profiles/use.local.desc for
>> package-specific flags.
>
> These are my USE flags for mplayer, they should be fine:
> [ebuild   R    ] media-video/mplayer-1.0_rc4_p20120213  USE="3dnow
> 3dnowext X a52 aalib alsa ass cdio dga directfb dts dv dvb dvd dvdnav
> enca encode faad fbcon ftp ggi gif iconv ipv6 jack jpeg jpeg2k live mad
> mmx mmxext mng mp3 nas network openal opengl osdmenu oss png pnm
> quicktime rar real rtc samba sdl shm speex sse sse2 ssse3 theora toolame
> tremor truetype twolame unicode vorbis x264 xinerama xscreensaver xv xvid
> (-altivec) (-aqua) -bidi -bindist -bl -bluray -bs2b -cddb -cdparanoia
> -cpudetection -debug -doc (-dxr3) (-esd) -faac -gsm -joystick -ladspa
> -libcaca -libmpeg2 -lirc -lzo -md5sum -nut -pulseaudio -pvr -radio -rtmp
> -tga -v4l -vdpau (-vidix) (-win32codecs) -xanim -xvmc -zoran"
> VIDEO_CARDS="-mga -s3virge -tdfx" 0 kB
>
> Now I'm bulding a new kernel, using genkernel, and without providing a
> custom made .config. Just in case I have some weird setting somewhere
> (debug output for SCSI stuff or something like that).
>
> [later...]
>
> So I did. Argh. I thought genkernel was smart enough to generate a
> working kernel from scratch, if no existing .config would be given. But
> the initramfs could not open my encrypted root partition, until I compiled
> XTS and AES directly into the kernel, not only as modules. Genkernel did
> not include modules for my NIC, somewhat annoying because I had to wait
> several minutes for mysql to start, until I could open a root shell. KDM
> was already running at that time, but I only saw a blank screen, because
> the radeon stuff was not compiled with KMS. There's also something
> wrong with my hardware clock. And iotop does not work, the kernel is
> missing CONFIG_TASKSTATS, CONFIG_TASK_DELAY_ACCT
> and CONFIG_TASK_IO_ACCOUNTING.
>
> But now I have KDE running again. As soon as I do my dd command, mplayer
> stutters, sometimes hanging for seconds. So it's probably not a flaw in
> my kernel .config. Too bad, I hoped that was the problem.
>
> I'm out of ideas now. This is really annoying, my system is fast, mplayer
> is using around 20% only, the system is mostly idle, but when there is
> I/O, videos do not run smoothly. And I do not want to copy every movie I
> want to watch to another drive first.
>
> BTW, when I do the dd if=/dev/zero of=/home/argh bs=10M count=1000
> command, and interrupt with Ctrl-C, it takes some seconds until it stops.
> Shouldn't this happen immediately, unless I specify a very large block
> size? Writing speed is okay, 100 MB/s.
>
> Time to go to sleep now.
>
> [*] http://en.gentoo-wiki.com/wiki/Safe_Cflags#-march.3Dnative
>
>        Wonko
>

Please try:
~/.mplayer/config
  lavdopts=threads=2
  # Use 128MiB input cache by default.
  cache = 131072
  # Prefill 20% of the cache before starting playback.
  cache-min = 20.0

Which should eliminate disk IO somewhat

Re: [gentoo-user] I want to play movies without hangs

[Alex Schuster]

James Broadhead writes:

> Please try:
> ~/.mplayer/config
>   lavdopts=threads=2
>   # Use 128MiB input cache by default.
>   cache = 131072
>   # Prefill 20% of the cache before starting playback.
>   cache-min = 20.0
> 
> Which should eliminate disk IO somewhat

James, thanks for your input. I already had threads = 2 in my config, now
I notice that this syntax is not valid. My cache setting was only 10M, and
I did not know about cache-min.

First, I thought these settings would help, the video I tested this with
this night no longer had problems. I commented the settings one after
another in order to find out which settings exactly helped most, and even
without them, it played fine. Maybe the whole video was in the cache at
that time already.

Then I tried another video, again with the settings you suggested, and it
stuttered. There were small pauses when the system did some stuff, and
when I did my dd test, the pauses were as long as five seconds. FIVE
SECONDS!

	Wonko

Re: [gentoo-user] I want to play movies without hangs

[Urs Schutz]

On Sat, 18 Feb 2012 18:04:07 +0100
Alex Schuster <wonko@wonkology.org> wrote:

...
> 
> Then I tried another video, again with the settings you
> suggested, and it stuttered. There were small pauses when
> the system did some stuff, and when I did my dd test, the
> pauses were as long as five seconds. FIVE SECONDS!
> 
> 	Wonko
> 

Just an idea: Is the disk OK? Replace /dev/sda with your
disk...

smartctl -t short /dev/sda

and after some minutes

smartctl --all /dev/sda

If all went OK then the status is «PASSED», and you could
try the extended or long tests with smartctl.
I had a bad disk here, which resulted in slow IO, but not
complete failure. Smart detected this immediately. Sorry,
I do not know how to check disks with LVM.

Urs

Re: [gentoo-user] I want to play movies without hangs

[Alex Schuster]

Urs Schutz writes:

> Just an idea: Is the disk OK? Replace /dev/sda with your
> disk...
> 
> smartctl -t short /dev/sda
> 
> and after some minutes
> 
> smartctl --all /dev/sda
> 
> If all went OK then the status is «PASSED», and you could
> try the extended or long tests with smartctl.

I have smartd running. A short self test is done every day, and a long test 
once per week.

> I had a bad disk here, which resulted in slow IO, but not
> complete failure. Smart detected this immediately. Sorry,
> I do not know how to check disks with LVM.

Didn't you get errors in yslog then?

I also thought about swapping the system drive - I have a larger backup 
drive, with nearly identical logical volumes on it, where I make backups 
with rdiffbackup. So even the content is identical, except for an additional 
rdiff-backup directory containing the increments. So all I have to do is to 
echange the two volume group names, reboot, and the system will run from the 
other drive. But I very much doubt this will help, transfer speed looks okay 
to me, around 100 MB/s with dd.

	Wonko

Re: [gentoo-user] I want to play movies without hangs

[Alex Schuster]

Some while ago, I wrote:

[
mplayer stutters when I/O is going on, even hangs for seconds when I do a
dd if=/dev/zero of=somefile bs=1M
]

> Urs Schutz writes:
> 
> > Just an idea: Is the disk OK? Replace /dev/sda with your
> > disk...
[...]
> > I had a bad disk here, which resulted in slow IO, but not
> > complete failure. Smart detected this immediately. Sorry,
> > I do not know how to check disks with LVM.
> 
> Didn't you get errors in yslog then?
> 
> I also thought about swapping the system drive - I have a larger backup 
> drive, with nearly identical logical volumes on it, where I make
> backups with rdiffbackup. So even the content is identical, except for
> an additional rdiff-backup directory containing the increments. So all
> I have to do is to echange the two volume group names, reboot, and the
> system will run from the other drive. But I very much doubt this will
> help, transfer speed looks okay to me, around 100 MB/s with dd.

I did it in another way. I created a large file system (LVM) on my 2nd
drive, copied /, /usr, /var, /opt and /home over. My whole system is
encrypted, but I omitted this, just to make sure this is not the
bottleneck.

Alas, no change. Another thing I tried was to change the SATA mode in my
BIOS from AHCI to whatever the other option is. This did not help either.

Now this is really annoying. I watch small clips mostly, and can live
with that, and when I want to watch stuff with others, I copy the file to
tmpfs, which seems to help a lot.

But now I found another solution: NOT USING KDE.

When X crashed (trying to make the old Unreal game play), I fired up
another window manager, and when I played a video in there, there was no
problem at all. So, I have another workaround.

But does anyone have an idea, why running KDE is the problem? Disabling
desktop effects does not help.

I must be totally crazy because I still want to use KDE, despite the big
trouble it gives me nearly every day. Yes, most things work fine now, but
there are many many little problems, daily application crashes, and every
time I log in I fear that the desktop won't come up. 8G of RAM was not
enough to avoid swapping, so now I have 16G, that's fine, I no longer
care about kwin using 1G of my RAM. Oh, and I no longer use KMail, after
it ate thousands of mails I just wanted to move. No problem, they were
not important, but I no longer trust the KDEPIM suite. And it seems the
developers do not care about this, the bug report got no replies.

But anyway. Any idea why it only happens with KDE? I will ask on the KDE
mailing list, but I thought I post here first, maybe there's something
Gentoo-specific going on here.

	Wonko

Re: [gentoo-user] I want to play movies without hangs

[Volker Armin Hemmann]

Am Montag, 7. Mai 2012, 14:41:34 schrieb Alex Schuster:
> Some while ago, I wrote:
> 
> [
> mplayer stutters when I/O is going on, even hangs for seconds when I do a
> dd if=/dev/zero of=somefile bs=1M
> ]
> 
> > Urs Schutz writes:
> > > Just an idea: Is the disk OK? Replace /dev/sda with your
> > > disk...
> 
> [...]
> 
> > > I had a bad disk here, which resulted in slow IO, but not
> > > complete failure. Smart detected this immediately. Sorry,
> > > I do not know how to check disks with LVM.
> > 
> > Didn't you get errors in yslog then?
> > 
> > I also thought about swapping the system drive - I have a larger backup
> > drive, with nearly identical logical volumes on it, where I make
> > backups with rdiffbackup. So even the content is identical, except for
> > an additional rdiff-backup directory containing the increments. So all
> > I have to do is to echange the two volume group names, reboot, and the
> > system will run from the other drive. But I very much doubt this will
> > help, transfer speed looks okay to me, around 100 MB/s with dd.
> 
> I did it in another way. I created a large file system (LVM) on my 2nd
> drive, copied /, /usr, /var, /opt and /home over. My whole system is
> encrypted, but I omitted this, just to make sure this is not the
> bottleneck.
> 
> Alas, no change. Another thing I tried was to change the SATA mode in my
> BIOS from AHCI to whatever the other option is. This did not help either.
> 
> Now this is really annoying. I watch small clips mostly, and can live
> with that, and when I want to watch stuff with others, I copy the file to
> tmpfs, which seems to help a lot.
> 
> But now I found another solution: NOT USING KDE.
> 
> When X crashed (trying to make the old Unreal game play), I fired up
> another window manager, and when I played a video in there, there was no
> problem at all. So, I have another workaround.
> 
> But does anyone have an idea, why running KDE is the problem? Disabling
> desktop effects does not help.

nepomuk/virtuoso running in the background

whenever you have more than 1 process doing IO linux sucks ass.

Now, you write to a partition, nepomuk tries to index it (1 read) and your 
player reading the video file (2nd read). Interactivity is shot.

Pause/kill nepomuk and look if it helps.

For me the worst case is: writing lots of files on a usb device - sucks 
everywhere, even on a vt.

-- 
#163933

Re: [gentoo-user] I want to play movies without hangs

[Michael Hampicke]

Am 07.05.2012 18:26, schrieb Volker Armin Hemmann:
> Am Montag, 7. Mai 2012, 14:41:34 schrieb Alex Schuster:
>> Some while ago, I wrote:
>>
>> [
>> mplayer stutters when I/O is going on, even hangs for seconds when I do a
>> dd if=/dev/zero of=somefile bs=1M
>> ]
>>
>>> Urs Schutz writes:
>>>> Just an idea: Is the disk OK? Replace /dev/sda with your
>>>> disk...
>>
>> [...]
>>
>>>> I had a bad disk here, which resulted in slow IO, but not
>>>> complete failure. Smart detected this immediately. Sorry,
>>>> I do not know how to check disks with LVM.
>>>
>>> Didn't you get errors in yslog then?
>>>
>>> I also thought about swapping the system drive - I have a larger backup
>>> drive, with nearly identical logical volumes on it, where I make
>>> backups with rdiffbackup. So even the content is identical, except for
>>> an additional rdiff-backup directory containing the increments. So all
>>> I have to do is to echange the two volume group names, reboot, and the
>>> system will run from the other drive. But I very much doubt this will
>>> help, transfer speed looks okay to me, around 100 MB/s with dd.
>>
>> I did it in another way. I created a large file system (LVM) on my 2nd
>> drive, copied /, /usr, /var, /opt and /home over. My whole system is
>> encrypted, but I omitted this, just to make sure this is not the
>> bottleneck.
>>
>> Alas, no change. Another thing I tried was to change the SATA mode in my
>> BIOS from AHCI to whatever the other option is. This did not help either.
>>
>> Now this is really annoying. I watch small clips mostly, and can live
>> with that, and when I want to watch stuff with others, I copy the file to
>> tmpfs, which seems to help a lot.
>>
>> But now I found another solution: NOT USING KDE.
>>
>> When X crashed (trying to make the old Unreal game play), I fired up
>> another window manager, and when I played a video in there, there was no
>> problem at all. So, I have another workaround.
>>
>> But does anyone have an idea, why running KDE is the problem? Disabling
>> desktop effects does not help.
> 
> nepomuk/virtuoso running in the background
> 
> whenever you have more than 1 process doing IO linux sucks ass.
> 
> Now, you write to a partition, nepomuk tries to index it (1 read) and your 
> player reading the video file (2nd read). Interactivity is shot.
> 
> Pause/kill nepomuk and look if it helps.
> 
> For me the worst case is: writing lots of files on a usb device - sucks 
> everywhere, even on a vt.
> 

Maybe changing the kernel io scheduler will help?

# /usr/src/linux/Documentation/block/switching-sched.txt
# /usr/src/linux/Documentation/block/deadline-iosched.txt
# /usr/src/linux/Documentation/block/cfq-iosched.txt

Re: [gentoo-user] I want to play movies without hangs

[Volker Armin Hemmann]

Am Montag, 7. Mai 2012, 18:44:05 schrieb Michael Hampicke:
> Maybe changing the kernel io scheduler will help?
> 
> # /usr/src/linux/Documentation/block/switching-sched.txt
> # /usr/src/linux/Documentation/block/deadline-iosched.txt
> # /usr/src/linux/Documentation/block/cfq-iosched.txt

nice idea - but that didn't help in the past. Why should it help now?

The question is - why does disk IO make the mouse jerky and delays keyboard 
input? That is just idiotic.

-- 
#163933

Re: [gentoo-user] I want to play movies without hangs

[Michael Hampicke]

> nice idea - but that didn't help in the past. Why should it help now?
> 
> The question is - why does disk IO make the mouse jerky and delays keyboard 
> input? That is just idiotic.

It was just an idea, but maybe there's something wrong on the hardware
side? Broken cable, hard drive about to die?

Have to checked the SMART data of your hard drive lately (error log,
relocate sector count). You can also run some tests with smartctl.

Or monitor your IO with dstat or iotop. iotop tells you which app causes
high io load.

Or maybe you kernel is using some generic and slow driver for your ata
controller instead of an optimized one?

Personally I never had IO problems on linux unless theres was something
wrong with either the hardware or the kernel driver.

Hope that helps.

Re: [gentoo-user] I want to play movies without hangs

[Volker Armin Hemmann]

Am Montag, 7. Mai 2012, 19:25:29 schrieb Michael Hampicke:
> > nice idea - but that didn't help in the past. Why should it help now?
> > 
> > The question is - why does disk IO make the mouse jerky and delays
> > keyboard
> > input? That is just idiotic.
> 
> It was just an idea, but maybe there's something wrong on the hardware
> side? Broken cable, hard drive about to die?

no, and this problem has been there since Suse 6.2 and kernel 2.2

> 
> Have to checked the SMART data of your hard drive lately (error log,
> relocate sector count). You can also run some tests with smartctl.

of course

> 
> Or monitor your IO with dstat or iotop. iotop tells you which app causes
> high io load.

cp of course

> 
> Or maybe you kernel is using some generic and slow driver for your ata
> controller instead of an optimized one?

seriously...

> 
> Personally I never had IO problems on linux unless theres was something
> wrong with either the hardware or the kernel driver.

good for you

> 
> Hope that helps.

no

-- 
#163933

Re: [gentoo-user] I want to play movies without hangs

[Mark Knecht]

On Mon, May 7, 2012 at 5:41 AM, Alex Schuster <wonko@wonkology.org> wrote:
> Some while ago, I wrote:
>
> [
> mplayer stutters when I/O is going on, even hangs for seconds when I do a
> dd if=/dev/zero of=somefile bs=1M
> ]
>
>> Urs Schutz writes:
>>
>> > Just an idea: Is the disk OK? Replace /dev/sda with your
>> > disk...
> [...]
>> > I had a bad disk here, which resulted in slow IO, but not
>> > complete failure. Smart detected this immediately. Sorry,
>> > I do not know how to check disks with LVM.
>>
>> Didn't you get errors in yslog then?
>>
>> I also thought about swapping the system drive - I have a larger backup
>> drive, with nearly identical logical volumes on it, where I make
>> backups with rdiffbackup. So even the content is identical, except for
>> an additional rdiff-backup directory containing the increments. So all
>> I have to do is to echange the two volume group names, reboot, and the
>> system will run from the other drive. But I very much doubt this will
>> help, transfer speed looks okay to me, around 100 MB/s with dd.
>
> I did it in another way. I created a large file system (LVM) on my 2nd
> drive, copied /, /usr, /var, /opt and /home over. My whole system is
> encrypted, but I omitted this, just to make sure this is not the
> bottleneck.
>
> Alas, no change. Another thing I tried was to change the SATA mode in my
> BIOS from AHCI to whatever the other option is. This did not help either.
>
> Now this is really annoying. I watch small clips mostly, and can live
> with that, and when I want to watch stuff with others, I copy the file to
> tmpfs, which seems to help a lot.
>
> But now I found another solution: NOT USING KDE.
>
> When X crashed (trying to make the old Unreal game play), I fired up
> another window manager, and when I played a video in there, there was no
> problem at all. So, I have another workaround.
>
> But does anyone have an idea, why running KDE is the problem? Disabling
> desktop effects does not help.
>
> I must be totally crazy because I still want to use KDE, despite the big
> trouble it gives me nearly every day. Yes, most things work fine now, but
> there are many many little problems, daily application crashes, and every
> time I log in I fear that the desktop won't come up. 8G of RAM was not
> enough to avoid swapping, so now I have 16G, that's fine, I no longer
> care about kwin using 1G of my RAM. Oh, and I no longer use KMail, after
> it ate thousands of mails I just wanted to move. No problem, they were
> not important, but I no longer trust the KDEPIM suite. And it seems the
> developers do not care about this, the bug report got no replies.
>
> But anyway. Any idea why it only happens with KDE? I will ask on the KDE
> mailing list, but I thought I post here first, maybe there's something
> Gentoo-specific going on here.
>
>        Wonko
>

Hey Wonko,
   OK, fire up two terminals. In one run top, hit 1 & z so you see all
your CPUs and then watch CPU usage. In the second terminal su to root
and run iotop -o. Now, watch for a few minutes and get a feel for
what's going on when video is not running. Then start your video and
watch IO usage and CPU usage. Where's the problem?

   Once you get an idea where the bottleneck is we can address what a
solution might be. In general, if the CPUs aren't maxed out and it's
an I/O problem then usually a bit more buffering is a simple solution.
Other more draconian solution might be a real-time kernel with a
player (if there is one) that is set up for real-time playback.

   Looking forward to hearing your test results.

Cheers,
Mark

Re: [gentoo-user] I want to play movies without hangs

[Alex Schuster]

Michael Hampicke writes:

> Am 07.05.2012 18:26, schrieb Volker Armin Hemmann:
> > Am Montag, 7. Mai 2012, 14:41:34 schrieb Alex Schuster:
> >> Some while ago, I wrote:
> >>
> >> [
> >> mplayer stutters when I/O is going on, even hangs for seconds when I
> >> do a dd if=/dev/zero of=somefile bs=1M
> >> ]
[...]
> >> But now I found another solution: NOT USING KDE.
> >>
> >> When X crashed (trying to make the old Unreal game play), I fired up
> >> another window manager, and when I played a video in there, there
> >> was no problem at all. So, I have another workaround.
> >>
> >> But does anyone have an idea, why running KDE is the problem?
> >> Disabling desktop effects does not help.
> > 
> > nepomuk/virtuoso running in the background
> > 
> > whenever you have more than 1 process doing IO linux sucks ass.
> > 
> > Now, you write to a partition, nepomuk tries to index it (1 read) and
> > your player reading the video file (2nd read). Interactivity is shot.
> > 
> > Pause/kill nepomuk and look if it helps.

Nice shot, but this cannot be the problem. Sorry, now that I read my post
again I see I did not mention that I _still_ have KDE running all the
time, doing whatever it does. I only need to play the videos outside in a
window manager I have running in parallel.
And the output of my dd action goes to another partition anyway, which is
not being indexed. I have 280,000 files indexed, a plain locate gives
nearly ten times as much.
Virtuoso has been a big pain in the past indeed. So I had it disabled
until KDE 4.8 I think. It does some indexing now when I log in, and it is
still doing so 8 hours after I logged in the last time, but I do not
notice this much, since KDE 4.8. Whether it affects my video playback I'm
not so sure, but I have the playback problems even when it is not running
- there's too much stuff going on all the time.

> > For me the worst case is: writing lots of files on a usb device -
> > sucks everywhere, even on a vt.

Indeed. But should this affect an mplayer using a huge cache?
Does it also suck when you are writing with cp, using ionice -c 3?

> Maybe changing the kernel io scheduler will help?
> 
> # /usr/src/linux/Documentation/block/switching-sched.txt
> # /usr/src/linux/Documentation/block/deadline-iosched.txt
> # /usr/src/linux/Documentation/block/cfq-iosched.txt

Nope. I just tried noop and deadline (cfq is my default) and did not see
much of a difference.

	Wonko

Re: [gentoo-user] I want to play movies without hangs

[Dale]

Alex Schuster wrote:
<<< SNIP >>>
> Now this is really annoying. I watch small clips mostly, and can live
> with that, and when I want to watch stuff with others, I copy the file to
> tmpfs, which seems to help a lot.
> 
> But now I found another solution: NOT USING KDE.
> 
> When X crashed (trying to make the old Unreal game play), I fired up
> another window manager, and when I played a video in there, there was no
> problem at all. So, I have another workaround.
> 
> But does anyone have an idea, why running KDE is the problem? Disabling
> desktop effects does not help.
> 
> I must be totally crazy because I still want to use KDE, despite the big
> trouble it gives me nearly every day. Yes, most things work fine now, but
> there are many many little problems, daily application crashes, and every
> time I log in I fear that the desktop won't come up. 8G of RAM was not
> enough to avoid swapping, so now I have 16G, that's fine, I no longer
> care about kwin using 1G of my RAM. Oh, and I no longer use KMail, after
> it ate thousands of mails I just wanted to move. No problem, they were
> not important, but I no longer trust the KDEPIM suite. And it seems the
> developers do not care about this, the bug report got no replies.
> 
> But anyway. Any idea why it only happens with KDE? I will ask on the KDE
> mailing list, but I thought I post here first, maybe there's something
> Gentoo-specific going on here.
> 
> 	Wonko
> 
> 


When I first built this rig, I ran into this issue as well.  What I did
was tell smplayer, in my case, to cache more of the video.  I have mine
set to cache 6Mbs and it plays fine even on HD videos.

This may not help you but if you have not tried it yet, may be worth a
shot.  It is aggravating when it does this tho.

Also, there was a thread a good while back with this issue and their fix
was to do a emerge -e world with everything optimized for their CPU and
such.  May be worth thinking about at least.

Dale

:-)  :-)

-- 
I am only responsible for what I said ... Not for what you understood or
how you interpreted my words!

Miss the compile output?  Hint:
EMERGE_DEFAULT_OPTS="--quiet-build=n"

Re: [gentoo-user] I want to play movies without hangs

[Alex Schuster]

Dale writes:

> When I first built this rig, I ran into this issue as well.  What I did
> was tell smplayer, in my case, to cache more of the video.  I have mine
> set to cache 6Mbs and it plays fine even on HD videos.

I have cache = 131072 and cache-min=20.0 in .mplayer/config. That's
128MB, this should be enough. Got this hint by James Broadhead some
months ago in this thread.

> Also, there was a thread a good while back with this issue and their fix
> was to do a emerge -e world with everything optimized for their CPU and
> such.  May be worth thinking about at least.

That was Walter Dnes, also in thos thread. But everything should be quite
optimized here, with:
 CFLAGS="-pipe -march=amdfam10 -O2 \
         -floop-interchange -floop-strip-mine -floop-block \
         -msse -msse2 -msse3 -msse4a -msse4.1 -msse4.2 -m3dnow \
         -mcx16 -msahf -maes -mpclmul -mpopcnt -mabm -mlwp -mavx \
         --param l1-cache-size=16 \
         --param l1-cache-line-size=64 \
         --param l2-cache-size=2048"

And I do not have high CPU load when playing movies.

	Wonko

Re: [gentoo-user] I want to play movies without hangs

[Alex Schuster]

Mark Knecht writes:

> On Mon, May 7, 2012 at 5:41 AM, Alex Schuster <wonko@wonkology.org>
> wrote:
> > Some while ago, I wrote:
> >
> > [
> > mplayer stutters when I/O is going on, even hangs for seconds when I
> > do a dd if=/dev/zero of=somefile bs=1M
> > ]
[...]
>    OK, fire up two terminals. In one run top, hit 1 & z so you see all
> your CPUs and then watch CPU usage. In the second terminal su to root
> and run iotop -o. Now, watch for a few minutes and get a feel for
> what's going on when video is not running. Then start your video and
> watch IO usage and CPU usage. Where's the problem?
> 
>    Once you get an idea where the bottleneck is we can address what a
> solution might be. In general, if the CPUs aren't maxed out and it's
> an I/O problem then usually a bit more buffering is a simple solution.
> Other more draconian solution might be a real-time kernel with a
> player (if there is one) that is set up for real-time playback.
> 
>    Looking forward to hearing your test results.

Thanks for your support, Mark!

I did this already, but sometimes I do not notice anything. I guess it's
short I/O operations in that case. CPU load is not the problem, and it
happens for both high-quality videos and small ones. 
Currently iotop shows stuff like kjournald, kworker, kdeinit4,
akonadiserver, firefox. And lots of virtuoso-t and nepomuk when I enable
indexing again, which I just suspended.
And mplayer of course, it shows up in about every 2nd redisplay, which
happens every second.

Well... but when I do the same in the other window manager, it seems I
see fewer processes then. Are they mostly suspended when I am on another
display?
And I should fire up the same stuff (Firefox, Chromium, maybe KDEPIM
stuff) in the other WM and see if this makes things worse. But I'll do
this tomorrow. Thanks for the inspiration, though, at least I have
something more to try now.

The interrupts are very small normally, but noticeable, annoying and
somewhat embarrassing. When they just happened I only noticed akonadi and
kjournald during that time. I can force larger interrupts by doing my dd
command.

But anyway - my intention is not so much to find out what all these
I/O processes are and how to make them calm down, renice them or whatever.
Four cores @ 3.6 GHz just should be able to play movies without any
interruption. And it _is_ possible, when I start the playback on another
window manager, while KDE is still running on the other display.

I could just switch to, um, ummmm.... Gnome maybe... or Xfce4... or
something else, but I would not like to do so. Despite by big KDE
problems. I hate KDE. But I still want it. I feel mad.

	Wonko

Re: [gentoo-user] I want to play movies without hangs

[Stroller]

On 7 May 2012, at 21:43, Dale wrote:
> ...
> Also, there was a thread a good while back with this issue and their fix
> was to do a emerge -e world with everything optimized for their CPU and
> such.  May be worth thinking about at least.

I understood that issue as significantly different - in that case there was stuttering of demanding video with no other i/o occurring. In this case *any* video, even low-res standard-def video, is pausing when i/o load is applied. The previous case is CPU / GPU video decoding performance issue, I think this one is clearly not.

Stroller.

[gentoo-user] Re: I want to play movies without hangs

[walt]

On 05/07/2012 10:14 AM, Volker Armin Hemmann wrote:
> Am Montag, 7. Mai 2012, 18:44:05 schrieb Michael Hampicke:
>> Maybe changing the kernel io scheduler will help?
>>
>> # /usr/src/linux/Documentation/block/switching-sched.txt
>> # /usr/src/linux/Documentation/block/deadline-iosched.txt
>> # /usr/src/linux/Documentation/block/cfq-iosched.txt
> 
> nice idea - but that didn't help in the past. Why should it help now?
> 
> The question is - why does disk IO make the mouse jerky and delays keyboard 
> input? That is just idiotic.

Have you tested ck-sources for comparison?

Re: [gentoo-user] Re: I want to play movies without hangs

[Volker Armin Hemmann]

Am Montag, 7. Mai 2012, 15:56:20 schrieb walt:
> On 05/07/2012 10:14 AM, Volker Armin Hemmann wrote:
> > Am Montag, 7. Mai 2012, 18:44:05 schrieb Michael Hampicke:
> >> Maybe changing the kernel io scheduler will help?
> >> 
> >> # /usr/src/linux/Documentation/block/switching-sched.txt
> >> # /usr/src/linux/Documentation/block/deadline-iosched.txt
> >> # /usr/src/linux/Documentation/block/cfq-iosched.txt
> > 
> > nice idea - but that didn't help in the past. Why should it help now?
> > 
> > The question is - why does disk IO make the mouse jerky and delays
> > keyboard
> > input? That is just idiotic.
> 
> Have you tested ck-sources for comparison?

in the past, yes. Complete disappointment.

But it is ok, if the io happens between two very fast drives (ssd - ssd or ssd 
- raid5). So the problem is:
io with one slow drive being written to
swapping

those cases are just horrible.

And vanilla-sources just work. Stable. Even with suspend to ram and ati 
drivers.

-- 
#163933

[gentoo-user] Re: I want to play movies without hangs

[walt]

On 05/07/2012 04:13 PM, Volker Armin Hemmann wrote:

> And vanilla-sources just work. Stable. Even with suspend to ram and ati 
> drivers.

That confuses me.  Are you saying the gentoo-sources don't 'just work'?

Re: [gentoo-user] Re: I want to play movies without hangs

[Volker Armin Hemmann]

Am Montag, 7. Mai 2012, 16:56:43 schrieb walt:
> On 05/07/2012 04:13 PM, Volker Armin Hemmann wrote:
> > And vanilla-sources just work. Stable. Even with suspend to ram and ati
> > drivers.
> 
> That confuses me.  Are you saying the gentoo-sources don't 'just work'?

I don't know, haven't used gentoo sources in ages. I don't need fb 
beautification, zcache, alps or livecd goodies. I like to be as close to 
upstream as possible, so bug reports are taken seriously.

vanilla-sources work reliably with that annoying io-stuff. But no matter which 
kernel I tried, ck, zen, gentoo, love etc pp that annoying io-stuff was always 
there.. so no reason at all to use a patched up kernel.


-- 
#163933

Re: [gentoo-user] I want to play movies without hangs

[Simon]

> Also, there was a thread a good while back with this issue and their fix
> was to do a emerge -e world with everything optimized for their CPU and
> such.  May be worth thinking about at least.

Video playback and CPU optimisations go hand in hand.

One video I have had for a long time, I could never play it at all on
any of my computers and I could not beleive what I saw next...
I got a new PC and thought to give it a try.  It worked painfully,
skipping and using 100% of one of my CPU cores.  I was sad and thought
this video might have been corrupt. After a wave of cleanup, I had
removed all CPU optimisation flags from my make.conf (I was originally
negating many, like -sse, etc, so I left it "to my profile" to choose
what was needed) and recompiled everything.  After that, the same
movie played in the best quality I had ever seen and CPU usage was
below 5% !!!!  I was even able to open 12 movies (4x3) all playing at
the same time (this was unthinkable before I optimized my system).
Anyway, keep in mind that my particular starting scenario was that I
had "un-optimized" it long ago!

Good luck!

Re: [gentoo-user] I want to play movies without hangs

[Alex Schuster]

I wrote:

> Mark Knecht writes:

> >    OK, fire up two terminals. In one run top, hit 1 & z so you see all
> > your CPUs and then watch CPU usage. In the second terminal su to root
> > and run iotop -o. Now, watch for a few minutes and get a feel for
> > what's going on when video is not running. Then start your video and
> > watch IO usage and CPU usage. Where's the problem?
> > 
> >    Once you get an idea where the bottleneck is we can address what a
> > solution might be. In general, if the CPUs aren't maxed out and it's
> > an I/O problem then usually a bit more buffering is a simple solution.
> > Other more draconian solution might be a real-time kernel with a
> > player (if there is one) that is set up for real-time playback.
> > 
> >    Looking forward to hearing your test results.
> 
> Thanks for your support, Mark!
> 
> I did this already, but sometimes I do not notice anything. I guess it's
> short I/O operations in that case. CPU load is not the problem, and it
> happens for both high-quality videos and small ones. 
> Currently iotop shows stuff like kjournald, kworker, kdeinit4,
> akonadiserver, firefox. And lots of virtuoso-t and nepomuk when I enable
> indexing again, which I just suspended.
> And mplayer of course, it shows up in about every 2nd redisplay, which
> happens every second.
> 
> Well... but when I do the same in the other window manager, it seems I
> see fewer processes then. Are they mostly suspended when I am on another
> display?

I watched for longer now, and this does not seem to be true.

> And I should fire up the same stuff (Firefox, Chromium, maybe KDEPIM
> stuff) in the other WM and see if this makes things worse. But I'll do
> this tomorrow. Thanks for the inspiration, though, at least I have
> something more to try now.

I am running Enlightenment 0.16 in parallel now, with Firefox, Chromium,
Kontact, Claws, Liferea, Amarok (which is doing a lot of I/OP stuff at
the moment according to iotop), and Dolphin showing a large directory of
multimedia files wit thumbnails. But I don't see akonadi related processes
in iotop, that is unusual.
I did the dd command to create more I/O. No gaps in video display at all.

When I play the video from within KDE (running Konsoles, Konqueror,
Dolphin and a lot of plasma stuff), I have gaps, and when I do the dd
command, there are in the range of seconds. Even for some seconds after
I canceled the dd.

I also tried a fresh, unconfigured KDE session by another user. I've
already done that, and there were also gaps in video playback, although
it seems they were fewer. But this time, I was not able to reproduce
them. Huh?

I guess I could remove anything running on my KDE desktop one by one,
including plasmoids, and see if playback gets better. But not now, I
finally have to actually do some work.

	Wonko

Re: [gentoo-user] I want to play movies without hangs

[Neil Bothwick]

[-- Attachment #1: Type: text/plain, Size: 587 bytes --]

On Wed, 9 May 2012 21:44:19 +0200, Alex Schuster wrote:

> I guess I could remove anything running on my KDE desktop one by one,
> including plasmoids, and see if playback gets better. But not now, I
> finally have to actually do some work.

I recently experienced slowdowns and delays with KDE. It turned out I had
inadvertently disabled swap (I'd rearranged my partitions and not updated
fstab). As soon as I gave it some swap space the delays disappeared.


-- 
Neil Bothwick

Sarchasm : The gulf between the author of sarcastic wit and the person
who doesn't get it.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: [gentoo-user] I want to play movies without hangs

[Alex Schuster]

Neil Bothwick writes:

> On Wed, 9 May 2012 21:44:19 +0200, Alex Schuster wrote:
> 
> > I guess I could remove anything running on my KDE desktop one by one,
> > including plasmoids, and see if playback gets better. But not now, I
> > finally have to actually do some work.
> 
> I recently experienced slowdowns and delays with KDE. It turned out I
> had inadvertently disabled swap (I'd rearranged my partitions and not
> updated fstab). As soon as I gave it some swap space the delays
> disappeared.

There's plenty of swap space available. With 16 G of RAM it should not
be needed, but sometimes my load gets really really high, and when I can
use the system again, there is 2-3 G of swap usage. I haven't found out
yet what this is, it seems to happen when emerging things, maybe related
to having 5 G tmpfs for portage, but when it happened the last time only
100 M were being used.

	Wonko

Re: [gentoo-user] I want to play movies without hangs

[Dale]

Alex Schuster wrote:
> Neil Bothwick writes:
> 
>> On Wed, 9 May 2012 21:44:19 +0200, Alex Schuster wrote:
>>
>>> I guess I could remove anything running on my KDE desktop one by one,
>>> including plasmoids, and see if playback gets better. But not now, I
>>> finally have to actually do some work.
>>
>> I recently experienced slowdowns and delays with KDE. It turned out I
>> had inadvertently disabled swap (I'd rearranged my partitions and not
>> updated fstab). As soon as I gave it some swap space the delays
>> disappeared.
> 
> There's plenty of swap space available. With 16 G of RAM it should not
> be needed, but sometimes my load gets really really high, and when I can
> use the system again, there is 2-3 G of swap usage. I haven't found out
> yet what this is, it seems to happen when emerging things, maybe related
> to having 5 G tmpfs for portage, but when it happened the last time only
> 100 M were being used.
> 
> 	Wonko
> 
> 


Is there a way to find out what is using swap?  Maybe something related
to the video is on swap which at times can be slow, certainly slower
than ram.

I have always wondered how to find this out myself.

Dale

:-)  :-)

-- 
I am only responsible for what I said ... Not for what you understood or
how you interpreted my words!

Miss the compile output?  Hint:
EMERGE_DEFAULT_OPTS="--quiet-build=n"

Re: [gentoo-user] I want to play movies without hangs

[Adam Carter]

> Is there a way to find out what is using swap?  Maybe something related
> to the video is on swap which at times can be slow, certainly slower
> than ram.
>
> I have always wondered how to find this out myself.

Well the OS uses swap, i dont know if its possible to then tie that
directly to a process. You can find out if swap is being at all using
with vmstat; so= swap out, si=swap in.

For example, watch the following when you view the video

adam@proxy ~ $ vmstat -S M 3
procs -----------memory---------- ---swap-- -----io---- -system-- ----cpu----
 r  b   swpd   free   buff  cache   si   so    bi    bo   in   cs us sy id wa
 0  0      0   1290    379   6617    0    0    26    56  108  107  2  3 93  2
 0  0      0   1290    379   6617    0    0     1    15   87   91  0  0 100  0
 0  0      0   1290    379   6617    0    0     0     0   59   54  0  0 100  0
 0  0      0   1290    379   6617    0    0     0     7   72   73  0  0 100  0

Re: [gentoo-user] I want to play movies without hangs

[Adam Carter]

> There's plenty of swap space available. With 16 G of RAM it should not
> be needed, but sometimes my load gets really really high, and when I can
> use the system again, there is 2-3 G of swap usage. I haven't found out
> yet what this is, it seems to happen when emerging things, maybe related
> to having 5 G tmpfs for portage, but when it happened the last time only
> 100 M were being used.

Yeah so it wont be swap related. This sounds more like the desktop
responsiveness issue discussed a while back. It might be worth
googling that (main issue was fixed in later kernels)
http://www.phoronix.com/scan.php?page=article&item=linux_2637_video&num=1

There's other things that may help, ie
CONFIG_HZ_1000=y
CONFIG_HZ=1000

Re: [gentoo-user] I want to play movies without hangs

[Dale]

Adam Carter wrote:
>> Is there a way to find out what is using swap?  Maybe something related
>> to the video is on swap which at times can be slow, certainly slower
>> than ram.
>>
>> I have always wondered how to find this out myself.
> 
> Well the OS uses swap, i dont know if its possible to then tie that
> directly to a process. You can find out if swap is being at all using
> with vmstat; so= swap out, si=swap in.
> 
> For example, watch the following when you view the video
> 
> adam@proxy ~ $ vmstat -S M 3
> procs -----------memory---------- ---swap-- -----io---- -system-- ----cpu----
>  r  b   swpd   free   buff  cache   si   so    bi    bo   in   cs us sy id wa
>  0  0      0   1290    379   6617    0    0    26    56  108  107  2  3 93  2
>  0  0      0   1290    379   6617    0    0     1    15   87   91  0  0 100  0
>  0  0      0   1290    379   6617    0    0     0     0   59   54  0  0 100  0
>  0  0      0   1290    379   6617    0    0     0     7   72   73  0  0 100  0
> 
> 


I'm not the OP but posted a two part post that didn't quite come out
like I expected.  One for the OP to try to find out what was using swap,
just in case it mattered.  Two to ask how that is done in case he didn't
know and for me since I don't know either.  lol

I was hoping for a command that says program foo is using X amount of
swap but I guess not.

Interesting that there isn't a real good tool for this.

Dale

:-)  :-)

-- 
I am only responsible for what I said ... Not for what you understood or
how you interpreted my words!

Miss the compile output?  Hint:
EMERGE_DEFAULT_OPTS="--quiet-build=n"

Re: [gentoo-user] I want to play movies without hangs

[Michael Hampicke]

Am 10.05.2012 05:59, schrieb Adam Carter:
>> There's plenty of swap space available. With 16 G of RAM it should not
>> be needed, but sometimes my load gets really really high, and when I can
>> use the system again, there is 2-3 G of swap usage. I haven't found out
>> yet what this is, it seems to happen when emerging things, maybe related
>> to having 5 G tmpfs for portage, but when it happened the last time only
>> 100 M were being used.
> 
> Yeah so it wont be swap related. This sounds more like the desktop
> responsiveness issue discussed a while back. It might be worth
> googling that (main issue was fixed in later kernels)
> http://www.phoronix.com/scan.php?page=article&item=linux_2637_video&num=1
> 
> There's other things that may help, ie
> CONFIG_HZ_1000=y
> CONFIG_HZ=1000
> 

Does the CONFIG_HZ setting today really have that much of an impact? I
mean with tickles kernels and high res timers it should matter that
much, or am I wrong?
Playing video is not really a low-latency-multimedia-app, on the other
hand: you never really know :)

Re: [gentoo-user] I want to play movies without hangs

[Paul Hartman]

On Wed, May 9, 2012 at 8:00 PM, Alex Schuster <wonko@wonkology.org> wrote:
> Neil Bothwick writes:
>
>> On Wed, 9 May 2012 21:44:19 +0200, Alex Schuster wrote:
>>
>> > I guess I could remove anything running on my KDE desktop one by one,
>> > including plasmoids, and see if playback gets better. But not now, I
>> > finally have to actually do some work.
>>
>> I recently experienced slowdowns and delays with KDE. It turned out I
>> had inadvertently disabled swap (I'd rearranged my partitions and not
>> updated fstab). As soon as I gave it some swap space the delays
>> disappeared.
>
> There's plenty of swap space available. With 16 G of RAM it should not
> be needed, but sometimes my load gets really really high, and when I can
> use the system again, there is 2-3 G of swap usage. I haven't found out
> yet what this is, it seems to happen when emerging things, maybe related
> to having 5 G tmpfs for portage, but when it happened the last time only
> 100 M were being used.

Hi,

I realize this thread is bigger than an encyclopedia by now, so I
apologize if this has already been suggested. :)

I'm curious if you look at /proc/interrupts if the disk with I/O
problems is sharing interrupt with some other device. Maybe there is a
conflict of some sort.

On my motherboard, one of the SATA controllers shares an interrupt
with the soundcard, for example.

Re: [gentoo-user] I want to play movies without hangs

[Alex Schuster]

Paul Hartman writes:

> I realize this thread is bigger than an encyclopedia by now, so I
> apologize if this has already been suggested. :)

Well, I'm happy for any input on this :)  This problem is really annoying.

> I'm curious if you look at /proc/interrupts if the disk with I/O
> problems is sharing interrupt with some other device. Maybe there is a
> conflict of some sort.
> 
> On my motherboard, one of the SATA controllers shares an interrupt
> with the soundcard, for example.

Good point. But this does not seem to be the case. The only lines that
have multiple entries are two interrupts with ohci_hdc and snd_hda_intel, that should be no problem. The two ahci entries have their own
interrupts.

I did not yet find the time to remove the plasmoids (all is fine in
other window managers, or for another user with a plain unconfigured KDE
desktop), today I dealt with a horrible and weird qt upgrade experience on
another PC. Finally I was able to downgrade to 4.7, and at least stuff is
working again there.

	Wonko

Re: [gentoo-user] I want to play movies without hangs [SOLVED, sort of]

[Alex Schuster]

Finally, I found something. It's Dolphin!

I've done some longer testing, always playing the same video parallel
with a dd if=/dev/zero of=/tmp/10G bs=1M count=10000, in mplayer, for a
minute, several times. When I do this by opening the file in Dolphin, I
get about 15 interruptions, some for longer than a second. Started on the
command line, there are very few, I can play the video for minutes
without a gap. Hooray!

In KDE, I usually play videos by opening them in Dolphin. I exchanged
'mplayer %U' by 'xterm -T MPLAYER -e mplayer %U' in the settings, now
mplayer runs in a terminal, and all is fine. I created a window rule so
the terminal automatically minimizes. Cool!

It only happens in mplayer and mplayer2. Other players work fine, but I
like mplayer best, and prefer to run it without any window decoration.

Now, would this be an MPlayer problem, or one of Dolphin?

	Wonko

Re: [gentoo-user] I want to play movies without hangs [SOLVED, sort of]

[Norman Invasion]

On 11 May 2012 21:40, Alex Schuster <wonko@wonkology.org> wrote:
> Finally, I found something. It's Dolphin!
>
> I've done some longer testing, always playing the same video parallel
> with a dd if=/dev/zero of=/tmp/10G bs=1M count=10000, in mplayer, for a
> minute, several times. When I do this by opening the file in Dolphin, I
> get about 15 interruptions, some for longer than a second. Started on the
> command line, there are very few, I can play the video for minutes
> without a gap. Hooray!
>
> In KDE, I usually play videos by opening them in Dolphin. I exchanged
> 'mplayer %U' by 'xterm -T MPLAYER -e mplayer %U' in the settings, now
> mplayer runs in a terminal, and all is fine. I created a window rule so
> the terminal automatically minimizes. Cool!
>
> It only happens in mplayer and mplayer2. Other players work fine, but I
> like mplayer best, and prefer to run it without any window decoration.
>
> Now, would this be an MPlayer problem, or one of Dolphin?
>

Apologies: I haven't followed this thread from the beginning,
but do you have any advanced power management features
enabled (especially hard drive related)?
When I pull the power cord on my lap-top, it goes into all kinds
of nutty "power-saving" and mplayer has long pauses while
the drive spins back up.

Re: [gentoo-user] I want to play movies without hangs [SOLVED, sort of]

[Alex Schuster]

Norman Invasion writes:

> On 11 May 2012 21:40, Alex Schuster <wonko@wonkology.org> wrote:
> > Finally, I found something. It's Dolphin!
[...]
> Apologies: I haven't followed this thread from the beginning,

Which was quite long ago :)

> but do you have any advanced power management features
> enabled (especially hard drive related)?

My drives spin down after 30 minutes of idle time, but this never happens
for the system drive. The CPU is set to throttle down from 3600 MHz to
1400 MHz with the ondemand governor, but changing to performance governor
makes no change.

> When I pull the power cord on my lap-top, it goes into all kinds
> of nutty "power-saving" and mplayer has long pauses while
> the drive spins back up.

Yeah, but those pauses are much longer than the small interruptions that
are a fraction of a second mostly, and do not happen 15 times per minute.
And it only happens when MPlayer is started from Dolphin. Well, mainly,
when there is much system load, I also had small interruptions when I run
mplayer from the command line, but they are much much less frequent, and
do not happen under normal circumstances, like when doing emerges while
playing videos.

	Wonko

Re: [gentoo-user] I want to play movies without hangs [SOLVED, sort of]

[Norman Invasion]

On 12 May 2012 11:05, Alex Schuster <wonko@wonkology.org> wrote:
> Norman Invasion writes:
>
>> On 11 May 2012 21:40, Alex Schuster <wonko@wonkology.org> wrote:
>> > Finally, I found something. It's Dolphin!
> [...]
>> Apologies: I haven't followed this thread from the beginning,
>
> Which was quite long ago :)
>
>> but do you have any advanced power management features
>> enabled (especially hard drive related)?
>
> My drives spin down after 30 minutes of idle time, but this never happens
> for the system drive. The CPU is set to throttle down from 3600 MHz to
> 1400 MHz with the ondemand governor, but changing to performance governor
> makes no change.
>
>> When I pull the power cord on my lap-top, it goes into all kinds
>> of nutty "power-saving" and mplayer has long pauses while
>> the drive spins back up.
>
> Yeah, but those pauses are much longer than the small interruptions that
> are a fraction of a second mostly, and do not happen 15 times per minute.
> And it only happens when MPlayer is started from Dolphin. Well, mainly,
> when there is much system load, I also had small interruptions when I run
> mplayer from the command line, but they are much much less frequent, and
> do not happen under normal circumstances, like when doing emerges while
> playing videos.
>

I'm just recalling that I get stuttering audio in freebsd, which is caused
by what-I-don't-know, but only happens when the CPU load is low.
Firing up burncpu or doing useless recompiles ameliorates it.

Re: [gentoo-user] I want to play movies without hangs

[Alex Schuster]

Dale writes:

> Is there a way to find out what is using swap?  Maybe something related
> to the video is on swap which at times can be slow, certainly slower
> than ram.
> 
> I have always wondered how to find this out myself.

Me too, so when I had this sudden swap problem for the first time, I
searched for a method to do this and found a script here:
http://northernmost.org/blog/find-out-what-is-using-your-swap/

There's lots of information for all processes in /proc/<pid>/. Trying to
read /proc/<pid>/mem (I think it was this file) in mc was not such a good
idea, the system froze with lots of HD activity, and after half an hour I
rebooted with Alt-SysRq-{K,E,I,S,U,B}.

I improved the script a little, it allows sorting by PID, size and name,
and can restrict the output to specific processes or show only those
using more swap than specified. If interested you can download it here:
http://www.wonkology.org/utils/getswap
You need to be root to see processes you do not own.

But of course, I forgot to run it after the sudden swap problem happened
lately. So I still do not know what was going on there. I'll wait for the
next time it happens.

	Wonko

Re: [gentoo-user] I want to play movies without hangs

[Michael Mol]

On Sat, May 12, 2012 at 8:34 PM, Alex Schuster <wonko@wonkology.org> wrote:
> Dale writes:
>
>> Is there a way to find out what is using swap?  Maybe something related
>> to the video is on swap which at times can be slow, certainly slower
>> than ram.
>>
>> I have always wondered how to find this out myself.
>
> Me too, so when I had this sudden swap problem for the first time, I
> searched for a method to do this and found a script here:
> http://northernmost.org/blog/find-out-what-is-using-your-swap/
>
> There's lots of information for all processes in /proc/<pid>/. Trying to
> read /proc/<pid>/mem (I think it was this file) in mc was not such a good
> idea, the system froze with lots of HD activity, and after half an hour I
> rebooted with Alt-SysRq-{K,E,I,S,U,B}.
>
> I improved the script a little, it allows sorting by PID, size and name,
> and can restrict the output to specific processes or show only those
> using more swap than specified. If interested you can download it here:
> http://www.wonkology.org/utils/getswap
> You need to be root to see processes you do not own.
>
> But of course, I forgot to run it after the sudden swap problem happened
> lately. So I still do not know what was going on there. I'll wait for the
> next time it happens.
>
>        Wonko
>

sys-process/htop

-- 
:wq

Re: [gentoo-user] I want to play movies without hangs

[Alex Schuster]

Michael Mol writes:

> On Sat, May 12, 2012 at 8:34 PM, Alex Schuster <wonko@wonkology.org>
> wrote:
> > Dale writes:
> >
> >> Is there a way to find out what is using swap?  Maybe something
> >> related to the video is on swap which at times can be slow,
> >> certainly slower than ram.
> >>
> >> I have always wondered how to find this out myself.
> >
> > Me too, so when I had this sudden swap problem for the first time, I
> > searched for a method to do this and found a script here:
> > http://northernmost.org/blog/find-out-what-is-using-your-swap/
> >
> > There's lots of information for all processes in /proc/<pid>/. Trying
> > to read /proc/<pid>/mem (I think it was this file) in mc was not such
> > a good idea, the system froze with lots of HD activity, and after
> > half an hour I rebooted with Alt-SysRq-{K,E,I,S,U,B}.
> >
> > I improved the script a little, it allows sorting by PID, size and
> > name, and can restrict the output to specific processes or show only
> > those using more swap than specified. If interested you can download
> > it here: http://www.wonkology.org/utils/getswap
> > You need to be root to see processes you do not own.
> >
> > But of course, I forgot to run it after the sudden swap problem
> > happened lately. So I still do not know what was going on there. I'll
> > wait for the next time it happens.
> >
> >        Wonko
> >
> 
> sys-process/htop

Huh? I only see the total amount of swap being used, but no entry per
process.

	Wonko

Re: [gentoo-user] I want to play movies without hangs

[Michael Mol]

On Sat, May 12, 2012 at 8:58 PM, Alex Schuster <wonko@wonkology.org> wrote:
> Michael Mol writes:
>
>> On Sat, May 12, 2012 at 8:34 PM, Alex Schuster <wonko@wonkology.org>
>> wrote:
>> > Dale writes:
>> >
>> >> Is there a way to find out what is using swap?  Maybe something
>> >> related to the video is on swap which at times can be slow,
>> >> certainly slower than ram.
>> >>
>> >> I have always wondered how to find this out myself.
>> >
>> > Me too, so when I had this sudden swap problem for the first time, I
>> > searched for a method to do this and found a script here:
>> > http://northernmost.org/blog/find-out-what-is-using-your-swap/
>> >
>> > There's lots of information for all processes in /proc/<pid>/. Trying
>> > to read /proc/<pid>/mem (I think it was this file) in mc was not such
>> > a good idea, the system froze with lots of HD activity, and after
>> > half an hour I rebooted with Alt-SysRq-{K,E,I,S,U,B}.
>> >
>> > I improved the script a little, it allows sorting by PID, size and
>> > name, and can restrict the output to specific processes or show only
>> > those using more swap than specified. If interested you can download
>> > it here: http://www.wonkology.org/utils/getswap
>> > You need to be root to see processes you do not own.
>> >
>> > But of course, I forgot to run it after the sudden swap problem
>> > happened lately. So I still do not know what was going on there. I'll
>> > wait for the next time it happens.
>> >
>> >        Wonko
>> >
>>
>> sys-process/htop
>
> Huh? I only see the total amount of swap being used, but no entry per
> process.

Hit F2, and go down to 'columns'. Anything per-process found under
/proc can be added as a column.

-- 
:wq

Re: [gentoo-user] I want to play movies without hangs [SOLVED, sort of]

[Alan McKinnon]

On Sat, 12 May 2012 11:41:33 -0400
Norman Invasion <invasivenorman@gmail.com> wrote:

> On 12 May 2012 11:05, Alex Schuster <wonko@wonkology.org> wrote:
> > Norman Invasion writes:
> >
> >> On 11 May 2012 21:40, Alex Schuster <wonko@wonkology.org> wrote:
> >> > Finally, I found something. It's Dolphin!
> > [...]
> >> Apologies: I haven't followed this thread from the beginning,
> >
> > Which was quite long ago :)
> >
> >> but do you have any advanced power management features
> >> enabled (especially hard drive related)?
> >
> > My drives spin down after 30 minutes of idle time, but this never
> > happens for the system drive. The CPU is set to throttle down from
> > 3600 MHz to 1400 MHz with the ondemand governor, but changing to
> > performance governor makes no change.
> >
> >> When I pull the power cord on my lap-top, it goes into all kinds
> >> of nutty "power-saving" and mplayer has long pauses while
> >> the drive spins back up.
> >
> > Yeah, but those pauses are much longer than the small interruptions
> > that are a fraction of a second mostly, and do not happen 15 times
> > per minute. And it only happens when MPlayer is started from
> > Dolphin. Well, mainly, when there is much system load, I also had
> > small interruptions when I run mplayer from the command line, but
> > they are much much less frequent, and do not happen under normal
> > circumstances, like when doing emerges while playing videos.
> >
> 
> I'm just recalling that I get stuttering audio in freebsd, which is
> caused by what-I-don't-know, but only happens when the CPU load is
> low. Firing up burncpu or doing useless recompiles ameliorates it.
> 

I was getting stuttering audio from a sizeable % of my .avi files
served from a FreeBSD NAS. The likely cause became obvious when I
noticed that it was only on .avi files - all real containers were
fine[1].

mencoder -ovc copy -oac copy -of avi -o <new_file> <old_file>

fixed it permanently. I'm won't go so far as to say this might apply
to your issue, but sometimes the simplest things are the actual
causes :-)


[1] .avi files are notorious for this shit. It's what happens when you
are Microsoft and you release any old crappy format without consulting
the other experts out there (who will always outnumber you)


-- 
Alan McKinnnon
alan.mckinnon@gmail.com

Re: [gentoo-user] I want to play movies without hangs [SOLVED, sort of]

[Michael Mol]

On Sun, May 13, 2012 at 4:56 AM, Alan McKinnon <alan.mckinnon@gmail.com> wrote:
> [1] .avi files are notorious for this shit. It's what happens when you
> are Microsoft and you release any old crappy format without consulting
> the other experts out there (who will always outnumber you)

Which better container formats were available at the time AVI was
released (1992)? The only contemporary container format I'm aware of
is RIFF, which came out in 1988. MPEG-1 didn't come out until 1993,
which was the same year the Ogg project started. Real's stuff didn't
come out until 1995. Matroska was announced a decade later, in 2005.

Matroska, MP4 and even OGG are nicer container formats, sure, but they
weren't around yet. And even with any of them, it's perfectly possible
to accidentally get A/V desync or stuttering if you don't mux your
streams properly.

(This post draws heavily on Wikipedia for date information, and dates
may be considered only as accurate as Wikipedia...)

-- 
:wq

Re: [gentoo-user] I want to play movies without hangs [SOLVED, sort of]

[Alan McKinnon]

On Sun, 13 May 2012 14:12:04 -0400
Michael Mol <mikemol@gmail.com> wrote:

> On Sun, May 13, 2012 at 4:56 AM, Alan McKinnon
> <alan.mckinnon@gmail.com> wrote:
> > [1] .avi files are notorious for this shit. It's what happens when
> > you are Microsoft and you release any old crappy format without
> > consulting the other experts out there (who will always outnumber
> > you)
> 
> Which better container formats were available at the time AVI was
> released (1992)? The only contemporary container format I'm aware of
> is RIFF, which came out in 1988. MPEG-1 didn't come out until 1993,
> which was the same year the Ogg project started. Real's stuff didn't
> come out until 1995. Matroska was announced a decade later, in 2005.
> 
> Matroska, MP4 and even OGG are nicer container formats, sure, but they
> weren't around yet. And even with any of them, it's perfectly possible
> to accidentally get A/V desync or stuttering if you don't mux your
> streams properly.
> 
> (This post draws heavily on Wikipedia for date information, and dates
> may be considered only as accurate as Wikipedia...)
> 

You missed the essence of my post entirely.

-- 
Alan McKinnnon
alan.mckinnon@gmail.com

Re: [gentoo-user] I want to play movies without hangs [SOLVED, sort of]

[Michael Mol]

On Sun, May 13, 2012 at 4:53 PM, Alan McKinnon <alan.mckinnon@gmail.com> wrote:
> On Sun, 13 May 2012 14:12:04 -0400
> Michael Mol <mikemol@gmail.com> wrote:
>
>> On Sun, May 13, 2012 at 4:56 AM, Alan McKinnon
>> <alan.mckinnon@gmail.com> wrote:
>> > [1] .avi files are notorious for this shit. It's what happens when
>> > you are Microsoft and you release any old crappy format without
>> > consulting the other experts out there (who will always outnumber
>> > you)
>>
>> Which better container formats were available at the time AVI was
>> released (1992)? The only contemporary container format I'm aware of
>> is RIFF, which came out in 1988. MPEG-1 didn't come out until 1993,
>> which was the same year the Ogg project started. Real's stuff didn't
>> come out until 1995. Matroska was announced a decade later, in 2005.
>>
>> Matroska, MP4 and even OGG are nicer container formats, sure, but they
>> weren't around yet. And even with any of them, it's perfectly possible
>> to accidentally get A/V desync or stuttering if you don't mux your
>> streams properly.
>>
>> (This post draws heavily on Wikipedia for date information, and dates
>> may be considered only as accurate as Wikipedia...)
>>
>
> You missed the essence of my post entirely.

Anti-Microsoft snark? I thought I was calling you on it.

-- 
:wq

Re: [gentoo-user] I want to play movies without hangs [SOLVED, sort of]

[Alan McKinnon]

On Sun, 13 May 2012 17:01:07 -0400
Michael Mol <mikemol@gmail.com> wrote:

> On Sun, May 13, 2012 at 4:53 PM, Alan McKinnon
> <alan.mckinnon@gmail.com> wrote:
> > On Sun, 13 May 2012 14:12:04 -0400
> > Michael Mol <mikemol@gmail.com> wrote:
> >
> >> On Sun, May 13, 2012 at 4:56 AM, Alan McKinnon
> >> <alan.mckinnon@gmail.com> wrote:
> >> > [1] .avi files are notorious for this shit. It's what happens
> >> > when you are Microsoft and you release any old crappy format
> >> > without consulting the other experts out there (who will always
> >> > outnumber you)
> >>
> >> Which better container formats were available at the time AVI was
> >> released (1992)? The only contemporary container format I'm aware
> >> of is RIFF, which came out in 1988. MPEG-1 didn't come out until
> >> 1993, which was the same year the Ogg project started. Real's
> >> stuff didn't come out until 1995. Matroska was announced a decade
> >> later, in 2005.
> >>
> >> Matroska, MP4 and even OGG are nicer container formats, sure, but
> >> they weren't around yet. And even with any of them, it's perfectly
> >> possible to accidentally get A/V desync or stuttering if you don't
> >> mux your streams properly.
> >>
> >> (This post draws heavily on Wikipedia for date information, and
> >> dates may be considered only as accurate as Wikipedia...)
> >>
> >
> > You missed the essence of my post entirely.
> 
> Anti-Microsoft snark? I thought I was calling you on it.
> 

I said .avi is a crappy format, and it is, that much is obvious to
anyone who understands the simple basics of what a container should do.
It would have been obvious to the .avi developers then. And yet it
somehow made it's way to market and got used extensively

You asked what alternatives were available. That is not a question I
asked. It matters nothing that the public used .avi so much (they had
precious little in the way of choice). So whether they had
alternatives or not is irrelevant.

The entire gist of my post was about how .avi as it stands is crappy
and should never have been released by an entity with the engineering
clout of Microsoft as they don't have the excuse of being one dude in
Mom's basement who didn't know better. They really should have known
better.


-- 
Alan McKinnnon
alan.mckinnon@gmail.com

Re: [gentoo-user] I want to play movies without hangs [SOLVED, sort of]

[Michael Mol]

On Sun, May 13, 2012 at 5:33 PM, Alan McKinnon <alan.mckinnon@gmail.com> wrote:
> On Sun, 13 May 2012 17:01:07 -0400
> Michael Mol <mikemol@gmail.com> wrote:
>
>> On Sun, May 13, 2012 at 4:53 PM, Alan McKinnon
>> <alan.mckinnon@gmail.com> wrote:
>> > On Sun, 13 May 2012 14:12:04 -0400
>> > Michael Mol <mikemol@gmail.com> wrote:
>> >
>> >> On Sun, May 13, 2012 at 4:56 AM, Alan McKinnon
>> >> <alan.mckinnon@gmail.com> wrote:
>> >> > [1] .avi files are notorious for this shit. It's what happens
>> >> > when you are Microsoft and you release any old crappy format
>> >> > without consulting the other experts out there (who will always
>> >> > outnumber you)
>> >>
>> >> Which better container formats were available at the time AVI was
>> >> released (1992)? The only contemporary container format I'm aware
>> >> of is RIFF, which came out in 1988. MPEG-1 didn't come out until
>> >> 1993, which was the same year the Ogg project started. Real's
>> >> stuff didn't come out until 1995. Matroska was announced a decade
>> >> later, in 2005.
>> >>
>> >> Matroska, MP4 and even OGG are nicer container formats, sure, but
>> >> they weren't around yet. And even with any of them, it's perfectly
>> >> possible to accidentally get A/V desync or stuttering if you don't
>> >> mux your streams properly.
>> >>
>> >> (This post draws heavily on Wikipedia for date information, and
>> >> dates may be considered only as accurate as Wikipedia...)
>> >>
>> >
>> > You missed the essence of my post entirely.
>>
>> Anti-Microsoft snark? I thought I was calling you on it.
>>
>
> I said .avi is a crappy format, and it is, that much is obvious to
> anyone who understands the simple basics of what a container should do.

The MPEG group had only been formed four years prior to AVI's release,
and didn't release their first standard until a year later. Meanwhile,
Microsoft needed a video file format that:

1) Was a file format that sat on disk
2) Synchronized audio and video
3) Integrated cleanly with their being-developed operating system (AVI
is very closely related to the Video for Windows API. It's worth
noting that WMF, another Microsoft format from this time, is
essentially a serialized form of their drawing primitives.)
4) Ran smoothly on an 80386 at 33MHz with a 16-bit, 8MHz data bus
between the CPU and persistent storage.

With the exception of perhaps (3), those are the "basics." Consider
that this was released in 1992, and then consider that it had probably
been under development for at least a couple years prior.

I won't disagree that AVI is a crappy format by today's standards, and
that it should be avoided where possible, but what you consider simple
and obvious today was *new* at the time, and so not simple and
obvious.

> It would have been obvious to the .avi developers then. And yet it
> somehow made it's way to market and got used extensively
>
> You asked what alternatives were available. That is not a question I
> asked. It matters nothing that the public used .avi so much (they had
> precious little in the way of choice). So whether they had
> alternatives or not is irrelevant.

It's entirely relevant if you want to consider whether not the
expertise to come up with a 2012-modern format *existed* in the
lead-up time to 1992.

>
> The entire gist of my post was about how .avi as it stands is crappy
> and should never have been released by an entity with the engineering
> clout of Microsoft as they don't have the excuse of being one dude in
> Mom's basement who didn't know better. They really should have known
> better.

Seriously, why? Why do you think that the entire engineering clout of
a company which hadn't yet taken over the desktop market(!) would be
focused on perfecting AVI, one piece of a large,
already-late-to-market product? They had a bunch of difficult things
to pay attention to, such as mixing protected-mode and real-mode
applications on hardware in a task-switching environment, and working
around compatibility for programs whose developers still assumed they
had full run of the system. On a 386.

-- 
:wq

Re: [gentoo-user] I want to play movies without hangs [SOLVED, sort of]

[Alan McKinnon]

On Sun, 13 May 2012 18:03:59 -0400
Michael Mol <mikemol@gmail.com> wrote:

> On Sun, May 13, 2012 at 5:33 PM, Alan McKinnon
> <alan.mckinnon@gmail.com> wrote:
> > On Sun, 13 May 2012 17:01:07 -0400
> > Michael Mol <mikemol@gmail.com> wrote:
> >
> >> On Sun, May 13, 2012 at 4:53 PM, Alan McKinnon
> >> <alan.mckinnon@gmail.com> wrote:
> >> > On Sun, 13 May 2012 14:12:04 -0400
> >> > Michael Mol <mikemol@gmail.com> wrote:
> >> >
> >> >> On Sun, May 13, 2012 at 4:56 AM, Alan McKinnon
> >> >> <alan.mckinnon@gmail.com> wrote:
> >> >> > [1] .avi files are notorious for this shit. It's what happens
> >> >> > when you are Microsoft and you release any old crappy format
> >> >> > without consulting the other experts out there (who will
> >> >> > always outnumber you)
> >> >>
> >> >> Which better container formats were available at the time AVI
> >> >> was released (1992)? The only contemporary container format I'm
> >> >> aware of is RIFF, which came out in 1988. MPEG-1 didn't come
> >> >> out until 1993, which was the same year the Ogg project
> >> >> started. Real's stuff didn't come out until 1995. Matroska was
> >> >> announced a decade later, in 2005.
> >> >>
> >> >> Matroska, MP4 and even OGG are nicer container formats, sure,
> >> >> but they weren't around yet. And even with any of them, it's
> >> >> perfectly possible to accidentally get A/V desync or stuttering
> >> >> if you don't mux your streams properly.
> >> >>
> >> >> (This post draws heavily on Wikipedia for date information, and
> >> >> dates may be considered only as accurate as Wikipedia...)
> >> >>
> >> >
> >> > You missed the essence of my post entirely.
> >>
> >> Anti-Microsoft snark? I thought I was calling you on it.
> >>
> >
> > I said .avi is a crappy format, and it is, that much is obvious to
> > anyone who understands the simple basics of what a container should
> > do.
> 
> The MPEG group had only been formed four years prior to AVI's release,
> and didn't release their first standard until a year later. Meanwhile,
> Microsoft needed a video file format that:
> 
> 1) Was a file format that sat on disk
> 2) Synchronized audio and video


This is the part they got wrong.

Would you not agree that this is the second-most important feature
required, where the ability to actually play the audio/video at all is
the first?

Getting that wrong is to me akin to building a car and forgetting to
provide it with an adequate means of stopping. There are many other
things that can be forgiven where one would need a predictive crystal
ball, but needing time sync information in the container is just simply
self-evident.




> 3) Integrated cleanly with their being-developed operating system (AVI
> is very closely related to the Video for Windows API. It's worth
> noting that WMF, another Microsoft format from this time, is
> essentially a serialized form of their drawing primitives.)
> 4) Ran smoothly on an 80386 at 33MHz with a 16-bit, 8MHz data bus
> between the CPU and persistent storage.
> 
> With the exception of perhaps (3), those are the "basics." Consider
> that this was released in 1992, and then consider that it had probably
> been under development for at least a couple years prior.
> 
> I won't disagree that AVI is a crappy format by today's standards, and
> that it should be avoided where possible, but what you consider simple
> and obvious today was *new* at the time, and so not simple and
> obvious.

I'm not talking about today's standards. I'm talking about 1992
standards.

It's not reasonable to expect MS devs to anticipate algorithms that did
not exist then, or hardware that was 10 years away, or even that the
internet would be what it is. I do expect devs to get right aspects of
their software that will be used right at the time it is released.

> 
> > It would have been obvious to the .avi developers then. And yet it
> > somehow made it's way to market and got used extensively
> >
> > You asked what alternatives were available. That is not a question I
> > asked. It matters nothing that the public used .avi so much (they
> > had precious little in the way of choice). So whether they had
> > alternatives or not is irrelevant.
> 
> It's entirely relevant if you want to consider whether not the
> expertise to come up with a 2012-modern format *existed* in the
> lead-up time to 1992.

Again, I'm not talking about 2012

> 
> >
> > The entire gist of my post was about how .avi as it stands is crappy
> > and should never have been released by an entity with the
> > engineering clout of Microsoft as they don't have the excuse of
> > being one dude in Mom's basement who didn't know better. They
> > really should have known better.
> 
> Seriously, why? Why do you think that the entire engineering clout of
> a company which hadn't yet taken over the desktop market(!) would be
> focused on perfecting AVI, one piece of a large,
> already-late-to-market product? They had a bunch of difficult things
> to pay attention to, such as mixing protected-mode and real-mode
> applications on hardware in a task-switching environment, and working
> around compatibility for programs whose developers still assumed they
> had full run of the system. On a 386.
> 

No, I expect them to get the basics right. Cars and brakes.

-- 
Alan McKinnnon
alan.mckinnon@gmail.com

Re: [gentoo-user] I want to play movies without hangs [SOLVED, sort of]

[Michael Mol]

On Sun, May 13, 2012 at 7:27 PM, Alan McKinnon <alan.mckinnon@gmail.com> wrote:
> On Sun, 13 May 2012 18:03:59 -0400
> Michael Mol <mikemol@gmail.com> wrote:
>
>> On Sun, May 13, 2012 at 5:33 PM, Alan McKinnon
>> <alan.mckinnon@gmail.com> wrote:
>> > On Sun, 13 May 2012 17:01:07 -0400
>> > Michael Mol <mikemol@gmail.com> wrote:
>> >
>> >> On Sun, May 13, 2012 at 4:53 PM, Alan McKinnon
>> >> <alan.mckinnon@gmail.com> wrote:
>> >> > On Sun, 13 May 2012 14:12:04 -0400
>> >> > Michael Mol <mikemol@gmail.com> wrote:
>> >> >
>> >> >> On Sun, May 13, 2012 at 4:56 AM, Alan McKinnon
>> >> >> <alan.mckinnon@gmail.com> wrote:
>> >> >> > [1] .avi files are notorious for this shit. It's what happens
>> >> >> > when you are Microsoft and you release any old crappy format
>> >> >> > without consulting the other experts out there (who will
>> >> >> > always outnumber you)
>> >> >>
>> >> >> Which better container formats were available at the time AVI
>> >> >> was released (1992)? The only contemporary container format I'm
>> >> >> aware of is RIFF, which came out in 1988. MPEG-1 didn't come
>> >> >> out until 1993, which was the same year the Ogg project
>> >> >> started. Real's stuff didn't come out until 1995. Matroska was
>> >> >> announced a decade later, in 2005.
>> >> >>
>> >> >> Matroska, MP4 and even OGG are nicer container formats, sure,
>> >> >> but they weren't around yet. And even with any of them, it's
>> >> >> perfectly possible to accidentally get A/V desync or stuttering
>> >> >> if you don't mux your streams properly.
>> >> >>
>> >> >> (This post draws heavily on Wikipedia for date information, and
>> >> >> dates may be considered only as accurate as Wikipedia...)
>> >> >>
>> >> >
>> >> > You missed the essence of my post entirely.
>> >>
>> >> Anti-Microsoft snark? I thought I was calling you on it.
>> >>
>> >
>> > I said .avi is a crappy format, and it is, that much is obvious to
>> > anyone who understands the simple basics of what a container should
>> > do.
>>
>> The MPEG group had only been formed four years prior to AVI's release,
>> and didn't release their first standard until a year later. Meanwhile,
>> Microsoft needed a video file format that:
>>
>> 1) Was a file format that sat on disk
>> 2) Synchronized audio and video
>
>
> This is the part they got wrong.
>
> Would you not agree that this is the second-most important feature
> required, where the ability to actually play the audio/video at all is
> the first?

You're going to have to go into detail. Last I checked, old versions
of Windows shipped with AVI files for their animations, and those AVI
files played fine. So it _sounds_ like they're able to play video, at
least.

And my largish collection of AMVs and videos I've put together myself
suggest that AVI can play synchronized audio and video.

> Getting that wrong is to me akin to building a car and forgetting to
> provide it with an adequate means of stopping. There are many other
> things that can be forgiven where one would need a predictive crystal
> ball, but needing time sync information in the container is just simply
> self-evident.

Only if you anticipate your audio and video streams deviating from
intended usages. AVI is used for far more things than it was designed
to do. Reading deeper into its history, it sounds like it was embraced
and extended by entities outside of Microsoft to do things it wasn't
designed for in the first place. So expecting it to handle VBR audio
or video with predictive frames is kinda like putting a supercharger
in a Pinto and complaining when it winds up sitting on its own roof.

>
>
>
>
>> 3) Integrated cleanly with their being-developed operating system (AVI
>> is very closely related to the Video for Windows API. It's worth
>> noting that WMF, another Microsoft format from this time, is
>> essentially a serialized form of their drawing primitives.)
>> 4) Ran smoothly on an 80386 at 33MHz with a 16-bit, 8MHz data bus
>> between the CPU and persistent storage.
>>
>> With the exception of perhaps (3), those are the "basics." Consider
>> that this was released in 1992, and then consider that it had probably
>> been under development for at least a couple years prior.
>>
>> I won't disagree that AVI is a crappy format by today's standards, and
>> that it should be avoided where possible, but what you consider simple
>> and obvious today was *new* at the time, and so not simple and
>> obvious.
>
> I'm not talking about today's standards. I'm talking about 1992
> standards.

_Those standards didn't exist._ That's been my key point.

Yes, there was SMPTE, but that's for video recording and production
houses, and that was certainly not a planned usage for AVI.

>
> It's not reasonable to expect MS devs to anticipate algorithms that did
> not exist then, or hardware that was 10 years away, or even that the
> internet would be what it is. I do expect devs to get right aspects of
> their software that will be used right at the time it is released.

The earliest AVI files I'm aware of were sequences of RLE bitmaps, and
the code doing playback knew *exactly* what the framerate was, because
it knew what the video was for. Framerate support was added by
external parties because external parties wanted to extend AVI for
their own purposes. For that matter, AVI was an extension of RIFF.

>
>>
>> > It would have been obvious to the .avi developers then. And yet it
>> > somehow made it's way to market and got used extensively
>> >
>> > You asked what alternatives were available. That is not a question I
>> > asked. It matters nothing that the public used .avi so much (they
>> > had precious little in the way of choice). So whether they had
>> > alternatives or not is irrelevant.
>>
>> It's entirely relevant if you want to consider whether not the
>> expertise to come up with a 2012-modern format *existed* in the
>> lead-up time to 1992.
>
> Again, I'm not talking about 2012

No, but you're talking from the perspective of 2012, with a 20-year hindsight.

>
>>
>> >
>> > The entire gist of my post was about how .avi as it stands is crappy
>> > and should never have been released by an entity with the
>> > engineering clout of Microsoft as they don't have the excuse of
>> > being one dude in Mom's basement who didn't know better. They
>> > really should have known better.
>>
>> Seriously, why? Why do you think that the entire engineering clout of
>> a company which hadn't yet taken over the desktop market(!) would be
>> focused on perfecting AVI, one piece of a large,
>> already-late-to-market product? They had a bunch of difficult things
>> to pay attention to, such as mixing protected-mode and real-mode
>> applications on hardware in a task-switching environment, and working
>> around compatibility for programs whose developers still assumed they
>> had full run of the system. On a 386.
>>
>
> No, I expect them to get the basics right. Cars and brakes.

Pintos and superchargers. AVI does far more than what it was designed
for, and even what you want it to have been designed to do wasn't even
in the forecast in 1992. In 1992, the biggest and most interesting
applications were still conceptual derivatives of VisiCalc. (Actually,
that's probably still true today, but in 1992 there wasn't as big a
base of software developers to work other concepts.)

-- 
:wq

Re: [gentoo-user] I want to play movies without hangs [SOLVED, sort of]

[Paul Hartman]

On Fri, May 11, 2012 at 8:40 PM, Alex Schuster <wonko@wonkology.org> wrote:
> Finally, I found something. It's Dolphin!
>
> I've done some longer testing, always playing the same video parallel
> with a dd if=/dev/zero of=/tmp/10G bs=1M count=10000, in mplayer, for a
> minute, several times. When I do this by opening the file in Dolphin, I
> get about 15 interruptions, some for longer than a second. Started on the
> command line, there are very few, I can play the video for minutes
> without a gap. Hooray!
>
> In KDE, I usually play videos by opening them in Dolphin. I exchanged
> 'mplayer %U' by 'xterm -T MPLAYER -e mplayer %U' in the settings, now
> mplayer runs in a terminal, and all is fine. I created a window rule so
> the terminal automatically minimizes. Cool!
>
> It only happens in mplayer and mplayer2. Other players work fine, but I
> like mplayer best, and prefer to run it without any window decoration.
>
> Now, would this be an MPlayer problem, or one of Dolphin?

I wonder if Dolphin is generating thumbnails/preview indexes at the
same time you're trying to play, causing resource contention. I wonder
if you can disable thumbnail generation by Dolphin or remove the
association it has with video files and mplayer.

Re: [gentoo-user] I want to play movies without hangs [SOLVED, sort of]

[Alex Schuster]

Paul Hartman writes:

> On Fri, May 11, 2012 at 8:40 PM, Alex Schuster <wonko@wonkology.org>
> wrote:
> > Finally, I found something. It's Dolphin!
[...]
> > Now, would this be an MPlayer problem, or one of Dolphin?
> 
> I wonder if Dolphin is generating thumbnails/preview indexes at the
> same time you're trying to play, causing resource contention. 

Sometimes it does that, but that would happen when I play from the command
line, too. And now it even works from Dolphin, when it is set to open
mplayer in a terminal. So this cannot be the problem. And even if it
were, avoiding thumbnails would be a bad workaround only, such operations
should not affect video playback.

Maybe this is somehow related to the other problem I had with mplayer2
only, using 100% CPU when idle, only when started from a file manager. 

	Wonko

Re: [gentoo-user] I want to play movies without hangs

[Alex Schuster]

Michael Mol wrote:

> On Sat, May 12, 2012 at 8:58 PM, Alex Schuster <wonko@wonkology.org>
> wrote:

[about showing which processes use how much swap]

> > Michael Mol writes:
[...]
> >> sys-process/htop
> >
> > Huh? I only see the total amount of swap being used, but no entry per
> > process.
> 
> Hit F2, and go down to 'columns'. Anything per-process found under
> /proc can be added as a column.

Whoa! This is amazing, I did not know that htop can do all this. Thanks!

But I still cannot get it to display the swap used by processes. When I
add NSWAP and CNSWAP columns, they are not displayed. I found some
information on that, looks to me like this is not really supported:

http://stackoverflow.com/questions/479953/how-to-find-out-which-processes-are-swapping-in-linux
http://wiki.directi.com/display/tu/Understanding+Processes+in+Linux

	Wonko

Re: [gentoo-user] I want to play movies without hangs

[Mark Knecht]

On Sat, May 12, 2012 at 6:04 PM, Michael Mol <mikemol@gmail.com> wrote:
> On Sat, May 12, 2012 at 8:58 PM, Alex Schuster <wonko@wonkology.org> wrote:
<SNIP>
>>> sys-process/htop
>>
>> Huh? I only see the total amount of swap being used, but no entry per
>> process.
>
> Hit F2, and go down to 'columns'. Anything per-process found under
> /proc can be added as a column.
>
> --
> :wq
>

Anything in there show network through-put per process? I've been
looking for a way to monitor what's going to each of my VMs?

Cheers,
Mark

Re: [gentoo-user] I want to play movies without hangs

[Michael Mol]

On Thu, May 17, 2012 at 9:51 AM, Mark Knecht <markknecht@gmail.com> wrote:
> On Sat, May 12, 2012 at 6:04 PM, Michael Mol <mikemol@gmail.com> wrote:
>> On Sat, May 12, 2012 at 8:58 PM, Alex Schuster <wonko@wonkology.org> wrote:
> <SNIP>
>>>> sys-process/htop
>>>
>>> Huh? I only see the total amount of swap being used, but no entry per
>>> process.
>>
>> Hit F2, and go down to 'columns'. Anything per-process found under
>> /proc can be added as a column.

> Anything in there show network through-put per process? I've been
> looking for a way to monitor what's going to each of my VMs?

NAFAIK. Though if you get a kernel patch that gets per-process socket
auditing added, then it should show up. :)

I usually use iftop for watching flows. There's another tool I
installed which handles some things (such as IPv6) better, but inara
and kaylee are still down, so I can't peek at their world files to
find out what it was.

-- 
:wq

Re: [gentoo-user] I want to play movies without hangs

[Mark Knecht]

On Thu, May 17, 2012 at 7:13 AM, Michael Mol <mikemol@gmail.com> wrote:
> On Thu, May 17, 2012 at 9:51 AM, Mark Knecht <markknecht@gmail.com> wrote:
>> On Sat, May 12, 2012 at 6:04 PM, Michael Mol <mikemol@gmail.com> wrote:
>>> On Sat, May 12, 2012 at 8:58 PM, Alex Schuster <wonko@wonkology.org> wrote:
>> <SNIP>
>>>>> sys-process/htop
>>>>
>>>> Huh? I only see the total amount of swap being used, but no entry per
>>>> process.
>>>
>>> Hit F2, and go down to 'columns'. Anything per-process found under
>>> /proc can be added as a column.
>
>> Anything in there show network through-put per process? I've been
>> looking for a way to monitor what's going to each of my VMs?
>
> NAFAIK. Though if you get a kernel patch that gets per-process socket
> auditing added, then it should show up. :)
>
> I usually use iftop for watching flows. There's another tool I
> installed which handles some things (such as IPv6) better, but inara
> and kaylee are still down, so I can't peek at their world files to
> find out what it was.
>
> --
> :wq
>

Thanks. iftop is interesting but seems more focused on the provider of
the media source and less on the sink. I also use nettop to watch
overall bitrates but I suspect you have that one also.

Assume I have 3 VMs running and they are all streaming media.
VM1->Netflix, VM2->Hulu, VM3->Amazon, etc. What I'm really interested
in is something that would tell me how much bandwidth each VM is
getting. Per-process would almost certainly do that, and maybe that's
what I'll eventually have to do, but I'm hoping to find some little
app that maybe someone has put together.

Thanks,
Mark

Re: [gentoo-user] I want to play movies without hangs

[Alex Schuster]

Mark Knecht writes:

> Anything in there show network through-put per process? I've been
> looking for a way to monitor what's going to each of my VMs?

net-analyzer/nethogs does that.

	Wonko