[gentoo-amd64] Tcp transfer has a pause every few seconds

[gentoo-amd64] Tcp transfer has a pause every few seconds

[P.V.Anthony]

Hi,

I have a problem with my nfs transfer using tcp or udp. There is a pause every 
few seconds when I am writing to the server. This causes the video that I am 
recording to have jerks and lost frames. The tcp stream is not constant.

I will try to explain the best I can.

I have a server for video use. I connect the videoserver with Macs using cross 
cable. Using NFS version 3 and using tcp. The filesystem on the server is XFS.

On the macs there is a dv camera connected. Using a video editing software, I am 
recording the video to the nfs mount on the mac. After the recording when I play 
back the recording there are jerks in the video every 20 seconds or so.

Next I copy a big file to the nfs mount. The file goes through but I notice that 
  there is pause every 20 seconds or so. DV video only need about 3.5MB per 
second. All the ethernet connection is gigabit.

What am I doing wrong? Please do advice.

P.V.Anthony
-- 
gentoo-amd64@gentoo.org mailing list

[gentoo-amd64] Re: Tcp transfer has a pause every few seconds

[Duncan]

P.V.Anthony posted <435D3729.4070507@singnet.com.sg>, excerpted below,  on
Tue, 25 Oct 2005 03:34:01 +0800:

> I have a problem with my nfs transfer using tcp or udp. There is a pause
> every few seconds when I am writing to the server. This causes the video
> that I am recording to have jerks and lost frames. The tcp stream is not
> constant.
> 
> I will try to explain the best I can.
> 
> I have a server for video use. I connect the videoserver with Macs using
> cross cable. Using NFS version 3 and using tcp. The filesystem on the
> server is XFS.
> 
> On the macs there is a dv camera connected. Using a video editing
> software, I am recording the video to the nfs mount on the mac. After
> the recording when I play back the recording there are jerks in the
> video every 20 seconds or so.
> 
> Next I copy a big file to the nfs mount. The file goes through but I
> notice that
>   there is pause every 20 seconds or so. DV video only need about 3.5MB
>   per
> second. All the ethernet connection is gigabit.
> 
> What am I doing wrong? Please do advice.

Pause every few seconds.  I've seen two commonly reported solutions to
this -- two different problems resulting in the same symtoms -- and
another one less common in general but likely equally common on amd64.

1)  DMA, or rather, the lack thereof, or interrupt sharing.  Ensure that
you have  the correct chipset drivers installed, both for whatever bus the
NIC is on, and for your hard drive interface (the PATA/SATA/SCSI chipset).
Often, generic or the wrong drivers will work in degraded mode, but DMA
won't be turned on, and performance will suffer.  For your hard drives
(yes, wrong drivers there /can/ cause bottlenecks up the chain, including
onto the same bus as the NIC), use hdparm (or sdparm) to verify that both
the drive and the chipset agree that DMA is on.

Also check interrupt distribution and sharing.  You may be able to change
these settings in the BIOS, or try switching some cards around
(particularly your NIC if it's not on-board built-in).

2)  Do you have an "extra" network interface?  I've seen this one on
MSWormOS and am unsure how Linux would even have the problem, unless you
have a spare interface (say a wireless card) that's running but not
connected.  Anyway, the problem here is an interface set to DHCP mode that
can't get an IP because it's not connected, so of course can't see a
server.  The system would pause every few minutes (every few seconds seems
a bit much, but...) and try to obtain an IP, stalling all I/O in the
process.  (Again, the reason it's stalling all I/O is likely rooted in
misconfigured DMA or IRQs, but anyway...)

3)  Seen frequently on this list as a problem resulting in one or more of
a number of symtoms, including this one:  APIC or ACPI problems.  Take a
look at the kernel documentation under $kerneldir/Documentation/x86_64,
the apic and acpi boot options, and idle=poll as well.  Note that those
needing these often find they don't after a mature BIOS is available and
flashed.  idle=poll is particularly nasty, as it will cause your CPU to
work far harder and run far hotter than it ordinarily would, but some
early amd64 boards required it until an updated BIOS fixed the issue.  I'm
not sure if that's still a problem with the newer boards or not.

Also note that for specific net chip, various configuration options may be
available.  In particular, there's the Broadcom tigon3/tg3 kernel driver,
and then another (I forget what it's called) for the same net chip.  Often
one will work well, the other won't work or will have issues. If that's
your chip, look it up and consider trying the other driver.

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman in
http://www.linuxdevcenter.com/pub/a/linux/2004/12/22/rms_interview.html


-- 
gentoo-amd64@gentoo.org mailing list

Re: [gentoo-amd64] Re: Tcp transfer has a pause every few seconds

[P.V.Anthony]

Duncan wrote:
> 
> Pause every few seconds.  I've seen two commonly reported solutions to
> this -- two different problems resulting in the same symtoms -- and
> another one less common in general but likely equally common on amd64.
> 
> 1)  DMA, or rather, the lack thereof, or interrupt sharing.  Ensure that
> you have  the correct chipset drivers installed, both for whatever bus the
> NIC is on, and for your hard drive interface (the PATA/SATA/SCSI chipset).
> Often, generic or the wrong drivers will work in degraded mode, but DMA
> won't be turned on, and performance will suffer.  For your hard drives
> (yes, wrong drivers there /can/ cause bottlenecks up the chain, including
> onto the same bus as the NIC), use hdparm (or sdparm) to verify that both
> the drive and the chipset agree that DMA is on.

I am using the Promise EX8350 raid card and using there driver.

> 
> Also check interrupt distribution and sharing.  You may be able to change
> these settings in the BIOS, or try switching some cards around
> (particularly your NIC if it's not on-board built-in).

This one I was thinking of trying the last. (Read below)

> 
> 2)  Do you have an "extra" network interface?  I've seen this one on
> MSWormOS and am unsure how Linux would even have the problem, unless you
> have a spare interface (say a wireless card) that's running but not
> connected.  Anyway, the problem here is an interface set to DHCP mode that
> can't get an IP because it's not connected, so of course can't see a
> server.  The system would pause every few minutes (every few seconds seems
> a bit much, but...) and try to obtain an IP, stalling all I/O in the
> process.  (Again, the reason it's stalling all I/O is likely rooted in
> misconfigured DMA or IRQs, but anyway...)

All the network cards are set to static address.

> 
> 3)  Seen frequently on this list as a problem resulting in one or more of
> a number of symtoms, including this one:  APIC or ACPI problems.  Take a
> look at the kernel documentation under $kerneldir/Documentation/x86_64,
> the apic and acpi boot options, and idle=poll as well.  Note that those
> needing these often find they don't after a mature BIOS is available and
> flashed.  idle=poll is particularly nasty, as it will cause your CPU to
> work far harder and run far hotter than it ordinarily would, but some
> early amd64 boards required it until an updated BIOS fixed the issue.  I'm
> not sure if that's still a problem with the newer boards or not.

I am using the latest bios. So far I am not using any apic boot options. Will 
read up.

> 
> Also note that for specific net chip, various configuration options may be
> available.  In particular, there's the Broadcom tigon3/tg3 kernel driver,
> and then another (I forget what it's called) for the same net chip.  Often
> one will work well, the other won't work or will have issues. If that's
> your chip, look it up and consider trying the other driver.

Will check this.

Thank you very much for replying and helping. Please read my observations below 
and if you have time please advice. Let me know if I still have to do the checks 
as mention above.

Here are some specs and observations.

Specs.
cpu: AMD 64 3000+
drives: Sata drives
mobo: MSI Neo 4 Platinum
Ram: 2GB
Raid: Promise EX8350
OS: Gentoo
Kernel: 2.6.13 r3
Filesystem: XFS

Observations.
When I was using the xfs I had the problem of the pauses in the tcp transfers. 
These pauses, I am guessing is because of the pdflush that occurs around every 
20 seconds. There is only one pdflush every 20 seconds. This caused lose of 
video frames when we did the video recording. The video would have a jerk or 
jump when played back.

I changed the dirty_ratio from the default of 40 to 5. This solved the problem. 
During the recording preview there seems to be some jerks but during the 
playback it seems to be ok. The pdflush happens more often and there is still 
one pdflush. I think the jerk that I see in the record preview coincides with 
the pdflush. More jerks in record preview but ok during playback.

Next I formatted the drives to the ext3 filesystem. The dirty_ratio was set back 
to the default value of 40. When the video started to record there is no loss of 
frames and no jerks in the preview window. During the playback there is no loss. 
Everything is good. Check the top for pdflush and noticed that the there is two 
pdflush. The pdflushs is happening at every 5 seconds or so. Very fast compared 
to when I was using xfs.

So the problem must be my xfs setting. For xfs I have only used the default 
setting. Or could there be a problem with xfs and AMD64 kernel?

What must I do to get xfs to work? Or what am I doing wrong?

Please comment and advise.

Once again thank you very much for the advice.

P.V.Anthony


-- 
gentoo-amd64@gentoo.org mailing list

[gentoo-amd64] Re: Re: Tcp transfer has a pause every few seconds

[Duncan]

P.V.Anthony posted <4364E65D.4070109@singnet.com.sg>, excerpted below,  on
Sun, 30 Oct 2005 23:27:25 +0800:

> specs and observations.
> 
> Specs.
> cpu: AMD 64 3000+
> drives: Sata drives
> mobo: MSI Neo 4 Platinum
> Ram: 2GB
> Raid: Promise EX8350
> OS: Gentoo
> Kernel: 2.6.13 r3
> Filesystem: XFS

Now that I see that mobo listed...

See the other threads currently active on the list/group right now.  (You
can read the list as either a newsgroup or a web forum, thru gmane.org.  I
use their list2news interface to read the list as a newsgroup.  That way
you can see posts from before you subscribed, if desired.)  

I haven't kept exact track but it seems I'm seeing that mobo mentioned in
three different active threads ATM, this one included.  The latest BIOS
(1.B IIRC?) apparently has some rather major issues with recent kernels.
The threads have been saying either revert to 1.9 (or whatever it was) or
try an earlier kernel (2.6.11 or earlier, IIRC).  They problems mentioned
haven't been this one, but it's a troublesome combination in any case, so
certainly read up, and consider reverting either the kernel or the BIOS.

NOTE THAT ONE GUY ENDED UP FRYING HIS BIOS REVERTING, and had to buy a new
chip ($10-ish, but he had to wait for it to be delivered).  However, he
was having issues getting into the BIOS already (the keyboard wasn't being
activated soon enough to detect the delete to put it in BIOS edit mode),
and it's possible the fried BIOS was related to that.

The $10 isn't bad, so you might wish to order the spare BIOS chip and have
it on hand before you try reverting, just in case.

...  As I said in a different thread, altho I understand it's changed a
bit now, back when I was shopping for my dual Opteron board, it was
between a Tyan and an MSI board.  Visiting the MSI site, all their BIOS
upgrades AND all their documentation was in MSWormOS executable format
(I'm guessing self-extracting zip, but all I know is it had the .exe at
the end).  Tyan had the more traditional pdf docs and zip file BIOS packs.
I emailed MSI and told them why they lost the sale.  (The MSI board was
cheaper and I would have likely gotten it if it looked like they cared at
all about their Linux customers.)  Maybe my mail was  part of what made
them decide to switch back to normal formats.  

Anyway, I've been quite impressed with Tyan -- they are quite responsive
on the Linux front and regularly have Linux drivers, and even have
customized lm_sensors configs for their boards.  As well, many of them
carry Linux certifications -- the manual for mine says Red Hat, SuSE, and
Turbo Linux, all three, supported and certified.

MSI OTOH, as I explained, I was rather disimpressed with, and the recent
threads here on the list haven't helped.  I consider myself lucky to have
gone Tyan, and don't think I'll be considering MSI any time soon.

> Observations.
> When I was using the xfs I had the problem of the pauses in the tcp
> transfers. These pauses, I am guessing is because of the pdflush that
> occurs around every 20 seconds. There is only one pdflush every 20
> seconds. This caused lose of video frames when we did the video recording.
> The video would have a jerk or jump when played back.
> 
> I changed the dirty_ratio from the default of 40 to 5. This solved the
> problem. During the recording preview there seems to be some jerks but
> during the playback it seems to be ok. The pdflush happens more often and
> there is still one pdflush. I think the jerk that I see in the record
> preview coincides with the pdflush. More jerks in record preview but ok
> during playback.
> 
> Next I formatted the drives to the ext3 filesystem. The dirty_ratio was
> set back to the default value of 40. When the video started to record
> there is no loss of frames and no jerks in the preview window. During the
> playback there is no loss. Everything is good. Check the top for pdflush
> and noticed that the there is two pdflush. The pdflushs is happening at
> every 5 seconds or so. Very fast compared to when I was using xfs.
> 
> So the problem must be my xfs setting. For xfs I have only used the
> default setting. Or could there be a problem with xfs and AMD64 kernel?
> 
> What must I do to get xfs to work? Or what am I doing wrong?

The dirty_ratio stuff you mention is beyond my level -- you know more
there than I do by far, it would seem.  I've only used reiserfs here,
since I switched to Linux, as well, so can't say much about ext2/3 or xfs,
beyond what's out there to read about them.  I /can/ say I have two
pdflush threads here, but I always figured that was one running on each
CPU...

Anyway, now that you mentioned the MSI Neo 4 Platinum mobo, with the other
threads on it, I'd say the BIOS/kernel issues from that very likely are
causing your issues as well.  Address that, and there's a good chance your
problems will disappear.  In any case, there are several others on the
list with that board to compare notes with, which should help.

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman in
http://www.linuxdevcenter.com/pub/a/linux/2004/12/22/rms_interview.html


-- 
gentoo-amd64@gentoo.org mailing list

Re: [gentoo-amd64] Re: Re: Tcp transfer has a pause every few seconds

[P.V.Anthony]

Duncan wrote:

> Anyway, now that you mentioned the MSI Neo 4 Platinum mobo, with the other
> threads on it, I'd say the BIOS/kernel issues from that very likely are
> causing your issues as well.  Address that, and there's a good chance your
> problems will disappear.  In any case, there are several others on the
> list with that board to compare notes with, which should help.

You are right. Just installed gentoo kernel 2.6.14 and xfs do not have any 
problems. Need to do more test to confirm.

I was looking at the change log for kernel 2.6.14 and there seems to be many 
entries for xfs. Maybe they solved some problems.

Will keep you all posted.

P.V.Anthony

-- 
gentoo-amd64@gentoo.org mailing list

[gentoo-amd64] Re: Re: Re: Tcp transfer has a pause every few seconds

[Duncan]

P.V.Anthony posted <43697D63.5060508@singnet.com.sg>, excerpted below,  on
Thu, 03 Nov 2005 11:00:51 +0800:

> You are right. Just installed gentoo kernel 2.6.14 and xfs do not have any 
> problems. Need to do more test to confirm.
> 
> I was looking at the change log for kernel 2.6.14 and there seems to be many 
> entries for xfs. Maybe they solved some problems.

Very glad it seems fixed, and thanks for the feedback!

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman in
http://www.linuxdevcenter.com/pub/a/linux/2004/12/22/rms_interview.html


-- 
gentoo-amd64@gentoo.org mailing list