[gentoo-amd64] Re: 2.6.16 and >=4G mem (was ndiswrapper)

[Duncan <1i5t5.duncan@cox.net>]

Boyd Stephen Smith Jr. posted <200603251758.33840.bss03@volumehost.net>,
excerpted below,  on Sat, 25 Mar 2006 17:58:15 -0600:

> I had something like this happening on my 4GB system.  With IOMMU off, I'd 
> only have access to like 2G of memory.  With IOMMU on, I had various 
> results, ranging from kernel panic before initrd is loaded to missing 
> memory (small, like maybe the size of my PCI IO window?) to a completely 
> working system with devices properly mapped to memory addresses beyond my 
> physical RAM.  These various behaviors were controlled, oddly enough, by 
> my BIOS settings.
> 
> I'm running a Tyan Dual-Opteron Dual-PCIe board... If you've got a similar 
> board, I can share my BIOS and kernel settings and maybe you can resolve 
> your IOMMU issues.

Not the same one, but similar, probably similar enough here to suffice.  I
see you mention the Tyan 2895 below, mine's the 2885, dual Opteron but
PCI-X and 8x AGP, not yet PCI-E.  Mine's the full AMD 8xxx chipset, both
south and north/tunnel.  Tyan is about the best choice there is in terms
of Linux support for Opteron.  Other makers do Opteron boards that do
Linux, but they don't very well actively support Linux, the way Tyan does
-- even to supplying lm_sensors config files and the like.  I've also been
very happy in that my board was like generation 1.3 or 1.5, but Tyan has
fully supported the board with dual-core capable firmware since that came
out with about generation 2.0.  That  is in fact one of the reasons I
upgraded the memory -- I plan on upgrading to dual core in 6 months or so,
to have 2 by dual-core = 4 cores on two sockets, and I don't expect to
grow into the memory until then.
 
The board was $400-some, more than I'd ever paid for a mobo before, but it
has been /well/ worth it, going on 2 1/2 years now, and I'm looking at at
least that long before I upgrade again, so 5 years easily, possibly 7
years, the longest I've ever run on the same mobo by far (I think 3-4
years was the longest previous). I could easily end up having to change
out the battery in the thing, for the first time in my life.  The thing
is, with 8 gigs memory, 4x300gig 5-yr warrantee Seagate SATA drives in
RAID, and 2xdual-cores, even 5 years in (2 1/2 from now), it's going to be
no slouch of a system. It's hard to even comprehend a seven year old
system (well, mobo anyway) being anything but antique, and it'll certainly
be showing it's age by then, but it should still be reasonable.  Consider
what a seven year old system is now, maybe half a GHz Athlon original? The
thing should be far better than that in comparison, considering it'll be 7
years old!

As for the issue at hand, I've had a similar experience.  If I use a mem=
kernel command line between 3 and 5 gig, it sees 2 gig, period.  I haven't
tried over 5 gig, but I was shocked that telling it 5 gig only gave me two.

Setting the BIOS memory hole between contiguous and discrete seems to make
no difference at all.  Setting the other one (I forget what it's called)
gives me 8 gig POST detect, 7-odd gig kernel detect one way, 10 gig POST
detect, the full 8 gig kernel detect, the other.  Naturally, I have it set
to the 10-POST/8-kernel.

With kernel 2.6.15, configuring IOMMU won't let me boot (without a mem=
kernel command line).  Configuring it in but forcing it off (iommu=off on
the kernel command line) gives me some IOMMU needed errors and it still
won't boot (again, without a mem=).  Configuring it out, the system boots
just fine.

With kernel 2.6.16, configuring IOMMU in or out, on or off, won't boot
(without a mem=).

Setting mem=2, 4, or 5 gig, results in 2 gig usable, and IOMMU
automatically turned off as it doesn't need it because it sees only 2 gig
of memory.  The system boots fine, but of course without full memory. 
That's with both kernels.

Writing this, it's obvious I should try mem=8g, just to see what it does,
but I haven't tried that yet.

So... yeah, the combination of BIOS and kernel IOMMU settings that work
for you, particularly if you have libata based SATA or regular SCSI
running fine on kernel 2.6.16, would be very useful.

The other thing I should look at is whether the Gentoo kernel includes any
patches that look apropos to >=4G on amd64, since I've only tried vanilla
kernels.  I know I didn't need anything in the Gentoo patch set before,
but it's possible they have something in there to do with big memory, that
I now need.

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman in
http://www.linuxdevcenter.com/pub/a/linux/2004/12/22/rms_interview.html


-- 
gentoo-amd64@gentoo.org mailing list