Re: [gentoo-kernel] sys-kernel/mips-firmware: thoughts?

[Joshua Kinard <kumba@gentoo.org>]

On 3/30/2020 19:37, Robin H. Johnson wrote:
> On Mon, Mar 30, 2020 at 05:49:20PM -0400, Joshua Kinard wrote:
>> For some time now, I've been trying to maintain a patch that kludges the
>> external firmware loader and specific firmware blobs back into the kernel
>> for sys-kernel/mips-sources.  This works fine up until 5.6, which seems to
>> have further changed the way that firmware is loaded that breaks my existing
>> patch in a not-so-obvious way.  I am not terribly inclined to keep debugging
>> this, so I think it's time to throw in the towel and use external firmware
>> loading.
>>
>> This brings up the issue that frankly, sys-kernel/linux-firmware is overkill
>> for currently-supported MIPS targets.  Speaking for just the SGI systems,
>> MIPS literally needs only 4 firmware blobs (1 is critical, 1 optional, the
>> other two are for the future):
>>   - acenic/tg2.bin:   optional for ACENIC/Tigon II PCI cards
>>   - qlogic/1040.bin:  required for QLA1040 on SGI IP27, IP30
>>   - qlogic/1280.bin:  future use (maybe on SGI IP35?)
>>   - qlogic/12160.bin: future use (maybe on SGI IP35?)
> The 1280 was crucial for SGI Visual Workstation 320, and also
> worked as an option on IP32's PCI64 slots (I used it on my O2 R12k).
> 
> Q1)
> What distfile would it use? If it's going to be just the same upstream
> giant 200+ MiB linux-firmware distfile, then you still have to get that
> copied in for the builds, and when you pin it, you're going to keep it
> on all Gentoo mirrors.

I am leaning to a stripped-down distfile containing just those four
precompiled *.bin files (versus the ihex variants), plus the necessary
license and any redistribution files.  Maybe a custom README pointing people
to the upstream firmware.git location, etc.

A quick test of those four bin blobs and two license files + TAR + XZ is
~65K.  I //could// get it smaller if I shipped the uncompiled ihex files and
then encoded in the ebuild some logic to convert the ihex to binary bobs.


> If you're going to produce a stripped down distfile, how is that going
> to interact when you might need a firmware that isn't in the stripped
> down package (see further about conflicts). I have used other NICs in my
> MIPS gear when I still used it (which I admit was more than a
> decade ago now).

Just add it, add its license file, and revbump the tarball.  Even shipping
the prebuilt *.bin files, it'll still be extremely small compared to the
full-blown linux-firmware package.  I've been re-kludging these same four
firmware blobs in mips-sources for 2-3 years now as patches and no one's
filed any bugs yet, so it shouldn't be an issue.  On the off chance things
ever got out of control in adding a bunch of more recent firmware files,
then this package would be last-rited and we switch to linux-firmware.


>> I wanted to see if there would be any problem with creating a
>> sys-kernel/mips-firmware package to contain, at least initially, these four
>> firmware files from the linux-firmware repo, and install them to either
>> /lib/firmware-mips or /usr/share/mips-firmware (thinking more the latter to
>> avoid polluting /lib any further).
> Q2)
> Do you intend to patch the kernel drivers/base/firmware_loader for these
> new paths, or what other way would you pass them?
> The firmware_class.path modparam takes only ONE path.
> 
> If you don't patch it to add more paths, then I worry that you can't use
> the new paths you propose, and you have to have a blocker against
> sys-kernel/linux-firmware.

This is why I //liked// the in-kernel firmware loader.  No messing about
with paths or such.  I've tested my kludge patch against 5.6_rc7, and while
the bin blobs are built and included in the kernel, something has changed
somewhere in how it tries to find them in a kernel image.  I just don't
really care to try and chase that down anymore.  Though, the option isn't
off the table yet.

That said, using an external patch with a custom file location, it's more a
matter of documentation.  Of our supported SGI systems, the only ones that
*need* firmware are IP27 and IP30 for enabling the SCSI subsystem.  PCI
support on both platforms is still in disarray to the point that most other
PCI devices are still not going to work well.  IP27 lacks code to enable
access to its "big windows" PCI alias space, and Octane has issues with >2GB
RAM and PCI DMA.  It just happens that Acenic 1GB NICs work well on the
Octane (especially the SGI-branded version available on eBay)

Additionally, using a separate install path avoids having to block against
linux-firmware, so users can still install that package if they have some
esoteric (to MIPS) PCI device that needs firmware to get going, and then
just point the firmware loader to linux-firmware's default path.  I'm just
trying to come up with a workable solution for the *common* case w/ IP27 and
IP30.


>> These four blobs have not seen updates in years, so I anticipate there being
>> only a single ebuild for sys-kernel/mips-firmware.  It'll be versioned like
>> the virtual packages (mips-firmware-1, -2, -3, etc), rather than by
>> datestamp, as the only times a version bump happens is if the firmware blobs
>> ever change or we add/remove a blob.
>>
>> Thoughts/feedback?
> 
> Can you clarify relative goals in this effort?
> 
> - Avoid packaging effort?
> - Save root FS disk space (a limited resource on many MIPS systems)
> - Avoid having to build the very large sys-kernel/linux-firmware (even
>   with savedconfig the download/build takes lots of space, starting with
>   the 200+ MiB distfile).

Really, it's just I am tired of kludging in-kernel firmware support back in
(that and Makefile syntax is borderline arcane).  I get it that the cool kid
thing to do these days is build an initramfs into your kernel, but it seems
that upstream is either trying to force that home, or they're just restless
about where they want firmware support to even live.  Stuff keeps moving
around and it's a PITA to debug and chase down.

Long-term, it seems more sensible to just distribute a small tarball of
several key prebuilt bin blobs that mips-sources can just reference when
needed.  I anticipate maybe not more than 2-3 version bumps over the next 5
years with this approach.


> I think you SHOULD keep the datestamp method, and have the PV explicitly
> match the upstream linux tarballs. I'm not sure about if it should have
> it's own smaller distfile or not.

Datestamp would give the wrong impression that these firmwares are updated.
 Acenic/tg2.bin hasn't seen an update since 2004, and you can't even get at
the original firmware C/asm source code anymore, even via archive.org.
Qlogic bins haven't seen updates since 2002 (1040.bin, 1280.bin) and 2003
(12160.bin).  So I can pick any snapshot of linux-firmware from now until
its creation as an external repository, and I am going to wind up with the
same firmware.  Distributing a tailored package means users won't be forced
to download and build new linux-firmware packages for a handful of binary
blobs that will likely never ever change again.

That makes versioning it by single digits, or even a decimal version a bit
better because the version will only change with firmware is added/removed
from the archive.

-- 
Joshua Kinard
Gentoo/MIPS
kumba@gentoo.org
rsa6144/5C63F4E3F5C6C943 2015-04-27
177C 1972 1FB8 F254 BAD0 3E72 5C63 F4E3 F5C6 C943

"The past tempts us, the present confuses us, the future frightens us.  And
our lives slip away, moment by moment, lost in that vast, terrible in-between."

--Emperor Turhan, Centauri Republic