Re: [gentoo-dev] New distfile mirror layout

[Joshua Kinard <kumba@gentoo.org>]

On 10/20/2019 16:57, Joshua Kinard wrote:> On 10/20/2019 05:44, Michal Górny wrote:
>> On Sun, 2019-10-20 at 05:21 -0400, Joshua Kinard wrote:
>>> On 10/20/2019 04:32, Michal Górny wrote:
[snip]
>> You believe it to be a problem.  Don't expect others to bother upstream
>> with your preferences.
> 
> Hah.  So you consider texlive having 16k+ distfiles to be completely within
> operating norms then?
> 
> I did a quick look, and it looks like the TeX project has a fairly
> comprehensive mirroring system distributed around the world.  In fact, it
> looks like they emulate Perl's CPAN system with "CTAN":
> 
> https://ctan.org/
> 
> I don't know the history of the texlive and other associated tex packages in
> Gentoo, but my guess is instead of doing what our Perl packages do, someone
> just decided to mirror the CTAN archive directly on the Gentoo distfiles
> system.  It seems to me that what should actually happen is that we leverage
> CTAN itself, much like CPAN, and use their mirroring system instead of
> burdening our infrastructure as an unofficial CTAN archive.
> 
> I know we've got a ton of Perl packages for the core set of Perl modules,
> but doesn't the CPAN eclass also have the capability to auto-generate an
> ebuild package for virtually any Perl package distributed via CPAN?  Can
> that logic be used with the CTAN system in its own eclass and then we remove
> the 16k+ texlive modules off of our mirrors completely?  Or at the worst, we
> might just have to generate ebuilds for texlive modules and treat them as
> discrete, installed packages.

So looking at texlive-latexextra-2019-r2.ebuild, it defines three variables:

  - TEXLIVE_MODULE_CONTENTS, with 1,241 space-delimited module names
  - TEXLIVE_MODULE_DOC_CONTENTS, with 1,227 space-delimited doc names
  - TEXLIVE_MODULE_SRC_CONTENTS, with 745 space-delimited src names

Then, in texlive-module.eclass, there's these loops:

for i in ${TEXLIVE_MODULE_CONTENTS}; do
        SRC_URI="${SRC_URI} mirror://gentoo/texlive-module-${i}-${PV}.${PKGEXT}"
done

# Forge doc SRC_URI
[ -n "${TEXLIVE_MODULE_DOC_CONTENTS}" ] && SRC_URI="${SRC_URI} doc? ("
for i in ${TEXLIVE_MODULE_DOC_CONTENTS}; do
        SRC_URI="${SRC_URI} mirror://gentoo/texlive-module-${i}-${PV}.${PKGEXT}"
done
[ -n "${TEXLIVE_MODULE_DOC_CONTENTS}" ] && SRC_URI="${SRC_URI} )"

# Forge source SRC_URI
if [ -n "${TEXLIVE_MODULE_SRC_CONTENTS}" ] ; then
        SRC_URI="${SRC_URI} source? ("
        for i in ${TEXLIVE_MODULE_SRC_CONTENTS}; do
                SRC_URI="${SRC_URI} mirror://gentoo/texlive-module-${i}-${PV}.${PKGEXT}"
        done
        SRC_URI="${SRC_URI} )"
fi

I think this is definitely an issue with how this package is laying out its
needed distfiles.  It really should be leveraging CTAN system at a minimum
to fetch all of the needed distfiles so we can get them off of our
distfiles mirror.  Then it would be interesting to re-run the math on
the distfiles distribution using the different schemes highlighted in the
GLEP-75 paper.

Longer-term, I think this entire approach should be revisited by the TeX
team to make it behave more like Perl or Python packages by having discrete
ebuilds for these modules.  That's not exactly a small undertaking, but
this current approach feels very kludgy in its design and is probably
asking for trouble.  I looked at several of the modules on CTAN, and they
each have their own version and even have different licenses.

E.g.,

  - altfont is licensed under "GNU General Public License" (version ??)
  - achemso is licensed under "The LaTeX Project Public License 1.3c"
  - arraysort is licensed under "The LaTeX Project Public License 1.2"
  - amsfonts is licensed under "The SIL Open Font License"
  - a0poster is licensed under "The LaTeX Project Public License" (ver ??)
  - arydshln is licensed under "The LaTeX Project Public License 1"
  - aurl is licensed under "Public Domain Software"

That's just a random selection from the 'a' category.  Do we have copies
of those licenses in the tree?  Do they allow redistribution of the
distfiles?  For the users that want "free" software, do any of the licenses
in any of the TeX modules put up any disagreeable restrictions?

Etc...

-- 
Joshua Kinard
Gentoo/MIPS
kumba@gentoo.org
rsa6144/5C63F4E3F5C6C943 2015-04-27
177C 1972 1FB8 F254 BAD0 3E72 5C63 F4E3 F5C6 C943

"The past tempts us, the present confuses us, the future frightens us.  And
our lives slip away, moment by moment, lost in that vast, terrible in-between."

--Emperor Turhan, Centauri Republic