[gentoo-dev] [RFC] Begin a dev-libs/nodejs category?

[gentoo-dev] [RFC] Begin a dev-libs/nodejs category?

[Herb Miller Jr.]

[-- Attachment #1: Type: text/plain, Size: 768 bytes --]

When I did my homework on creating nodejs ebuilds (not nodejs itself but
packages written in node), it seems the topic has come up a few times in
the past but the time commitment and general disorganization of upstream
has scared off any serious attempts at packaging.

Seeing as there has been interest in nodejs packages, and in the end we
could be talking 400-600+ packages, would it be possible to create
dev-libs/nodejs category? I would be happy to write and proxy-maintain
ebuilds for a large number of them to get things in motion. I've already
opened pull request #7427 [1] for chalk and its dependencies. I'm aiming
to build up to all the dependencies needed for Visual Studio Code OSS.

[1]:https://github.com/gentoo/gentoo/pull/7427

----
Herb Miller Jr.


[-- Attachment #2: Type: text/html, Size: 22128 bytes --]

Re: [gentoo-dev] [RFC] Begin a dev-libs/nodejs category?

[Michał Górny]

W dniu wto, 20.03.2018 o godzinie 07∶50 -0400, użytkownik Herb Miller
Jr. napisał:
> When I did my homework on creating nodejs ebuilds (not nodejs itself but
> packages written in node), it seems the topic has come up a few times in
> the past but the time commitment and general disorganization of upstream
> has scared off any serious attempts at packaging.
> 
> Seeing as there has been interest in nodejs packages, and in the end we
> could be talking 400-600+ packages, would it be possible to create
> dev-libs/nodejs category? I would be happy to write and proxy-maintain
> ebuilds for a large number of them to get things in motion. I've already
> opened pull request #7427 [1] for chalk and its dependencies. I'm aiming
> to build up to all the dependencies needed for Visual Studio Code OSS.

I think you meant 'dev-nodejs'.

> 
> [1]:https://github.com/gentoo/gentoo/pull/7427
> 
> ----
> Herb Miller Jr.
> 

-- 
Best regards,
Michał Górny

Re: [gentoo-dev] [RFC] Begin a dev-libs/nodejs category?

[Herb Miller Jr.]

Indeed I did. Thank you for pointing that out. It had been a long night.

----
Herb Miller Jr.


On 03/20/2018 02:34 PM, Michał Górny wrote:
> W dniu wto, 20.03.2018 o godzinie 07∶50 -0400, użytkownik Herb Miller
> Jr. napisał:
>> When I did my homework on creating nodejs ebuilds (not nodejs itself but
>> packages written in node), it seems the topic has come up a few times in
>> the past but the time commitment and general disorganization of upstream
>> has scared off any serious attempts at packaging.
>>
>> Seeing as there has been interest in nodejs packages, and in the end we
>> could be talking 400-600+ packages, would it be possible to create
>> dev-libs/nodejs category? I would be happy to write and proxy-maintain
>> ebuilds for a large number of them to get things in motion. I've already
>> opened pull request #7427 [1] for chalk and its dependencies. I'm aiming
>> to build up to all the dependencies needed for Visual Studio Code OSS.
> I think you meant 'dev-nodejs'.
>
>> [1]:https://github.com/gentoo/gentoo/pull/7427
>>
>> ----
>> Herb Miller Jr.
>>

Re: [gentoo-dev] [RFC] Begin a dev-libs/nodejs category?

[Michael Orlitzky]

On 03/20/2018 07:50 AM, Herb Miller Jr. wrote:
> When I did my homework on creating nodejs ebuilds (not nodejs itself but
> packages written in node), it seems the topic has come up a few times in
> the past but the time commitment and general disorganization of upstream
> has scared off any serious attempts at packaging.

There's a real technical problem hidden in there. Since npm
(recursively!) bundles every dependency, nobody worries about
compatibility in their JS packages. You'll quickly find yourself stuck.

For example, if you want to package an end-user application "foo", it
might depend on libraries "bar-1.0" and "baz-2.0". But then "bar-1.0"
itself depends on "baz-1.0". What do you do? Slot everything? How do you
make NodeJS look in the right place? You're going to need a plan,
because this situation is not at all uncommon.

Re: [gentoo-dev] [RFC] Begin a dev-libs/nodejs category?

[Herb Miller Jr.]

On 03/20/2018 02:48 PM, Michael Orlitzky wrote:
> On 03/20/2018 07:50 AM, Herb Miller Jr. wrote:
>> When I did my homework on creating nodejs ebuilds (not nodejs itself but
>> packages written in node), it seems the topic has come up a few times in
>> the past but the time commitment and general disorganization of upstream
>> has scared off any serious attempts at packaging.
> There's a real technical problem hidden in there. Since npm
> (recursively!) bundles every dependency, nobody worries about
> compatibility in their JS packages. You'll quickly find yourself stuck.
>
> For example, if you want to package an end-user application "foo", it
> might depend on libraries "bar-1.0" and "baz-2.0". But then "bar-1.0"
> itself depends on "baz-1.0". What do you do? Slot everything? How do you
> make NodeJS look in the right place? You're going to need a plan,
> because this situation is not at all uncommon.
>
That is scary. I hadn't noticed there are node_modules directories under
many node modules and that npm list outputs different versions of the
same dependency. To help me better understand the situation, when you
see this happen does "bar-1.0" normally depend on "baz-1.0" because...

A) There is some huge technical hurdle in upgrading to "baz-2.0"?
B) I was too lazy or didn't care to upgrade to "baz-2.0"?
C) My package.json is outdated?

If A, can you point me to a good example I can take a look at? If it's
normally B or C, I have no problem making lots of upstream pull
requests. It's just Javascript. Though I understand that's not a true
solution in the long-run and has burnout written all over it. I'll have
to give the problem much more thought.

----
Herb Miller Jr.

Re: [gentoo-dev] [RFC] Begin a dev-libs/nodejs category?

[Michael Orlitzky]

On 03/20/2018 04:14 PM, Herb Miller Jr. wrote:
> That is scary. I hadn't noticed there are node_modules directories under
> many node modules and that npm list outputs different versions of the
> same dependency. To help me better understand the situation, when you
> see this happen does "bar-1.0" normally depend on "baz-1.0" because...
> 
> A) There is some huge technical hurdle in upgrading to "baz-2.0"?
> B) I was too lazy or didn't care to upgrade to "baz-2.0"?
> C) My package.json is outdated?
> 
> If A, can you point me to a good example I can take a look at?

It's usually B or C, but I hit several cases where there was a real
incompatibility. When I opened Github issues, I got a lot of WONTFIX
responses telling me that I have to use npm.

No particular examples come to mind, though -- this was about 2.5 years ago.

Re: [gentoo-dev] [RFC] Begin a dev-libs/nodejs category?

[Kent Fredric]

[-- Attachment #1: Type: text/plain, Size: 942 bytes --]

On Tue, 20 Mar 2018 14:48:29 -0400
Michael Orlitzky <mjo@gentoo.org> wrote:

> There's a real technical problem hidden in there. Since npm
> (recursively!) bundles every dependency, nobody worries about
> compatibility in their JS packages. You'll quickly find yourself stuck.

Honestly, I expected at some point we'd reach for slotting and normalization,
and recursive trees of symlinks....

eg: 
	/usr/lib/nodejs/<packagename>/<version>/lib/<dependency-name>  -> /usr/lib/nodejs/<packagename>/<version>

Or something like that.

So you'd wind up with

	/usr/lib/nodejs/foo/1.0/lib/bar  -> /usr/lib/nodejs/bar/1.0
	/usr/lib/nodejs/foo/1.0/lib/baz  -> /usr/lib/nodejs/baz/2.0
        /usr/lib/nodejs/bar/1.0/lib/baz  -> /usr/lib/nodejs/baz/1.0
        /usr/lib/nodejs/baz/1.0/...
        /usr/lib/nodejs/baz/2.0/...

Or something like that. But I imagine constructing such a thing a
rather painful exercise.



 

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [gentoo-dev] [RFC] Begin a dev-libs/nodejs category?

[M. J. Everitt]

[-- Attachment #1.1: Type: text/plain, Size: 1069 bytes --]

On 21/03/18 01:27, Kent Fredric wrote:
> On Tue, 20 Mar 2018 14:48:29 -0400
> Michael Orlitzky <mjo@gentoo.org> wrote:
>
>> There's a real technical problem hidden in there. Since npm
>> (recursively!) bundles every dependency, nobody worries about
>> compatibility in their JS packages. You'll quickly find yourself stuck.
> Honestly, I expected at some point we'd reach for slotting and normalization,
> and recursive trees of symlinks....
>
> eg: 
> 	/usr/lib/nodejs/<packagename>/<version>/lib/<dependency-name>  -> /usr/lib/nodejs/<packagename>/<version>
>
> Or something like that.
>
> So you'd wind up with
>
> 	/usr/lib/nodejs/foo/1.0/lib/bar  -> /usr/lib/nodejs/bar/1.0
> 	/usr/lib/nodejs/foo/1.0/lib/baz  -> /usr/lib/nodejs/baz/2.0
>         /usr/lib/nodejs/bar/1.0/lib/baz  -> /usr/lib/nodejs/baz/1.0
>         /usr/lib/nodejs/baz/1.0/...
>         /usr/lib/nodejs/baz/2.0/...
>
> Or something like that. But I imagine constructing such a thing a
> rather painful exercise.
>
.... Said the voice of bitter experience ... *eg* :]


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

Re: [gentoo-dev] [RFC] Begin a dev-libs/nodejs category?

[Benda Xu]

Hello Herb,

"Herb Miller Jr." <gentooherb@gmail.com> writes:

> When I did my homework on creating nodejs ebuilds (not nodejs itself
> but packages written in node), it seems the topic has come up a few
> times in the past but the time commitment and general disorganization
> of upstream has scared off any serious attempts at packaging.
>
> Seeing as there has been interest in nodejs packages, and in the end
> we could be talking 400-600+ packages, would it be possible to create
> dev-libs/nodejs category? I would be happy to write and proxy-maintain
> ebuilds for a large number of them to get things in motion. I've
> already opened pull request #7427 [1] for chalk and its
> dependencies. I'm aiming to build up to all the dependencies needed
> for Visual Studio Code OSS.
>
> [1]:https://github.com/gentoo/gentoo/pull/7427

Your effort will be much appreciated.  I support your plan.

Should the category be dev-js intead?  To me, node.js is but an
implementation of javascript runtime.


BTW, for the 400-500 packages, are you going to create them with a
generator script?

Cheers,
Benda

Re: [gentoo-dev] [RFC] Begin a dev-libs/nodejs category?

[Herb Miller Jr.]

On 03/20/2018 07:50 PM, Benda Xu wrote:
> Hello Herb,
>
> "Herb Miller Jr." <gentooherb@gmail.com> writes:
>
>> When I did my homework on creating nodejs ebuilds (not nodejs itself
>> but packages written in node), it seems the topic has come up a few
>> times in the past but the time commitment and general disorganization
>> of upstream has scared off any serious attempts at packaging.
>>
>> Seeing as there has been interest in nodejs packages, and in the end
>> we could be talking 400-600+ packages, would it be possible to create
>> dev-libs/nodejs category? I would be happy to write and proxy-maintain
>> ebuilds for a large number of them to get things in motion. I've
>> already opened pull request #7427 [1] for chalk and its
>> dependencies. I'm aiming to build up to all the dependencies needed
>> for Visual Studio Code OSS.
>>
>> [1]:https://github.com/gentoo/gentoo/pull/7427
> Your effort will be much appreciated.  I support your plan.
>
> Should the category be dev-js intead?  To me, node.js is but an
> implementation of javascript runtime.
>
>
> BTW, for the 400-500 packages, are you going to create them with a
> generator script?
>
> Cheers,
> Benda
>
>
Hopefully by the time it gets to that many I won't be the only one
creating/maintaining them. If I am, then yes, some kind of automation
would be the only sane way to keep up. I figure I'll face that challenge
when I get there. As I'm writing them I am seeing a general skeleton
forming that most of them fit into.

----
Herb Miller Jr.

Re: [gentoo-dev] [RFC] Begin a dev-libs/nodejs category?

[Kent Fredric]

[-- Attachment #1: Type: text/plain, Size: 785 bytes --]

On Wed, 21 Mar 2018 01:44:11 +0000
Herb Miller Jr. <herb@hlmjr.com> wrote:

> If I am, then yes, some kind of automation
> would be the only sane way to keep up

In my experience you can't *really* rely on automation 100% for this
sort of thing. Not while achieving quality results.

Its viable for an overlay where there's no expectations of quality, but
for the main tree, I find you want to have a human san-check everything
and manually vet each upstream version for "anomalous things".

Automation is good at handling the "known predictable" cases, humans
are better at detecting "huh, that's weird, why did they do that?"

Because you absolutely want to know if upstream added some stupid
change that is harmful to Gentoo users before you blindly replicate it.

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [gentoo-dev] [RFC] Begin a dev-libs/nodejs category?

[Herb Miller Jr.]

On 03/21/2018 03:33 AM, Kent Fredric wrote:
> On Wed, 21 Mar 2018 01:44:11 +0000
> Herb Miller Jr. <herb@hlmjr.com> wrote:
>
>> If I am, then yes, some kind of automation
>> would be the only sane way to keep up
> In my experience you can't *really* rely on automation 100% for this
> sort of thing. Not while achieving quality results.
>
> Its viable for an overlay where there's no expectations of quality, but
> for the main tree, I find you want to have a human san-check everything
> and manually vet each upstream version for "anomalous things".
>
> Automation is good at handling the "known predictable" cases, humans
> are better at detecting "huh, that's weird, why did they do that?"
>
> Because you absolutely want to know if upstream added some stupid
> change that is harmful to Gentoo users before you blindly replicate it.

And I agree with you 100%. I would never rely on automation exclusively.
I would use it to write boilerplate sections, check for updates, check
for breakage, etc. I'd never open a PR for something I hadn't polished
myself.

Re: [gentoo-dev] [RFC] Begin a dev-libs/nodejs category?

[X dej]

2018-03-21 2:44 UTC+01:00, Herb Miller Jr. <herb@hlmjr.com>:
>>> When I did my homework on creating nodejs ebuilds (not nodejs itself
>>> but packages written in node), it seems the topic has come up a few
>>> times in the past but the time commitment and general disorganization
>>> of upstream has scared off any serious attempts at packaging.

Dear Herb,

I would like to make sure that you knew about the disorganized and outdated
attempt named app-portage/g-npm:
URL: https://www.reddit.com/r/Gentoo/comments/38txil/how_do_you_guys_manage_languagespecific_package/
Title: "How do you guys manage language-specific package managers
(npm, gem, pip, etc)? : Gentoo"

I found out about that while trying to make an ebuild for
www-apps/etherpad-lite-1.2.91, see https://bugs.gentoo.org/328897 for
that.

You might also want to know about that attempt:
https://forums.gentoo.org/viewtopic-t-1064550-start-0.html
titled "Portage and NodeJS and similarity with Python, Perl, etc."

Perl monks using gentoo might want to know about app-portage/g-cpan, see first
URL above.

I just found out that app-portage/g-octave app-portage/g-sorcery existed.

I did not find anything wannabe "g-pip" for python.

-- 
xdej@#gentoo-prefix

[gentoo-dev] Pypi generator (Was: [RFC] Begin a dev-libs/nodejs category?)

[Benda Xu]

X dej <dreplaceelettereejbyeletterea@gmail.com> writes:

> I did not find anything wannabe "g-pip" for python.

Check out app-portage/gs-pypi.