[gentoo-dev] [RFC] making rust-bin ordered first in virtual/rust

[gentoo-dev] [RFC] making rust-bin ordered first in virtual/rust

[Georgy Yakovlev]

Hi,

I've been approached multiple times with that request, and a lot of
time I see new users completely destroyed by rust build time and disk
space requirements.

WDYT about switching order of rusts in a virtual?

RDEPEND="|| (
        ~dev-lang/rust-${PV}
        ~dev-lang/rust-bin-${PV}
)"


becomes

RDEPEND="|| (
        ~dev-lang/rust-bin-${PV}
	~dev-lang/rust-${PV}
)"


Existing installs should be unaffected ofc.
But portage may prefer to depclean rust and not rust-bin if both are
present.
Users who wish to use source version at all times can just add it to
world file.

I see both positives and negatives of doing that, but would like to
reach out to community first.

Thanks!

--
Georgy

Re: [gentoo-dev] [RFC] making rust-bin ordered first in virtual/rust

[Dale]

Georgy Yakovlev wrote:
> Hi,
>
> I've been approached multiple times with that request, and a lot of
> time I see new users completely destroyed by rust build time and disk
> space requirements.
>
> WDYT about switching order of rusts in a virtual?
>
> RDEPEND="|| (
>         ~dev-lang/rust-${PV}
>         ~dev-lang/rust-bin-${PV}
> )"
>
>
> becomes
>
> RDEPEND="|| (
>         ~dev-lang/rust-bin-${PV}
> 	~dev-lang/rust-${PV}
> )"
>
>
> Existing installs should be unaffected ofc.
> But portage may prefer to depclean rust and not rust-bin if both are
> present.
> Users who wish to use source version at all times can just add it to
> world file.
>
> I see both positives and negatives of doing that, but would like to
> reach out to community first.
>
> Thanks!
>
> --
> Georgy
>
>


As a user, if I recall correctly, if a user wants rust-bin, all they
have to do is emerge -1 rust-bin and then let whatever package pull in
the virtual for rust.  The virtual will be satisfied and rust-bin will
be used.  From my understanding, that is the purpose of virtuals,
allowing users to pick what they want to use to satisfy the
requirement.  I wouldn't want either rust in my world file as it is not
a package I use directly.  It should be a dependency of another package,
such as Firefox. To me, this sounds like there should be a note added to
a install guide so new users are aware of this, provided it isn't
already mentioned. 

Gentoo is known for compiling from sources.  Defaulting to a package
that is a binary but available as a package from source that requires
compiling, that would be unexpected. 

Just my $0.02 worth, as a user. 

Dale

:-)  :-) 

Re: [gentoo-dev] [RFC] making rust-bin ordered first in virtual/rust

[Ionen Wolkens]

[-- Attachment #1: Type: text/plain, Size: 1656 bytes --]

On Mon, Jan 17, 2022 at 03:24:23PM -0800, Georgy Yakovlev wrote:
> Hi,
> 
> I've been approached multiple times with that request, and a lot of
> time I see new users completely destroyed by rust build time and disk
> space requirements.

fwiw it may be a bit mitigated by the new desktop stages that come with
rust, or at least it won't be the first thing they see when they try
to build their brand new desktop install until a rust update.

It's nice if users can get a basic desktop before worrying about how to
handle rust. Not that desktop stages are heavily visible/known so new
users may not always pick them.

> 
> WDYT about switching order of rusts in a virtual?
> 
> RDEPEND="|| (
>         ~dev-lang/rust-${PV}
>         ~dev-lang/rust-bin-${PV}
> )"
> 
> 
> becomes
> 
> RDEPEND="|| (
>         ~dev-lang/rust-bin-${PV}
> 	~dev-lang/rust-${PV}
> )"
> 
> 
> Existing installs should be unaffected ofc.
> But portage may prefer to depclean rust and not rust-bin if both are
> present.

Haven't tested how it reacts, but wouldn't that be an issue with
system-bootstrap in situations where it pulls rust-bin to build itself?

> Users who wish to use source version at all times can just add it to
> world file.
> 
> I see both positives and negatives of doing that, but would like to
> reach out to community first.

Unsure what I like best, I generally agree should default to sources
but I do see new users complaining about building rust every few days.
Not that the step of telling them that rust-bin exists is that bad
(part of the issue is that they don't know it's an option).

-- 
ionen

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: [gentoo-dev] [RFC] making rust-bin ordered first in virtual/rust

[Georgy Yakovlev]

On Mon, 2022-01-17 at 21:58 -0500, Ionen Wolkens wrote:
> On Mon, Jan 17, 2022 at 03:24:23PM -0800, Georgy Yakovlev wrote:
> > Hi,
> > 
> > I've been approached multiple times with that request, and a lot of
> > time I see new users completely destroyed by rust build time and
> > disk
> > space requirements.
> 
> fwiw it may be a bit mitigated by the new desktop stages that come
> with
> rust, or at least it won't be the first thing they see when they try
> to build their brand new desktop install until a rust update.
> 
> It's nice if users can get a basic desktop before worrying about how
> to
> handle rust. Not that desktop stages are heavily visible/known so new
> users may not always pick them.
indeed it helps, but it takes time for users to discover/consider this.
it mostly hits very new users or users of containers/vms, who don't
have enough disk space or ram.
> 
> > 
> > WDYT about switching order of rusts in a virtual?
> > 
> > RDEPEND="|| (
> >         ~dev-lang/rust-${PV}
> >         ~dev-lang/rust-bin-${PV}
> > )"
> > 
> > 
> > becomes
> > 
> > RDEPEND="|| (
> >         ~dev-lang/rust-bin-${PV}
> >         ~dev-lang/rust-${PV}
> > )"
> > 
> > 
> > Existing installs should be unaffected ofc.
> > But portage may prefer to depclean rust and not rust-bin if both
> > are
> > present.
> 
> Haven't tested how it reacts, but wouldn't that be an issue with
> system-bootstrap in situations where it pulls rust-bin to build
> itself?
it can build itself with non-bin too. as long as it's same or ver-
minus-1 and is installed, if not installed it will pull rust-bin of
matching version.
also system-bootstrap is stable-masked (for various reasons), so I'm
not too worried about it. ~kw users know their way around.

> 
> > Users who wish to use source version at all times can just add it
> > to
> > world file.
> > 
> > I see both positives and negatives of doing that, but would like to
> > reach out to community first.
> 
> Unsure what I like best, I generally agree should default to sources
> but I do see new users complaining about building rust every few
> days.
> Not that the step of telling them that rust-bin exists is that bad
> (part of the issue is that they don't know it's an option).
> 

If I had clear answer myself I would not be asking opinions here =)
just trying to understand if I'm missing something or my view is too
different.
developer and user opinions all are helpful to build picture.
I receive at least weekly complaint about how resource-intensive rust
build is, that counts too.

for now on stable systems, if user types, let's say,
 $ emerge ripgrep

with USE=-system-bootsrap (the default):
portage will pull virtual/rust, which will pull dev-lang/rust, which
will bootstrap itself by downloading proper rust-bin tarball, but not
as a package, just as a tarball behind the scenes.

with USE=system-bootstrap:
portage will pull virtual/rust, which will pull dev-lang/rust-bin, and
build dev-lang/rust, to build ripgrep.

USE=system-llvm (also stable-masked) complicates picture even more.

We've had openjdk-bin and icedtea-bin as default provider for as long
as I can remember, before I started to maintain it. works fine.

PS:
and just to be clear - I'm not pushing for it, just asking for input.

Re: [gentoo-dev] [RFC] making rust-bin ordered first in virtual/rust

[Joonas Niilola]

[-- Attachment #1.1: Type: text/plain, Size: 716 bytes --]

On 18.1.2022 4.58, Ionen Wolkens wrote:
> 
> Unsure what I like best, I generally agree should default to sources
> but I do see new users complaining about building rust every few days.
> Not that the step of telling them that rust-bin exists is that bad
> (part of the issue is that they don't know it's an option).
> 

Indeed, can see both ways. On my desktop I use rust, and on my
laptop+containers rust-bin. Having to package.mask rust so rust-bin gets
pulled might not be the first thing new users are aware.

To make things more complicated...
IUSE="+binary"
RDEPEND="binary? ( ~dev-lang/rust-bin-${PV} )
	!binary? ( ~dev-lang/rust-${PV} )"

or "+pre-compiled" or something ;)

-- juippis

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 618 bytes --]

Re: [gentoo-dev] [RFC] making rust-bin ordered first in virtual/rust

[Jaco Kroon]

Hi,

On 2022/01/18 04:58, Ionen Wolkens wrote:

>
> Unsure what I like best, I generally agree should default to sources
> but I do see new users complaining about building rust every few days.
> Not that the step of telling them that rust-bin exists is that bad
> (part of the issue is that they don't know it's an option).
>
Perhaps output a notice in rust's pkg_setup() that you're using the
source version and that it takes a long time to build, and consumes lots
of resources and you may want to consider rust-bin?  Possibly with some
variable in make.conf or some USE to turn that off?

Kind Regards,
Jaco

Re: [gentoo-dev] [RFC] making rust-bin ordered first in virtual/rust

[Sam James]

[-- Attachment #1: Type: text/plain, Size: 1208 bytes --]



> On 17 Jan 2022, at 23:24, Georgy Yakovlev <gyakovlev@gentoo.org> wrote:
> 
> Hi,
> 
> I've been approached multiple times with that request, and a lot of
> time I see new users completely destroyed by rust build time and disk
> space requirements.
> 

I'll out myself as being one of these people!

> WDYT about switching order of rusts in a virtual?
> 
> RDEPEND="|| (
>        ~dev-lang/rust-${PV}
>        ~dev-lang/rust-bin-${PV}
> )"
> 
> 
> becomes
> 
> RDEPEND="|| (
>        ~dev-lang/rust-bin-${PV}
> 	~dev-lang/rust-${PV}
> )"
> 
> 
> Existing installs should be unaffected ofc.
> But portage may prefer to depclean rust and not rust-bin if both are
> present.
> Users who wish to use source version at all times can just add it to
> world file.
> 
> I see both positives and negatives of doing that, but would like to
> reach out to community first.

I'd like to do -bin first to be consistent with OpenJDK, IcedTea,
and to improve first-install experience.

Not that I'm advocating for removing source builds or anything
like that, but I dare say Rust is generally not something that people
can customise much anyway.

As Ionen has noted, we did already switch in desktop stages.

Best,
sam


[-- Attachment #2: Message signed with OpenPGP --]
[-- Type: application/pgp-signature, Size: 618 bytes --]

Re: [gentoo-dev] [RFC] making rust-bin ordered first in virtual/rust

[Toralf Förster]

[-- Attachment #1.1: Type: text/plain, Size: 168 bytes --]

On 1/18/22 18:13, Sam James wrote:
> Not that I'm advocating for removing source builds or anything
> like that,
I'm not ? ;)

-- 
Toralf
PGP 23217DA7 9B888F45

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 840 bytes --]

Re: [gentoo-dev] [RFC] making rust-bin ordered first in virtual/rust

[Piotr Karbowski]

On 18/01/2022 00.24, Georgy Yakovlev wrote:
> Hi,
> 
> I've been approached multiple times with that request, and a lot of
> time I see new users completely destroyed by rust build time and disk
> space requirements.
> 
> WDYT about switching order of rusts in a virtual?
> 
> RDEPEND="|| (
>          ~dev-lang/rust-${PV}
>          ~dev-lang/rust-bin-${PV}
> )"
> 
> 
> becomes
> 
> RDEPEND="|| (
>          ~dev-lang/rust-bin-${PV}
> 	~dev-lang/rust-${PV}
> )"
> 
> 
> Existing installs should be unaffected ofc.
> But portage may prefer to depclean rust and not rust-bin if both are
> present.
> Users who wish to use source version at all times can just add it to
> world file.
> 
> I see both positives and negatives of doing that, but would like to
> reach out to community first.

I would prefer to have source based one first.

Ideally we'd have some way to mark binary packages with new EAPI and 
have FEATURES flag like 'prefer-binary' and go with -bin in case there's 
|| ( ) dependencies list, regardless of the original order in virtual. 
This way everyone could be happy and not choose one workflow over another.

-- Piotr.

Re: [gentoo-dev] [RFC] making rust-bin ordered first in virtual/rust

[Toralf Förster]

[-- Attachment #1.1: Type: text/plain, Size: 245 bytes --]

On 1/20/22 22:10, Piotr Karbowski wrote:
> I and have FEATURES flag like 'prefer-binary' and go with -bin in case 
> there's || ( ) dependencies list, regardless of the original order in 
> virtual.
+1

-- 
Toralf
PGP 23217DA7 9B888F45

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 840 bytes --]

Re: [gentoo-dev] [RFC] making rust-bin ordered first in virtual/rust

[Rich Freeman]

On Thu, Jan 20, 2022 at 4:10 PM Piotr Karbowski <slashbeast@gentoo.org> wrote:
>
> Ideally we'd have some way to mark binary packages with new EAPI and
> have FEATURES flag like 'prefer-binary' and go with -bin in case there's
> || ( ) dependencies list, regardless of the original order in virtual.
> This way everyone could be happy and not choose one workflow over another.

Ideally we'd just have a repository of binary builds for everything
with default USE flags for a few profiles, and users could choose to
configure portage to just download the binary package if the flags
match, and of course this could be overridden per-package.  Then there
would be no need for -bin anything.  We have to maintain half of that
for the stage builds anyway.

I don't have a strong preference but I doubt that many of our users
receive any benefit whatsoever from building rust from source, and so
I'd suggest -bin ought to be the default.  However, having a
pre-installation notice that the -bin is available with a link/etc to
how to use it seems reasonable.  I bet a lot of users spend more time
getting the source build to work than it would take to just
reconfigure things so that they get the -bin.  If users are building
in a tmpfs or with numerous parallel jobs odds are that they'll end up
with a build failure for rust - CPU cores are way cheaper than RAM.

-- 
Rich

Re: [gentoo-dev] [RFC] making rust-bin ordered first in virtual/rust

[Joshua Kinard]

On 1/20/2022 16:32, Rich Freeman wrote:
> On Thu, Jan 20, 2022 at 4:10 PM Piotr Karbowski <slashbeast@gentoo.org> wrote:
>>
>> Ideally we'd have some way to mark binary packages with new EAPI and
>> have FEATURES flag like 'prefer-binary' and go with -bin in case there's
>> || ( ) dependencies list, regardless of the original order in virtual.
>> This way everyone could be happy and not choose one workflow over another.
> 
> Ideally we'd just have a repository of binary builds for everything
> with default USE flags for a few profiles, and users could choose to
> configure portage to just download the binary package if the flags
> match, and of course this could be overridden per-package.  Then there
> would be no need for -bin anything.  We have to maintain half of that
> for the stage builds anyway.
> 
[snip]

I've learned from running FreeBSD on some of my network equipment that this
is the path they take.  They have flags called USES on their Ports system,
and for binary packages, defaults are selected and used for building binary
packages.  In a handful of cases, such as mtr, they'll offer both an
X11-enabled version as well as a non-X11 version of the binpkg.  For all
other cases, it's fairly straight forward to build a few packages yourself
via Ports with your own USES selections.  E.g., I don't need python support
in bind-tools, so I build that one on my appliances myself to reduce the
required dependencies that need to be installed.

-- 
Joshua Kinard
Gentoo/MIPS
kumba@gentoo.org
rsa6144/5C63F4E3F5C6C943 2015-04-27
177C 1972 1FB8 F254 BAD0 3E72 5C63 F4E3 F5C6 C943

"The past tempts us, the present confuses us, the future frightens us.  And
our lives slip away, moment by moment, lost in that vast, terrible in-between."

--Emperor Turhan, Centauri Republic

Re: [gentoo-dev] [RFC] making rust-bin ordered first in virtual/rust

[Brian Evans]

[-- Attachment #1.1: Type: text/plain, Size: 677 bytes --]

On 1/17/2022 6:24 PM, Georgy Yakovlev wrote:
> Hi,
> 
> I've been approached multiple times with that request, and a lot of
> time I see new users completely destroyed by rust build time and disk
> space requirements.
> 

I bet that the pending polkit merge request[1] (which is just about 
ready and blessed but not merged by devs) would seriously drop many 
people's concerns over rust.

Using duktape on most systems will suffice for polkit once included.

GNOME and Mozilla products still pull in spidermonkey but other users 
will have a much reduced requirement for rust.

Brian

[1] https://gitlab.freedesktop.org/polkit/polkit/-/merge_requests/97


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 840 bytes --]

Re: [gentoo-dev] [RFC] making rust-bin ordered first in virtual/rust

[Michael Orlitzky]

On 2022-01-20 16:32:30, Brian Evans wrote:
> 
> GNOME and Mozilla products still pull in spidermonkey but other users 
> will have a much reduced requirement for rust.
> 

Avoiding librsvg used to be difficult because it's required by our GTK
icon packages to render PNGs from SVGs. Luckily dilfridge's binrepo
now makes it easy to download pre-rendered icons, and there's no
security/performance/legal concerns with pre-rendered PNGs, so using
them shouldn't present any moral dilemmata. You mainly just have to
replace Firefox, Thunderbird, and GIMP.

Re: [gentoo-dev] [RFC] making rust-bin ordered first in virtual/rust

[Joonas Niilola]

[-- Attachment #1.1: Type: text/plain, Size: 500 bytes --]

On 20.1.2022 23.32, Brian Evans wrote:
> 
> GNOME and Mozilla products still pull in spidermonkey but other users
> will have a much reduced requirement for rust.
> 

While Firefox and Thunderbird contain a js engine, no mozilla product
depends externally on spidermonkey. I was wondering recently why mozilla
project maintains it and not Gnome or Freedesktop, but I guess it's
because of the similarities shared by the ebuilds between firefox,
thunderbird and spidermonkey.

-- juippis

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 618 bytes --]

Re: [gentoo-dev] [RFC] making rust-bin ordered first in virtual/rust

[Mart Raudsepp]

[-- Attachment #1: Type: text/plain, Size: 1999 bytes --]

Ühel kenal päeval, N, 20.01.2022 kell 16:32, kirjutas Brian Evans:
> On 1/17/2022 6:24 PM, Georgy Yakovlev wrote:
> > Hi,
> > 
> > I've been approached multiple times with that request, and a lot of
> > time I see new users completely destroyed by rust build time and
> > disk
> > space requirements.
> > 
> 
> I bet that the pending polkit merge request[1] (which is just about 
> ready and blessed but not merged by devs) would seriously drop many 
> people's concerns over rust.
> 
> Using duktape on most systems will suffice for polkit once included.
> 
> GNOME and Mozilla products still pull in spidermonkey but other users
> will have a much reduced requirement for rust.

librsvg, dev-python/cryptography (masked so far but eventually needs to
move forard, albeit it looks like it won't have as big an impact as it
once would have), probably rather more to come.

Ühel kenal päeval, N, 20.01.2022 kell 20:01, kirjutas Michael Orlitzky:
> Avoiding librsvg used to be difficult because it's required by our
> GTK
> icon packages to render PNGs from SVGs. Luckily dilfridge's binrepo
> now makes it easy to download pre-rendered icons, and there's no
> security/performance/legal concerns with pre-rendered PNGs, so using
> them shouldn't present any moral dilemmata. You mainly just have to
> replace Firefox, Thunderbird, and GIMP.

And it is required on GTK itself to actually be able to render any SVG
icons. I can not suggest avoiding librsvg with GTK, nor let such
suggestions of such be uncountered.
rust is a toolchain, just like GCC, and if you are running a source-
based distribution, you get to have the toolchains required to compile
from source.


Anyhow, my vote is to default to rust-bin - people can easily be told
to move to dev-lang/rust at their convenience and then explicitly
depclean rust-bin. They can even use system-bootstrap on dev-lang/rust
with rust-bin to do so, in some situations (~arch-only apparently).

Mart

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 981 bytes --]

Re: [gentoo-dev] [RFC] making rust-bin ordered first in virtual/rust

[Peter Böhm]

Dear Developers,

I like your goal to make Gentoo more user-friendly but Gentoo is a source
based distribution and I dont like binary versions as a default. My question
is:

Who has problems with "big" packages like rust or firefox ?

Only User which doesnt know there is a binary version. So, in every case we
need to describe it in our AMD64 handbook.

Am Freitag, 21. Januar 2022, 10:22:14 CET schrieb Mart Raudsepp:
> Anyhow, my vote is to default to rust-bin - people can easily be told
> to move to dev-lang/rust at their convenience and then explicitly
> depclean rust-bin.

I am dreaming about another solution where this is not needed:

In our /etc/portage/make.conf we can have a new:

MAKEBIN="rust firefox"

... resulting in an automatic switch to the binary version of all included
packages ... of course this is also as recommendation in our AMD64 handbook
(with a clue to delete it if not desired).

Kind reagards,
Peter

Re: [gentoo-dev] [RFC] making rust-bin ordered first in virtual/rust

[Fabian Groffen]

[-- Attachment #1: Type: text/plain, Size: 1437 bytes --]

On 21-01-2022 11:29:54 +0100, Peter Böhm wrote:
> Dear Developers,
> 
> I like your goal to make Gentoo more user-friendly but Gentoo is a source
> based distribution and I dont like binary versions as a default. My question
> is:
> 
> Who has problems with "big" packages like rust or firefox ?
> 
> Only User which doesnt know there is a binary version. So, in every case we
> need to describe it in our AMD64 handbook.
> 
> Am Freitag, 21. Januar 2022, 10:22:14 CET schrieb Mart Raudsepp:
> > Anyhow, my vote is to default to rust-bin - people can easily be told
> > to move to dev-lang/rust at their convenience and then explicitly
> > depclean rust-bin.
> 
> I am dreaming about another solution where this is not needed:
> 
> In our /etc/portage/make.conf we can have a new:
> 
> MAKEBIN="rust firefox"
> 
> ... resulting in an automatic switch to the binary version of all included
> packages ... of course this is also as recommendation in our AMD64 handbook
> (with a clue to delete it if not desired).

or ... if we could have Portage check the requirements for building a
package, and if it cannot be met, that it tries to resolve the || case,
which would be the -bin variant in this case.

Not sure if the information is available to Portage at dependency
resolution time though.

Fabian

> 
> Kind reagards,
> Peter
> 
> 
> 
> 

-- 
Fabian Groffen
Gentoo on a different level

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: [gentoo-dev] [RFC] making rust-bin ordered first in virtual/rust

[Alec Warner]

On Mon, Jan 17, 2022 at 3:24 PM Georgy Yakovlev <gyakovlev@gentoo.org> wrote:
>
> Hi,
>
> I've been approached multiple times with that request, and a lot of
> time I see new users completely destroyed by rust build time and disk
> space requirements.

What does this actually mean? Can we more clearly document the negatives?

Is it that Rust takes a long time to build? This seems like just a
consequence of a sourced based distribution. Many things take a long
time to build. I suspect rust is not the only package and again we are
back to staffing binhost like projects for much of the tree so that
this particular complaint can be mitigated, not just for rust, but for
any package.

Is it that Rust build fails because users lack the resources to build
it? We should be using check-reqs here to detect this pre-build and
give the user choices (like telling them rust can't be compiled from
source on their machine and to use rust-bin.)

I tend to empathize more with Peter Böhm later in the thread, in that
Gentoo is sourced based and I'd expect portage to be installing things
from source by default. Building from source is resource intensive and
expensive, yes. I think if you want a binary-based distro you are
using the wrong distro at the moment.

-A

>
> WDYT about switching order of rusts in a virtual?
>
> RDEPEND="|| (
>         ~dev-lang/rust-${PV}
>         ~dev-lang/rust-bin-${PV}
> )"
>
>
> becomes
>
> RDEPEND="|| (
>         ~dev-lang/rust-bin-${PV}
>         ~dev-lang/rust-${PV}
> )"
>
>
> Existing installs should be unaffected ofc.
> But portage may prefer to depclean rust and not rust-bin if both are
> present.
> Users who wish to use source version at all times can just add it to
> world file.
>
> I see both positives and negatives of doing that, but would like to
> reach out to community first.
>
> Thanks!
>
> --
> Georgy
>