* [gentoo-dev] usr merge @ 2016-04-05 1:19 William Hubbs 2016-04-05 10:10 ` Alexis Ballier ` (4 more replies) 0 siblings, 5 replies; 89+ messages in thread From: William Hubbs @ 2016-04-05 1:19 UTC (permalink / raw To: gentoo development; +Cc: aballier [-- Attachment #1.1: Type: text/plain, Size: 1178 bytes --] All, I thought that since the usr merge is coming up again, and since I lost track of the message where it was brought up, I would open a new thread to discuss it. When it came up before, some were saying that the /usr merge violates the fhs. I don't remember the specifics of what the claim was at the time, (I'm sure someone will point it out if it is still a concern). I don't think creating usr merged stages would be that difficult. I think it would just be a matter of creating a new version of baselayout that puts these symlinks in place: /bin->usr/bin /lib->usr/lib /lib32->usr/lib32 /lib64->usr/lib64 /sbin->usr/bin /usr/sbin->bin Once that is in place in a new baselayout, I think portage's colission detection would be able to catch files that had the same names and were originally in different paths when building the new stages. I put some thought also in how to nigrate live systems, and I'm not sure what the best way to do that is. I wrote a script, which would do it in theory, but I haven't tested because I only have one system and if it breaks it the only way back would be to reinstall. The script is attached. Thoughts on any of this? William [-- Attachment #1.2: usrmerge --] [-- Type: text/plain, Size: 1232 bytes --] #!/bin/bb is_internal() { [ -z "$1" ] && return 1 case $(command -v $1) in */*) rc=1 ;; *) rc=0 ;; esac return $rc } run_command() { local dryrun= [ $DRYRUN -eq 1 ] && dryrun=echo $dryrun "$@" } for cmd in cp ln; do if ! is_internal $cmd; then echo "Please rebuild busybox and include the $cmd command." exit 1 fi done if [ -L /bin -a -L /sbin ]; then echo "It appears that the /usr merge has already been done on this system." exit 0 fi DRYRUN=1 HELP=0 while [ $# -gt 0 ]; do case $1 in --dryrun|--dry-run) DRYRUN=1 ;; -h|--help) HELP=1 ;; esac shift done if [ $HELP -eq 1 ]; then echo "$(basename $0) -h \| --help - displays this message" echo "$(basename $0) --dryrun \| --dry-run - show what would be done" exit 0 fi # copy binaries for dir in /bin /sbin /usr/sbin; do run_command cp -a $dir/* /usr/bin done # copy libraries for dir in /lib*; do [ -L $dir ] && continue run_command cp -a $dir/* /usr$dir done # Create the /usr merge compatibility symlinks for dir in /bin /sbin; do run_command rm -rf $dir run_command ln -s usr/bin $dir done run_command rm -rf /usr/sbin run_command ln -s bin /usr/sbin for dir in /lib*; do run_command rm -rf $dir run_command ln -s usr$dir $dir done [-- Attachment #2: Digital signature --] [-- Type: application/pgp-signature, Size: 181 bytes --] ^ permalink raw reply [flat|nested] 89+ messages in thread
* Re: [gentoo-dev] usr merge 2016-04-05 1:19 [gentoo-dev] usr merge William Hubbs @ 2016-04-05 10:10 ` Alexis Ballier 2016-04-05 12:26 ` [gentoo-dev] " Duncan 2016-04-06 4:15 ` [gentoo-dev] " Richard Yao ` (3 subsequent siblings) 4 siblings, 1 reply; 89+ messages in thread From: Alexis Ballier @ 2016-04-05 10:10 UTC (permalink / raw To: gentoo-dev On Tuesday, April 5, 2016 3:19:59 AM CEST, William Hubbs wrote: [...] > I don't think creating usr merged stages would be that difficult. I > think it would just be a matter of creating a new version of baselayout > that puts these symlinks in place: > > /bin->usr/bin > /lib->usr/lib > /lib32->usr/lib32 > /lib64->usr/lib64 (OT: maybe it'd be a good oportunity to kill SYMLINK_LIB too :p) > /sbin->usr/bin > /usr/sbin->bin > > Once that is in place in a new baselayout, I think portage's colission > detection would be able to catch files that had the same names and were > originally in different paths when building the new stages. I think that prior to that we have to ensure that packages with intra collisions are not merged: What happens with current coreutils ebuilds that install, e.g., /bin/seq and a /usr/bin/seq symlink to it ? I haven't looked at the actual code, thus I can only assume there are no guarantees, which is definitely bad. > I put some thought also in how to nigrate live systems, and I'm not sure > what the best way to do that is. I wrote a script, which would do it in > theory, but I haven't tested because I only have one system and if > it breaks it the only way back would be to reinstall. Does it behave properly wrt portage's way of tracking of package files? I remember that modifying files owned by portage used to cause issues. What should baselayout ebuild do on systems that have not run that script ? Also, I think your script may not work: # copy binaries for dir in /bin /sbin /usr/sbin; do run_command cp -a $dir/* /usr/bin done ---> Here it is important to ensure nothing /usr/bin/ is a symlink to /bin, otherwise this would just copy, e.g., /bin/seq onto /bin/seq # Create the /usr merge compatibility symlinks for dir in /bin /sbin; do run_command rm -rf $dir ---> where are the 'ln' and 'rm' taken from after this step ? If this fails here, you're also leaving the system in a broken state. run_command ln -s usr/bin $dir done ^ permalink raw reply [flat|nested] 89+ messages in thread
* [gentoo-dev] Re: usr merge 2016-04-05 10:10 ` Alexis Ballier @ 2016-04-05 12:26 ` Duncan 2016-04-05 16:53 ` [gentoo-dev] " Alexis Ballier 0 siblings, 1 reply; 89+ messages in thread From: Duncan @ 2016-04-05 12:26 UTC (permalink / raw To: gentoo-dev Alexis Ballier posted on Tue, 05 Apr 2016 12:10:51 +0200 as excerpted: > On Tuesday, April 5, 2016 3:19:59 AM CEST, William Hubbs wrote: > [...] >> I don't think creating usr merged stages would be that difficult. I >> think it would just be a matter of creating a new version of baselayout >> that puts these symlinks in place: >> >> /bin->usr/bin /lib->usr/lib /lib32->usr/lib32 /lib64->usr/lib64 > > (OT: maybe it'd be a good oportunity to kill SYMLINK_LIB too :p) > >> /sbin->usr/bin /usr/sbin->bin >> >> Once that is in place in a new baselayout, I think portage's colission >> detection would be able to catch files that had the same names and were >> originally in different paths when building the new stages. > > > I think that prior to that we have to ensure that packages with intra > collisions are not merged: What happens with current coreutils ebuilds > that install, e.g., /bin/seq and a /usr/bin/seq symlink to it ? > I haven't looked at the actual code, thus I can only assume there are no > guarantees, which is definitely bad. As I said in the other thread, I'm running merged /usr and bin/sbin here, except that I merged them the other way, with /usr -> . so everything in /usr is now in /. Portage has long "just worked" in that regard, tho I've no idea whether the other PMs do. Portage has enough intelligence to avoid replacing a file with a symlink pointing to it (and thus to itself once the replacement is done), regardless of which way the directory symlinks point. As such, coreutils "just works". If the two would end up in the same canonical location, the file wins and the symlink isn't installed. There are a few individual package bugs, including one open right now where the gcc-config ebuild does an unconditional rm -f of any old versions found in its old sbin location, even when it removes the executable it just installed into the bin location, because they're the same canonical location. (Bug number for that and other bugs in the reply on the other thread.) And cmake can get mixed up in some instances so a few packages (baloo) have problems with some cmake versions. But the bugs aren't with portage, they're with the ebuild or the upstream sources, and the number of them I've run into in the two years plus I've been running merged can fit on one hand. Certainly, they're few enough to deal with on a case-by-case basis. >> I put some thought also in how to nigrate live systems, and I'm not >> sure what the best way to do that is. I wrote a script, which would do >> it in theory, but I haven't tested because I only have one system and >> if it breaks it the only way back would be to reinstall. > > > Does it behave properly wrt portage's way of tracking of package files? > I remember that modifying files owned by portage used to cause issues. What I did for my migration was simply move everything from /usr to / and create the /usr -> . symlink. I did that with mc, and kept it running just in case I ended up not being able to start something, until I had the symlink in place and had tested starting various things, including X/ KDE, so I knew it was working. Similarly for the sbin -> bin moves and symlinks. The moves worked fine, and with the directory symlinks replacing the old dirs, everything else, including portage on updates after that, worked just fine. There are a couple things that behave slightly differently regarding packages, that one needs to be aware of, but in general it just works. Those couple things are: 1) Unless one is sure of the actual install path used and uses it, equery belongs and I assume q and similar tools with the same query, need the bare file name, not the full path, because you might use the wrong one. For instance, /bin/grep, /sbin/grep, /usr/sbin/grep, and /usr/bin/grep, are all the same file due to directory level symlinks. However, if you try equery belongs with all four paths, only one of them will return the hit you're looking for. Easily solved by simply doing equery belongs grep (no path), and letting equery supply the installed path in its results. That's actually how I find out which path a file was actually installed to, these days, as well. =:^) 2) revdep-rebuild will, in its default configuration, end up processing files using all paths. So grep, to use the same example as above, will be processed four times, one each for /bin and /sbin, /usr/bin and /usr/ sbin. While it's possible to reconfigure revdep-rebuild to ignore three of those paths and only check one of them (and similarly, ignore one of /lib64 and /usr/lib64, etc), doing so will result in revdep-rebuild complaining about unowned broken binaries if they're installed using a different path than the one it processed. That's not a big problem, because equery belongs <file> (without the path) will tell you what owns it as well as the installed path it used, and then that package can be remerged manually, if needed. So with revdep-rebuild, it's a tradeoff. You can either have it taking 4X as long to resolve executables and 2X as long to resolve libs, but handle the rebuilds automatically when it's done, or shorten its processing time by only processing one of the paths, and then have to figure out what owns some of the files it complains about and remerge them manually. (FWIW, I chose the short but often manual method when I first setup the merge, as the revdep-rebuild shell script was slow and processing things four times took way too long. But the newer python-based revdep-rebuild is far faster, and I've been going to switch back to letting it process all the paths now so it handles all its rebuilds automatically once again. But I've yet to get around to switching the config around, so it's still using the short but often semi-manual method.) > Also, I think your script may not work: > > # copy binaries for dir in /bin /sbin /usr/sbin; do > run_command cp -a $dir/* /usr/bin > done > > ---> Here it is important to ensure nothing /usr/bin/ is a symlink to > /bin, > otherwise this would just copy, e.g., /bin/seq onto /bin/seq Very good point! When I did it the migration here, I did the move (in mc) skipping files that existed in the new location. There were twenty or so skipped, IIRC, few enough to manually investigate. From memory, all of them were symlinks in one place or the other, so I could delete the symlinks and move the rest of the files over, without hitting skips the second time. Obviously a naive move will replace some of the files with dead (once replaced) symlinks, so valid point raised. =:^) > # Create the /usr merge compatibility symlinks for dir in /bin /sbin; do > run_command rm -rf $dir > > ---> where are the 'ln' and 'rm' taken from after this step ? > If this fails here, you're also leaving the system in a broken state. > > run_command ln -s usr/bin $dir done In my case I was using the mc binary, which continued to run after the transfer, so it wasn't an issue. But using the individual ln and rm binaries, while they'll still be on the path, you may need to run hash -r in the shell so it forgets the old location and checks the path again. Similar thing for the libs, since the lib cache will be screwed after the move, until the symlink has been created so the old paths work again, at least. In my case I was using the mc binary which continued to run and thus could be used to create the symlink, but for one-shot executables like ln, that could be an issue. One way around the problem would be to create a few static-linked executables for the purpose, and ship them in a tarball that's unpacked to an unchanging tmp location for the run, so they can be called from there to finish up regardless of whether the dynamically linked normal executables can still be invoked. Smart use of the shell's builtin read, echo and redirection could probably do some of it too, but can ln be emulated using shell builtins? -- Duncan - List replies preferred. No HTML msgs. "Every nonfree program has a lord, a master -- and if you use the program, he is your master." Richard Stallman ^ permalink raw reply [flat|nested] 89+ messages in thread
* Re: [gentoo-dev] usr merge 2016-04-05 12:26 ` [gentoo-dev] " Duncan @ 2016-04-05 16:53 ` Alexis Ballier 2016-04-06 0:06 ` [gentoo-dev] " Jonathan Callen 0 siblings, 1 reply; 89+ messages in thread From: Alexis Ballier @ 2016-04-05 16:53 UTC (permalink / raw To: gentoo-dev On Tuesday, April 5, 2016 2:26:53 PM CEST, Duncan wrote: > Alexis Ballier posted on Tue, 05 Apr 2016 12:10:51 +0200 as excerpted: > >> On Tuesday, April 5, 2016 3:19:59 AM CEST, William Hubbs wrote: >> [...] ... > > As I said in the other thread, I'm running merged /usr and bin/sbin here, > except that I merged them the other way, with /usr -> . so everything in > /usr is now in /. > > Portage has long "just worked" in that regard, tho I've no idea whether > the other PMs do. Portage has enough intelligence to avoid replacing a > file with a symlink pointing to it (and thus to itself once the > replacement is done), regardless of which way the directory symlinks > point. > > As such, coreutils "just works". If the two would end up in the same > canonical location, the file wins and the symlink isn't installed. What about the unlikely case with two files ? > There are a few individual package bugs, including one open right now > where the gcc-config ebuild does an unconditional rm -f of any old > versions found in its old sbin location, even when it removes the > executable it just installed into the bin location, because they're the > same canonical location. (Bug number for that and other bugs in the > reply on the other thread.) And cmake can get mixed up in some instances > so a few packages (baloo) have problems with some cmake versions. > > But the bugs aren't with portage, they're with the ebuild or the upstream > sources, and the number of them I've run into in the two years plus I've > been running merged can fit on one hand. Certainly, they're few enough > to deal with on a case-by-case basis. Yeah, these cases need to be handled on a case by case basis, there's no other choice anyway :) If we want to move on on this, we should definitely track these properly. [...] > 1) Unless one is sure of the actual install path used and uses it, equery > belongs and I assume q and similar tools with the same query, need the > bare file name, not the full path, because you might use the wrong one. > > For instance, /bin/grep, /sbin/grep, /usr/sbin/grep, and /usr/bin/grep, > are all the same file due to directory level symlinks. However, if you > try equery belongs with all four paths, only one of them will return the > hit you're looking for. > > Easily solved by simply doing equery belongs grep (no path), and letting > equery supply the installed path in its results. That's actually how I > find out which path a file was actually installed to, these days, as > well. =:^) no real issue here, and anyway, since it parses portage tree & profiles, support for guessing usr-merge can be added to restore old behavior > 2) revdep-rebuild will, in its default configuration, end up processing > files using all paths. So grep, to use the same example as above, will > be processed four times, one each for /bin and /sbin, /usr/bin and /usr/ > sbin. > > While it's possible to reconfigure revdep-rebuild to ignore three of > those paths and only check one of them (and similarly, ignore one of > /lib64 and /usr/lib64, etc), doing so will result in revdep-rebuild > complaining about unowned broken binaries if they're installed using a > different path than the one it processed. > > That's not a big problem, because equery belongs <file> (without the > path) will tell you what owns it as well as the installed path it used, > and then that package can be remerged manually, if needed. kind of defeats the point of revdep-rebuild though :) [...] >> # Create the /usr merge compatibility symlinks for dir in /bin /sbin; do >> run_command rm -rf $dir >> >> ---> where are the 'ln' and 'rm' taken from after this step ? >> If this fails here, you're also leaving the system in a broken state. >> >> run_command ln -s usr/bin $dir done > > In my case I was using the mc binary, which continued to run after the > transfer, so it wasn't an issue. > > But using the individual ln and rm binaries, while they'll still be on > the path, you may need to run hash -r in the shell so it forgets the old > location and checks the path again. > > Similar thing for the libs, since the lib cache will be screwed after the > move, until the symlink has been created so the old paths work again, at > least. In my case I was using the mc binary which continued to run and > thus could be used to create the symlink, but for one-shot executables > like ln, that could be an issue. > > One way around the problem would be to create a few static-linked > executables for the purpose, and ship them in a tarball that's unpacked > to an unchanging tmp location for the run, so they can be called from > there to finish up regardless of whether the dynamically linked normal > executables can still be invoked. > > Smart use of the shell's builtin read, echo and redirection could > probably do some of it too, but can ln be emulated using shell builtins? > I'd rather use a binary that ensures everything that needs to be loaded is loaded at the beginning and even better if it can enusre system consistency and can do rollbacks in case of failure. A python script would probably work if there is no dynamic import, or a C version that doesnt use dlopen() nor execv() & friends. It's probably even easier to check that / and /usr are on the same mount point. Alexis. ^ permalink raw reply [flat|nested] 89+ messages in thread
* [gentoo-dev] Re: usr merge 2016-04-05 16:53 ` [gentoo-dev] " Alexis Ballier @ 2016-04-06 0:06 ` Jonathan Callen 0 siblings, 0 replies; 89+ messages in thread From: Jonathan Callen @ 2016-04-06 0:06 UTC (permalink / raw To: gentoo-dev -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA512 On 04/05/2016 12:53 PM, Alexis Ballier wrote: > On Tuesday, April 5, 2016 2:26:53 PM CEST, Duncan wrote: >> >> As I said in the other thread, I'm running merged /usr and >> bin/sbin here, except that I merged them the other way, with /usr >> -> . so everything in /usr is now in /. >> >> Portage has long "just worked" in that regard, tho I've no idea >> whether the other PMs do. Portage has enough intelligence to >> avoid replacing a file with a symlink pointing to it (and thus to >> itself once the replacement is done), regardless of which way the >> directory symlinks point. >> >> As such, coreutils "just works". If the two would end up in the >> same canonical location, the file wins and the symlink isn't >> installed. > > What about the unlikely case with two files ? > Having actually run this way myself, I did find one case that I haven't filed a bug for yet: the plymouth ebuild tries to install symlinks in /sbin pointing at /usr/sbin, and portage chose to install the symlinks instead of the real files, for whatever reason (apparently because the $ED/sbin directory is created after the $ED/usr directory). Because of this, it might be best to ensure that packages that do install in both places are modified not to do so under such a configuration. - -- Jonathan Callen -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iQIcBAEBCgAGBQJXBFMbAAoJEEIQbvYRB3mg2e4P/2lPBxpyjY311LP7gN2Nndn4 Dd4EtFbh8tQWoedPJQgr2CIeVgpPFA7l/stuvcoZAqLVDuFnn4ZmMWSIQOgHmgPp +mIiCDPuLMjhqw/yINlTGGVVhffHFG4PrHcd2MwP6Gm9ME0NH8+Z0cgAznHsHQ5c lgNdfXDsgBdrSrKu5/JTw7jDOv1A1TwIACJoLpEYZTlVCBClp6J01kqH1oyEzPf8 FO6fqAvFJXCq1um6/+ve8LOpS0OLBpg0dh5kcdkFgV1430FqNwUczMINhav5J0mp qTAIZTO4OSLxyswOUiKoxROl4xrQ1ByYi1ZF7g24oh7M1fmkreNClrhJ1kA3M6ff OJ3LJ6m350LEIVzAED66pnKOTNDOLJSaz6MsPk8CHzuJ2RCMatKjBA3Lb0tkkepp 5LOCBXbnVfSPRI+TQM91cHXVnh87T1zZSeGT8qOCfNoF7rFWNSlpIRnxMeeFlv2n 0kXfJo9YeiUAA9BYXBryMIsWr4StM4I9oq0ITc7h9WmB/WKW6zJhl7WHd7SgiePW Lb2fHJtz0R8dUIc53Yxuls1Cbt8AUAFYmN9Ve615cVLs3+jO8HWmwiuFfiYH71k1 JaS51cgBjPBnQuiET0iNxu/gjIekwIjoNptn/cCr9IZ4jnZ9L13ai6Wug49vUwwK bed4Tt3nl8GSbRtlV+rk =PHpB -----END PGP SIGNATURE----- ^ permalink raw reply [flat|nested] 89+ messages in thread
* Re: [gentoo-dev] usr merge 2016-04-05 1:19 [gentoo-dev] usr merge William Hubbs 2016-04-05 10:10 ` Alexis Ballier @ 2016-04-06 4:15 ` Richard Yao 2016-04-06 5:34 ` [gentoo-dev] " Duncan ` (2 more replies) 2016-04-06 14:58 ` M. J. Everitt ` (2 subsequent siblings) 4 siblings, 3 replies; 89+ messages in thread From: Richard Yao @ 2016-04-06 4:15 UTC (permalink / raw To: gentoo-dev; +Cc: aballier > On Apr 4, 2016, at 9:19 PM, William Hubbs <williamh@gentoo.org> wrote: > > All, > > I thought that since the usr merge is coming up again, and since I lost > track of the message where it was brought up, I would open a > new thread to discuss it. > > When it came up before, some were saying that the /usr merge violates > the fhs. I don't remember the specifics of what the claim was at the > time, (I'm sure someone will point it out if it is still a concern). Here are the violations: http://refspecs.linuxfoundation.org/FHS_3.0/fhs-3.0.html#binEssentialUserCommandBinaries http://refspecs.linuxfoundation.org/FHS_3.0/fhs-3.0.html#sbinSystemBinaries http://refspecs.linuxfoundation.org/FHS_3.0/fhs-3.0.html#libEssentialSharedLibrariesAndKern > > I don't think creating usr merged stages would be that difficult. I > think it would just be a matter of creating a new version of baselayout > that puts these symlinks in place: > > /bin->usr/bin > /lib->usr/lib > /lib32->usr/lib32 > /lib64->usr/lib64 > /sbin->usr/bin > /usr/sbin->bin > > Once that is in place in a new baselayout, I think portage's colission > detection would be able to catch files that had the same names and were > originally in different paths when building the new stages. We will have users whose system configurations rely on the FHS complain about us breaking boot if we force this. > I put some thought also in how to nigrate live systems, and I'm not sure > what the best way to do that is. I wrote a script, which would do it in > theory, but I haven't tested because I only have one system and if > it breaks it the only way back would be to reinstall. > > The script is attached. > > > Thoughts on any of this? This was invented in Solaris and copied by RHEL. The upgrade path for the /usr merge on those systems is a complete reinstall. Upgrading from RHEL6 to RHEL7 this Solaris 10 to Solaris 11 is not supported. The reason being that there are ways of configuring the system boot process with the original layout that break if you try using scripts to migrate to the new one. A USE flag for the /usr merge that is off by default would allow us to have both worlds without putting any systems at risk. This has been an almost annual debate. I do not have much incentive to keep up with it. The reason I ever bothered to explain why this is a bad idea for Gentoo was that I was concerned for our user base. My systems would not be negatively affected and arguing against changes that originated in Solaris is awkward for me. If others are not willing to be advocates for those users that would only make themselves known after an a fundamental change has been made and people are determined to go ahead with this, I suggest having and testing a plan for backing out the change should the backlash from users after systems break be more than people can stomach. This is not the sort of change we should make without an "exit strategy". > William > > <usrmerge> ^ permalink raw reply [flat|nested] 89+ messages in thread
* [gentoo-dev] Re: usr merge 2016-04-06 4:15 ` [gentoo-dev] " Richard Yao @ 2016-04-06 5:34 ` Duncan 2016-04-06 14:32 ` Richard Yao 2016-04-06 7:42 ` [gentoo-dev] " Alexis Ballier 2016-04-06 17:06 ` waltdnes 2 siblings, 1 reply; 89+ messages in thread From: Duncan @ 2016-04-06 5:34 UTC (permalink / raw To: gentoo-dev Richard Yao posted on Wed, 06 Apr 2016 00:15:58 -0400 as excerpted: >> On Apr 4, 2016, at 9:19 PM, William Hubbs <williamh@gentoo.org> wrote: >> >> All, >> >> I thought that since the usr merge is coming up again, and since I lost >> track of the message where it was brought up, I would open a new thread >> to discuss it. >> >> When it came up before, some were saying that the /usr merge violates >> the fhs. I don't remember the specifics of what the claim was at the >> time, (I'm sure someone will point it out if it is still a concern). > > Here are the violations: > > http://refspecs.linuxfoundation.org/FHS_3.0/ fhs-3.0.html#binEssentialUserCommandBinaries > > http://refspecs.linuxfoundation.org/FHS_3.0/ fhs-3.0.html#sbinSystemBinaries > > http://refspecs.linuxfoundation.org/FHS_3.0/ fhs-3.0.html#libEssentialSharedLibrariesAndKern (Those links are wrapped and I'm not bothering to jump thru the hoops to unwrap them, since readers can either unwrap them manually or refer to the parent post I'm quoting for the unwrapped versions.) If those are the "violations", then putting everything in /usr and making the /bin and /sbin locations symlinks isn't going to be a problem, since /bin and /sbin are specifically allowed to contain symlinks to the executables, instead of the executables themselves, and if the dirs themselves are symlinks to the locations in /usr with the files, that fulfills that requirement. And the requirement for /lib is rather vague, saying only that it contains the libs linked by the executables in /bin and /sbin. So once / bin and /sbin are symlinks to the dirs with the executables, /lib (or the arch-specific alternative libdirs) can be a symlink as well. Tho I must say doing the reverse, making either /usr itself or /usr/bin and /usr/sbin symlinks to the root dirs, as I did here, actually makes more sense and bends the rules less. Basically, what the FHS says, at least in the 3.0 version you linked, is that the executables must be reachable via whatever specific path, but using symlinks to do it is fine. Which means the merge is allowed, as long as symlinks allow the executables to be reached by their specifically defined paths. And I'm not aware of anyone seriously proposing that said symlinks be omitted, so... -- Duncan - List replies preferred. No HTML msgs. "Every nonfree program has a lord, a master -- and if you use the program, he is your master." Richard Stallman ^ permalink raw reply [flat|nested] 89+ messages in thread
* Re: [gentoo-dev] Re: usr merge 2016-04-06 5:34 ` [gentoo-dev] " Duncan @ 2016-04-06 14:32 ` Richard Yao 0 siblings, 0 replies; 89+ messages in thread From: Richard Yao @ 2016-04-06 14:32 UTC (permalink / raw To: gentoo-dev On 04/06/2016 01:34 AM, Duncan wrote: > Richard Yao posted on Wed, 06 Apr 2016 00:15:58 -0400 as excerpted: > > >>> On Apr 4, 2016, at 9:19 PM, William Hubbs <williamh@gentoo.org> wrote: >>> >>> All, >>> >>> I thought that since the usr merge is coming up again, and since I lost >>> track of the message where it was brought up, I would open a new thread >>> to discuss it. >>> >>> When it came up before, some were saying that the /usr merge violates >>> the fhs. I don't remember the specifics of what the claim was at the >>> time, (I'm sure someone will point it out if it is still a concern). >> >> Here are the violations: >> >> http://refspecs.linuxfoundation.org/FHS_3.0/ > fhs-3.0.html#binEssentialUserCommandBinaries >> >> http://refspecs.linuxfoundation.org/FHS_3.0/ > fhs-3.0.html#sbinSystemBinaries >> >> http://refspecs.linuxfoundation.org/FHS_3.0/ > fhs-3.0.html#libEssentialSharedLibrariesAndKern > > (Those links are wrapped and I'm not bothering to jump thru the hoops to > unwrap them, since readers can either unwrap them manually or refer to > the parent post I'm quoting for the unwrapped versions.) > > If those are the "violations", then putting everything in /usr and making > the /bin and /sbin locations symlinks isn't going to be a problem, since > /bin and /sbin are specifically allowed to contain symlinks to the > executables, instead of the executables themselves, and if the dirs > themselves are symlinks to the locations in /usr with the files, that > fulfills that requirement. Alexis Ballier pointed out that the following had been added to make this okay: > The following directories, or symbolic links to directories, are required in /. As far as the Single UNIX Specification is concerned, this sort of thing is okay. The Linux Filesystem Hierarchy Standard that further restricts things had not been updated to permit this until mid-2015. > And the requirement for /lib is rather vague, saying only that it > contains the libs linked by the executables in /bin and /sbin. So once / > bin and /sbin are symlinks to the dirs with the executables, /lib (or the > arch-specific alternative libdirs) can be a symlink as well. > > Tho I must say doing the reverse, making either /usr itself or /usr/bin > and /usr/sbin symlinks to the root dirs, as I did here, actually makes > more sense and bends the rules less. I agree, but Solaris picked the other way and RHEL followed. My opinion is that we should support both. Any automated upgrade scripts that do the /usr merge on behalf of users should detect this state and leave it alone. That being said, I am not of the opinion that we should migrate existing installs or drop support for the older layout for a couple reasons. One is that existing installs will see no benefit from this layout change. The other is that supporting the existing installs is necessary so that we can still do QA on them. I suspect breakage will be more likely with the older layout than the newer one. If we end up not migrating existing installs automatically (maybe via a profile change), we might be able to get away with doing the majority of testing with the older one. If we do not want to do that long term, we could always do it for something like a year to minimize the effect on the userbase. A portage QA check could be made to detect the presence of files in both /{bin,sbin,lib,lib64,lib32} and /usr/{bin,sbin,lib,lib64,lib32} to make verifying that packaging does not do anything weird that would negatively affect the newer layout easier when testing with the older layout. The only exception being the linker scripts. There are at least a few of them still around: richard@desktop ~ $ for fA in /lib/*; do > fB=/usr/lib/${fA##*/}; > [[ -f $fA && -f $fB ]] && echo "$fB"; > done /usr/lib/libeinfo.so /usr/lib/libpamc.so /usr/lib/libpam_misc.so /usr/lib/libpam.so /usr/lib/librc.so richard@desktop ~ $ cat /usr/lib/librc.so /* GNU ld script Since Gentoo has critical dynamic libraries in /lib, and the static versions in /usr/lib, we need to have a "fake" dynamic lib in /usr/lib, otherwise we run into linking problems. This "fake" dynamic lib is a linker script that redirects the linker to the real lib. And yes, this works in the cross- compiling scenario as the sysroot-ed linker will prepend the real path. See bug https://bugs.gentoo.org/4411 for more info. */ OUTPUT_FORMAT ( elf64-x86-64 ) GROUP ( /lib64/librc.so ) I am a bit out of date on the status of these. Do we still need them? If we have not already, we could patch GNU ld to eliminate the need for this. > Basically, what the FHS says, at least in the 3.0 version you linked, is > that the executables must be reachable via whatever specific path, but > using symlinks to do it is fine. > > Which means the merge is allowed, as long as symlinks allow the > executables to be reached by their specifically defined paths. And I'm > not aware of anyone seriously proposing that said symlinks be omitted, > so... My mistake. I had not spotted that change in my cursory read. At least we now know that this is not a Linux FHS violation anymore. :) ^ permalink raw reply [flat|nested] 89+ messages in thread
* Re: [gentoo-dev] usr merge 2016-04-06 4:15 ` [gentoo-dev] " Richard Yao 2016-04-06 5:34 ` [gentoo-dev] " Duncan @ 2016-04-06 7:42 ` Alexis Ballier 2016-04-06 8:55 ` James Le Cuirot 2016-04-06 14:04 ` Richard Yao 2016-04-06 17:06 ` waltdnes 2 siblings, 2 replies; 89+ messages in thread From: Alexis Ballier @ 2016-04-06 7:42 UTC (permalink / raw To: gentoo-dev On Wednesday, April 6, 2016 6:15:58 AM CEST, Richard Yao wrote: >> On Apr 4, 2016, at 9:19 PM, William Hubbs <williamh@gentoo.org> wrote: >> >> All, >> >> I thought that since the usr merge is coming up again, and since I lost >> track of the message where it was brought up, I would open a >> new thread to discuss it. ... > > Here are the violations: > > http://refspecs.linuxfoundation.org/FHS_3.0/fhs-3.0.html#binEssentialUserCommandBinaries > > http://refspecs.linuxfoundation.org/FHS_3.0/fhs-3.0.html#sbinSystemBinaries > > http://refspecs.linuxfoundation.org/FHS_3.0/fhs-3.0.html#libEssentialSharedLibrariesAndKern > well, those are not violations: fhs mandates a certain set of binaries in those paths; this is still the case with a usr-merged system. i thought the symlinks would be a problem, but fhs states: > The following directories, or symbolic links to directories, are required in /. so, really, i dont see any violation there >> I don't think creating usr merged stages would be that difficult. I >> think it would just be a matter of creating a new version of baselayout >> that puts these symlinks in place: >> >> /bin->usr/bin >> /lib->usr/lib ... > > We will have users whose system configurations rely on the FHS > complain about us breaking boot if we force this. i dont think anybody is talking about forcing this >> I put some thought also in how to nigrate live systems, and I'm not sure >> what the best way to do that is. I wrote a script, which would do it in >> theory, but I haven't tested because I only have one system and if >> it breaks it the only way back would be to reinstall. >> >> The script is attached. >> >> >> Thoughts on any of this? > > > This was invented in Solaris and copied by RHEL. The upgrade > path for the /usr merge on those systems is a complete > reinstall. Upgrading from RHEL6 to RHEL7 this Solaris 10 to > Solaris 11 is not supported. The reason being that there are > ways of configuring the system boot process with the original > layout that break if you try using scripts to migrate to the new > one. A USE flag for the /usr merge that is off by default would > allow us to have both worlds without putting any systems at > risk. that's what i'm actually more worried about: the fact they failed to have a proper upgrade path doesnt mean it is impossible, just that it is not easy. only being able to propose usr-merge for new installs makes it useless for gentoo imho: i, for once, use only 1 gentoo install for the lifetime of my / disk... note that being able to properly migrate is a *requirement* for having it as a useflag > If others are not willing to be advocates for those users that > would only make themselves known after an a fundamental change > has been made and people are determined to go ahead with this, I > suggest having and testing a plan for backing out the change > should the backlash from users after systems break be more than > people can stomach. This is not the sort of change we should > make without an "exit strategy". ironing out that kind of strategy is the point of this thread :) Alexis. ^ permalink raw reply [flat|nested] 89+ messages in thread
* Re: [gentoo-dev] usr merge 2016-04-06 7:42 ` [gentoo-dev] " Alexis Ballier @ 2016-04-06 8:55 ` James Le Cuirot 2016-04-06 14:06 ` Richard Yao 2016-04-06 14:04 ` Richard Yao 1 sibling, 1 reply; 89+ messages in thread From: James Le Cuirot @ 2016-04-06 8:55 UTC (permalink / raw To: gentoo-dev On Wed, 06 Apr 2016 09:42:04 +0200 Alexis Ballier <aballier@gentoo.org> wrote: > > This was invented in Solaris and copied by RHEL. The upgrade > > path for the /usr merge on those systems is a complete > > reinstall. Upgrading from RHEL6 to RHEL7 this Solaris 10 to > > Solaris 11 is not supported. The reason being that there are > > ways of configuring the system boot process with the original > > layout that break if you try using scripts to migrate to the new > > one. A USE flag for the /usr merge that is off by default would > > allow us to have both worlds without putting any systems at > > risk. > > that's what i'm actually more worried about: the fact they failed to > have a proper upgrade path doesnt mean it is impossible, just that it > is not easy. What about Fedora? This system I'm on now started as Fedora 16 and has been upgraded step by step to 23. /bin, /lib, /lib64, and /sbin are symlinks but I'm pretty sure it didn't start out that way. I knew the change was coming but when it actually happened, I didn't notice for quite a while. -- James Le Cuirot (chewi) Gentoo Linux Developer ^ permalink raw reply [flat|nested] 89+ messages in thread
* Re: [gentoo-dev] usr merge 2016-04-06 8:55 ` James Le Cuirot @ 2016-04-06 14:06 ` Richard Yao 0 siblings, 0 replies; 89+ messages in thread From: Richard Yao @ 2016-04-06 14:06 UTC (permalink / raw To: gentoo-dev > On Apr 6, 2016, at 4:55 AM, James Le Cuirot <chewi@gentoo.org> wrote: > > On Wed, 06 Apr 2016 09:42:04 +0200 > Alexis Ballier <aballier@gentoo.org> wrote: > >>> This was invented in Solaris and copied by RHEL. The upgrade >>> path for the /usr merge on those systems is a complete >>> reinstall. Upgrading from RHEL6 to RHEL7 this Solaris 10 to >>> Solaris 11 is not supported. The reason being that there are >>> ways of configuring the system boot process with the original >>> layout that break if you try using scripts to migrate to the new >>> one. A USE flag for the /usr merge that is off by default would >>> allow us to have both worlds without putting any systems at >>> risk. >> >> that's what i'm actually more worried about: the fact they failed to >> have a proper upgrade path doesnt mean it is impossible, just that it >> is not easy. > > What about Fedora? This system I'm on now started as Fedora 16 and has > been upgraded step by step to 23. /bin, /lib, /lib64, and /sbin are > symlinks but I'm pretty sure it didn't start out that way. I knew the > change was coming but when it actually happened, I didn't notice for > quite a while. The common case can be done by automated scripts. The case where the system is configured to mount /usr after init has started is not. In Fedora's case, they tell users to expect upgrades to break things, so the people bitten by it at least had been warned of breakage before they installed Fedora. > > -- > James Le Cuirot (chewi) > Gentoo Linux Developer > ^ permalink raw reply [flat|nested] 89+ messages in thread
* Re: [gentoo-dev] usr merge 2016-04-06 7:42 ` [gentoo-dev] " Alexis Ballier 2016-04-06 8:55 ` James Le Cuirot @ 2016-04-06 14:04 ` Richard Yao 2016-04-06 14:48 ` Alexis Ballier ` (2 more replies) 1 sibling, 3 replies; 89+ messages in thread From: Richard Yao @ 2016-04-06 14:04 UTC (permalink / raw To: gentoo-dev > On Apr 6, 2016, at 3:42 AM, Alexis Ballier <aballier@gentoo.org> wrote: > > On Wednesday, April 6, 2016 6:15:58 AM CEST, Richard Yao wrote: >>> On Apr 4, 2016, at 9:19 PM, William Hubbs <williamh@gentoo.org> wrote: >>> All, >>> I thought that since the usr merge is coming up again, and since I lost >>> track of the message where it was brought up, I would open a >>> new thread to discuss it. ... >> >> Here are the violations: >> >> http://refspecs.linuxfoundation.org/FHS_3.0/fhs-3.0.html#binEssentialUserCommandBinaries >> >> http://refspecs.linuxfoundation.org/FHS_3.0/fhs-3.0.html#sbinSystemBinaries >> >> http://refspecs.linuxfoundation.org/FHS_3.0/fhs-3.0.html#libEssentialSharedLibrariesAndKern > > well, those are not violations: fhs mandates a certain set of binaries in those paths; this is still the case with a usr-merged system. > > i thought the symlinks would be a problem, but fhs states: >> The following directories, or symbolic links to directories, are required in /. > > so, really, i dont see any violation there Nice. They added that to fix it. > > >>> I don't think creating usr merged stages would be that difficult. I >>> think it would just be a matter of creating a new version of baselayout >>> that puts these symlinks in place: >>> /bin->usr/bin >>> /lib->usr/lib ... >> >> We will have users whose system configurations rely on the FHS complain about us breaking boot if we force this. > > > i dont think anybody is talking about forcing this > > >>> I put some thought also in how to nigrate live systems, and I'm not sure >>> what the best way to do that is. I wrote a script, which would do it in >>> theory, but I haven't tested because I only have one system and if >>> it breaks it the only way back would be to reinstall. >>> The script is attached. >>> Thoughts on any of this? >> >> >> This was invented in Solaris and copied by RHEL. The upgrade path for the /usr merge on those systems is a complete reinstall. Upgrading from RHEL6 to RHEL7 this Solaris 10 to Solaris 11 is not supported. The reason being that there are ways of configuring the system boot process with the original layout that break if you try using scripts to migrate to the new one. A USE flag for the /usr merge that is off by default would allow us to have both worlds without putting any systems at risk. > > that's what i'm actually more worried about: the fact they failed to have a proper upgrade path doesnt mean it is impossible, just that it is not easy. > > only being able to propose usr-merge for new installs makes it useless for gentoo imho: i, for once, use only 1 gentoo install for the lifetime of my / disk... > > > note that being able to properly migrate is a *requirement* for having it as a useflag Most systems would switch fine. The ones configured to depend on /usr not being mounted in early boot would not. That is the reason automatically migrating people is not the best idea. That being said, this is only useful for new installs where people want to take advantage of the Solaris way of doing management. It should have no benefit for existing installs. > >> If others are not willing to be advocates for those users that would only make themselves known after an a fundamental change has been made and people are determined to go ahead with this, I suggest having and testing a plan for backing out the change should the backlash from users after systems break be more than people can stomach. This is not the sort of change we should make without an "exit strategy". > > > ironing out that kind of strategy is the point of this thread :) > > Alexis. > ^ permalink raw reply [flat|nested] 89+ messages in thread
* Re: [gentoo-dev] usr merge 2016-04-06 14:04 ` Richard Yao @ 2016-04-06 14:48 ` Alexis Ballier 2016-04-06 22:01 ` [gentoo-dev] " Duncan 2016-04-07 11:27 ` [gentoo-dev] " Tom H 2 siblings, 0 replies; 89+ messages in thread From: Alexis Ballier @ 2016-04-06 14:48 UTC (permalink / raw To: gentoo-dev On Wednesday, April 6, 2016 4:04:05 PM CEST, Richard Yao wrote: > Most systems would switch fine. The ones configured to depend > on /usr not being mounted in early boot would not. That is the > reason automatically migrating people is not the best idea. I believe any script/binary should not run automagically but rather provided for manual update, and also bail out early if any of the paths it'd touch are not on the same filesystem. This would exclude /usr on a separate partition systems. ^ permalink raw reply [flat|nested] 89+ messages in thread
* [gentoo-dev] Re: usr merge 2016-04-06 14:04 ` Richard Yao 2016-04-06 14:48 ` Alexis Ballier @ 2016-04-06 22:01 ` Duncan 2016-04-07 11:27 ` [gentoo-dev] " Tom H 2 siblings, 0 replies; 89+ messages in thread From: Duncan @ 2016-04-06 22:01 UTC (permalink / raw To: gentoo-dev Richard Yao posted on Wed, 06 Apr 2016 10:04:05 -0400 as excerpted: > That being said, this is only useful for new installs where people want > to take advantage of the Solaris way of doing management. It should have > no benefit for existing installs. I don't know enough about solaris to comment on that, but my (reverse) merging /usr and bin/sbin to / certainly had benefits for my existing install. The biggest one was no longer having the brain overhead of having to track whether something's in /usr or direct in /, or in the bin or sbin location in /usr or /. If it's an on-path executable that I didn't manually create/install myself, it's now in /bin as the fully dereferenced canonical path, tho /usr/bin /sbin, and /usr/sbin also work via symlinks, no questions asked. Similarly, libs are found in /lib64 as the fully dereferenced canonical path, tho /lib, /usr/lib64 and /usr/lib all work as well, via symlinks. =:^) There remains a slight down side in that the PM's idea of where the files are located may differ in that it's one of the symlinked versions, and various standard paths are slightly less efficient due to having to dereference possibly multiple symlinks, but automatic and fast tracking of such things while minimizing the wetware tracking load is what computers excel at, so on balance I consider it a pretty large benefit. So there's certainly benefit for existing installs. =:^) -- Duncan - List replies preferred. No HTML msgs. "Every nonfree program has a lord, a master -- and if you use the program, he is your master." Richard Stallman ^ permalink raw reply [flat|nested] 89+ messages in thread
* Re: [gentoo-dev] usr merge 2016-04-06 14:04 ` Richard Yao 2016-04-06 14:48 ` Alexis Ballier 2016-04-06 22:01 ` [gentoo-dev] " Duncan @ 2016-04-07 11:27 ` Tom H 2 siblings, 0 replies; 89+ messages in thread From: Tom H @ 2016-04-07 11:27 UTC (permalink / raw To: gentoo-dev On Wed, Apr 6, 2016 at 4:04 PM, Richard Yao <ryao@gentoo.org> wrote: > On Apr 6, 2016, at 3:42 AM, Alexis Ballier <aballier@gentoo.org> wrote: >> On Wednesday, April 6, 2016 6:15:58 AM CEST, Richard Yao wrote: >>> >>> Here are the violations: >>> >>> http://refspecs.linuxfoundation.org/FHS_3.0/fhs-3.0.html#binEssentialUserCommandBinaries >>> >>> http://refspecs.linuxfoundation.org/FHS_3.0/fhs-3.0.html#sbinSystemBinaries >>> >>> http://refspecs.linuxfoundation.org/FHS_3.0/fhs-3.0.html#libEssentialSharedLibrariesAndKern >> >> well, those are not violations: fhs mandates a certain set of >> binaries in those paths; this is still the case with a usr-merged >> system. >> >> i thought the symlinks would be a problem, but fhs states: >>> >>> The following directories, or symbolic links to directories, are required in /. >> >> so, really, i dont see any violation there > > Nice. They added that to fix it. More likely you missed it in the past because 2004's FHS 2.3 has "The following directories, or symbolic links to directories, are required in /." in http://refspecs.linuxfoundation.org/FHS_2.3/fhs-2.3.html#REQUIREMENTS ^ permalink raw reply [flat|nested] 89+ messages in thread
* Re: [gentoo-dev] usr merge 2016-04-06 4:15 ` [gentoo-dev] " Richard Yao 2016-04-06 5:34 ` [gentoo-dev] " Duncan 2016-04-06 7:42 ` [gentoo-dev] " Alexis Ballier @ 2016-04-06 17:06 ` waltdnes 2 siblings, 0 replies; 89+ messages in thread From: waltdnes @ 2016-04-06 17:06 UTC (permalink / raw To: gentoo-dev On Wed, Apr 06, 2016 at 12:15:58AM -0400, Richard Yao wrote > If others are not willing to be advocates for ***THOSE USERS THAT WOULD > ONLY MAKE THEMSELVES KNOWN AFTER AN A FUNDAMENTAL CHANGE HAS BEEN MADE > AND PEOPLE ARE DETERMINED TO GO AHEAD WITH THIS***, I suggest having > and testing a plan for backing out the change should the backlash > from users after systems break be more than people can stomach. This > is not the sort of change we should make without an "exit strategy". The problem is that those end users didn't know about it until it they read the news item during an emerge. That is why they "would only make themselves known after a fundamental change has been made". There are a couple of alternatives... a) tell all end-users that they should regularly monitor this list. The disadvantage is that you probably don't want to be flooded with questions from newbies on this list. b) ask on the Gentoo-user mailing list... ***BEFORE MAKING A DECISION AND INVESTING ANY WORK IN A MAJOR CHANGE*** Otherwise, you end up with a scenario similar to the following "fair use" snippett from "The Hitch Hiker's Guide To The Galaxy" series... ARTHUR DENT: Didn't anyone consider the alternatives? MISTER PROSSER: There aren't any alternatives! But you are quite entitled to make any suggestions or protests at the appropriate time! ARTHUR DENT: Appropriate time? MISTER PROSSER: Yes. ARTHUR DENT: The first I knew about it was when a workmen arrived at the door yesterday. MISTER PROSSER: T- oh! ARTHUR DENT: I asked him if he'd come to clean the windows and he said he'd come to demolish the house! He didn't tell me straight away of course. Oh no. First he wiped a couple of windows and charged me a fiver. Then he told me. MISTER PROSSER: But Mister Dent the plans have been available in the planning office for the last nine months! ARTHUR DENT: Yes! I went round to find them yesterday afternoon. You'd hadn't exactly gone out of your way to pull much attention to them have you? I mean, like actually telling anybody or anything. MISTER PROSSER: The plans were on display. ARTHUR DENT: Ah! And how many members of the public are in the habit of casually dropping around the local planning office of an evening? MISTER PROSSER: Er - ah! ARTHUR DENT: It's not exactly a noted social venue is it? And even if you had popped in on the off chance that some raving bureaucrat wanted to knock your house down, the plans weren't immediately obvious to the eye were they? MISTER PROSSER: That depends where you were looking. ARTHUR DENT: I eventually had to go down to the cellar! MISTER PROSSER: That's the display department. ARTHUR DENT: With a torch! MISTER PROSSER: The lights, had# probably gone. ARTHUR DENT: So had the stairs! MISTER PROSSER: Well you found the notice didn't you? ARTHUR DENT: Yes. It was on display in the bottom of a locked filing cabinet, stuck in a disused lavatory with a sign on the door saying "Beware of the Leopard". Ever thought of going into advertising? -- Walter Dnes <waltdnes@waltdnes.org> I don't run "desktop environments"; I run useful applications ^ permalink raw reply [flat|nested] 89+ messages in thread
* Re: [gentoo-dev] usr merge 2016-04-05 1:19 [gentoo-dev] usr merge William Hubbs 2016-04-05 10:10 ` Alexis Ballier 2016-04-06 4:15 ` [gentoo-dev] " Richard Yao @ 2016-04-06 14:58 ` M. J. Everitt 2016-04-06 15:11 ` Alexis Ballier 2016-04-06 15:52 ` Richard Yao 2016-04-09 11:41 ` Luca Barbato 2016-04-10 11:55 ` [gentoo-dev] " Joshua Kinard 4 siblings, 2 replies; 89+ messages in thread From: M. J. Everitt @ 2016-04-06 14:58 UTC (permalink / raw To: gentoo-dev [-- Attachment #1: Type: text/plain, Size: 2094 bytes --] What, if any, is the benefit of squashing /usr out of the equation? I happen to have a few workstations that load their /usr off an NFS share presently, with some bodgery-workarounds I did pre the udev notification about initramfs's which I have never got around to implementing (although I'm pretty sure I have the tools now to do, along with UUIDs for boot media). Whilst these aren't currently scheduled for upgrade, I don't personally see any merit, given discussions here about work needed to 'shore up' a change to match some particular use case. I would therefore definitely agree with those that have proposed that this is an Option and not a standard gentoo install item unless there are some specific caveats that this solves. Michael/veremit. On 05/04/16 02:19, William Hubbs wrote: > All, > > I thought that since the usr merge is coming up again, and since I lost > track of the message where it was brought up, I would open a > new thread to discuss it. > > When it came up before, some were saying that the /usr merge violates > the fhs. I don't remember the specifics of what the claim was at the > time, (I'm sure someone will point it out if it is still a concern). > > I don't think creating usr merged stages would be that difficult. I > think it would just be a matter of creating a new version of baselayout > that puts these symlinks in place: > > /bin->usr/bin > /lib->usr/lib > /lib32->usr/lib32 > /lib64->usr/lib64 > /sbin->usr/bin > /usr/sbin->bin > > Once that is in place in a new baselayout, I think portage's colission > detection would be able to catch files that had the same names and were > originally in different paths when building the new stages. > > I put some thought also in how to nigrate live systems, and I'm not sure > what the best way to do that is. I wrote a script, which would do it in > theory, but I haven't tested because I only have one system and if > it breaks it the only way back would be to reinstall. > > The script is attached. > > > Thoughts on any of this? > > William > [-- Attachment #2: OpenPGP digital signature --] [-- Type: application/pgp-signature, Size: 819 bytes --] ^ permalink raw reply [flat|nested] 89+ messages in thread
* Re: [gentoo-dev] usr merge 2016-04-06 14:58 ` M. J. Everitt @ 2016-04-06 15:11 ` Alexis Ballier 2016-04-06 16:06 ` Richard Yao 2016-04-06 15:52 ` Richard Yao 1 sibling, 1 reply; 89+ messages in thread From: Alexis Ballier @ 2016-04-06 15:11 UTC (permalink / raw To: gentoo-dev On Wednesday, April 6, 2016 4:58:05 PM CEST, M. J. Everitt wrote: > What, if any, is the benefit of squashing /usr out of the equation? I > happen to have a few workstations that load their /usr off an NFS share > presently, This is precisely one case where I see benefits: no need to correlate / and /usr. With the current way, this setup is broken if you don't pay attention: glibc is not backwards compatible (that is, stuff built for glibc 2.22 is not guaranteed to work with 2.21 and less), but you have glibc in /lib and stuff in /usr linking and dynamically loading it. If your nfs server updates glibc, you have to update every / on every of your workstations or fear the consequences of running binaries built for 2.22 but running against an older version. See https://fedoraproject.org/wiki/Features/UsrMove for a more complete discussion. ^ permalink raw reply [flat|nested] 89+ messages in thread
* Re: [gentoo-dev] usr merge 2016-04-06 15:11 ` Alexis Ballier @ 2016-04-06 16:06 ` Richard Yao 2016-04-06 16:12 ` M. J. Everitt ` (2 more replies) 0 siblings, 3 replies; 89+ messages in thread From: Richard Yao @ 2016-04-06 16:06 UTC (permalink / raw To: gentoo-dev [-- Attachment #1: Type: text/plain, Size: 2303 bytes --] On 04/06/2016 11:11 AM, Alexis Ballier wrote: > On Wednesday, April 6, 2016 4:58:05 PM CEST, M. J. Everitt wrote: >> What, if any, is the benefit of squashing /usr out of the equation? I >> happen to have a few workstations that load their /usr off an NFS share >> presently, > > > This is precisely one case where I see benefits: no need to correlate / > and /usr. > > With the current way, this setup is broken if you don't pay attention: > glibc is not backwards compatible (that is, stuff built for glibc 2.22 > is not guaranteed to work with 2.21 and less), but you have glibc in > /lib and stuff in /usr linking and dynamically loading it. If your nfs > server updates glibc, you have to update every / on every of your > workstations or fear the consequences of running binaries built for 2.22 > but running against an older version. > That is unless you put per-system state in /usr/local, do symlinks to it in / and mount /usr/local as part of system boot, which is the other way of doing this. I have seen a variant of this done in asuswrt-merlin on routers. > See https://fedoraproject.org/wiki/Features/UsrMove for a more complete > discussion. That does not address the problems of supporting this configuration in a rolling release. Formats in /etc can fall out of sync with software in /usr. If boot options change, the stuff in /etc/init.d is not updated. If you add software, the update to /etc/init.d is omitted. If you have a baselayout change, it is not propagated. Whether or not the package manager can be used is not discussed. It definitely can be in Solaris when this feature is used in Solaris zones, although I am not sure how that interacts with updates as I never looked. I do not have a VM with a member of the OpenSolaris family handy to check. Solaris and RHEL will see the benefits described on the Fedora page because they handled many of those problems. In most cases, they handled it by being stale non-rolling releases that do not support major version upgrades. Fedora handled it by having a disclaimer that things should be expected to break across Fedora versions. Neither are things that I expect us to do, so if we adopt this, we will need to do something entirely new to be able to gain these benefits. [-- Attachment #2: OpenPGP digital signature --] [-- Type: application/pgp-signature, Size: 819 bytes --] ^ permalink raw reply [flat|nested] 89+ messages in thread
* Re: [gentoo-dev] usr merge 2016-04-06 16:06 ` Richard Yao @ 2016-04-06 16:12 ` M. J. Everitt 2016-04-06 16:20 ` Alexis Ballier 2016-04-06 16:24 ` Richard Yao 2 siblings, 0 replies; 89+ messages in thread From: M. J. Everitt @ 2016-04-06 16:12 UTC (permalink / raw To: gentoo-dev [-- Attachment #1: Type: text/plain, Size: 1237 bytes --] On 06/04/16 17:06, Richard Yao wrote: > > That does not address the problems of supporting this configuration in a > rolling release. > > Formats in /etc can fall out of sync with software in /usr. If boot > options change, the stuff in /etc/init.d is not updated. If you add > software, the update to /etc/init.d is omitted. If you have a baselayout > change, it is not propagated. Whether or not the package manager can be > used is not discussed. It definitely can be in Solaris when this feature > is used in Solaris zones, although I am not sure how that interacts with > updates as I never looked. I do not have a VM with a member of the > OpenSolaris family handy to check. /usr/etc .. or /usr/local/etc *ew* ... ?!?! > Solaris and RHEL will see the benefits described on the Fedora page > because they handled many of those problems. In most cases, they handled > it by being stale non-rolling releases that do not support major version > upgrades. Fedora handled it by having a disclaimer that things should be > expected to break across Fedora versions. Neither are things that I > expect us to do, so if we adopt this, we will need to do something > entirely new to be able to gain these benefits. > [-- Attachment #2: OpenPGP digital signature --] [-- Type: application/pgp-signature, Size: 819 bytes --] ^ permalink raw reply [flat|nested] 89+ messages in thread
* Re: [gentoo-dev] usr merge 2016-04-06 16:06 ` Richard Yao 2016-04-06 16:12 ` M. J. Everitt @ 2016-04-06 16:20 ` Alexis Ballier 2016-04-06 16:33 ` Richard Yao 2016-04-06 16:24 ` Richard Yao 2 siblings, 1 reply; 89+ messages in thread From: Alexis Ballier @ 2016-04-06 16:20 UTC (permalink / raw To: gentoo-dev On Wednesday, April 6, 2016 6:06:35 PM CEST, Richard Yao wrote: > That is unless you put per-system state in /usr/local, do symlinks to it > in / and mount /usr/local as part of system boot, which is the other way > of doing this. I have seen a variant of this done in asuswrt-merlin on > routers. This doesnt seem to have anything to do with what I was describing. Another option I'm using a lot is nfsroot. This doesn't have the same level of flexibility: running multiple hosts with nfsroot and thus shared /etc/fstab tends to be annoying. >> See https://fedoraproject.org/wiki/Features/UsrMove for a more complete >> discussion. > > That does not address the problems of supporting this configuration in a > rolling release. > > Formats in /etc can fall out of sync with software in /usr. If boot > options change, the stuff in /etc/init.d is not updated. If you add > software, the update to /etc/init.d is omitted. If you have a baselayout > change, it is not propagated. Ever heard of CONFIG_PROTECT ? :) What you describe is already what happens and what most people want. ^ permalink raw reply [flat|nested] 89+ messages in thread
* Re: [gentoo-dev] usr merge 2016-04-06 16:20 ` Alexis Ballier @ 2016-04-06 16:33 ` Richard Yao 2016-04-06 16:57 ` Alexis Ballier 2016-04-06 17:43 ` Richard Yao 0 siblings, 2 replies; 89+ messages in thread From: Richard Yao @ 2016-04-06 16:33 UTC (permalink / raw To: gentoo-dev On 04/06/2016 12:20 PM, Alexis Ballier wrote: > On Wednesday, April 6, 2016 6:06:35 PM CEST, Richard Yao wrote: > >> That is unless you put per-system state in /usr/local, do symlinks to it >> in / and mount /usr/local as part of system boot, which is the other way >> of doing this. I have seen a variant of this done in asuswrt-merlin on >> routers. > > This doesnt seem to have anything to do with what I was describing. > > Another option I'm using a lot is nfsroot. This doesn't have the same > level of flexibility: running multiple hosts with nfsroot and thus > shared /etc/fstab tends to be annoying. > >>> See https://fedoraproject.org/wiki/Features/UsrMove for a more complete >>> discussion. >> >> That does not address the problems of supporting this configuration in a >> rolling release. >> >> Formats in /etc can fall out of sync with software in /usr. If boot >> options change, the stuff in /etc/init.d is not updated. If you add >> software, the update to /etc/init.d is omitted. If you have a baselayout >> change, it is not propagated. > > Ever heard of CONFIG_PROTECT ? :) What you describe is already what > happens and what most people want. Leveraging the /usr merge to enable easier updating of multiple systems means that you are updating a Gentoo system image on a build server, snapshotting /usr both before/after the update and then distributing the delta on /usr to other systems without any of the changes that occurred outside of /usr. A proper update requires finding all of those changes and then applying them manually. That really is not the same thing that RHEL and Solaris have, where the necessity of propagating changes outside of /usr is minimized by having none to propagate. I do not understand how CONFIG_PROTECT is relevant here. Whatever CONFIG_PROTECT did was done on the build system. The systems receiving the updates via ZFS send/recv or some similar mechanism are not going to have CONFIG_PROTECT evaluated. Even if it were somehow evaluated, all of the paths in CONFIG_PROTECT should be outside of /usr anyway. ^ permalink raw reply [flat|nested] 89+ messages in thread
* Re: [gentoo-dev] usr merge 2016-04-06 16:33 ` Richard Yao @ 2016-04-06 16:57 ` Alexis Ballier 2016-04-06 17:06 ` Alexis Ballier 2016-04-06 17:43 ` Richard Yao 1 sibling, 1 reply; 89+ messages in thread From: Alexis Ballier @ 2016-04-06 16:57 UTC (permalink / raw To: gentoo-dev On Wednesday, April 6, 2016 6:33:41 PM CEST, Richard Yao wrote: > On 04/06/2016 12:20 PM, Alexis Ballier wrote: >> On Wednesday, April 6, 2016 6:06:35 PM CEST, Richard Yao wrote: >> ... > > Leveraging the /usr merge to enable easier updating of multiple systems > means that you are updating a Gentoo system image on a build server, > snapshotting /usr both before/after the update and then distributing the > delta on /usr to other systems without any of the changes that occurred > outside of /usr. A proper update requires finding all of those changes > and then applying them manually. That really is not the same thing that > RHEL and Solaris have, where the necessity of propagating changes > outside of /usr is minimized by having none to propagate. > > I do not understand how CONFIG_PROTECT is relevant here. Whatever > CONFIG_PROTECT did was done on the build system. The systems receiving > the updates via ZFS send/recv or some similar mechanism are not going to > have CONFIG_PROTECT evaluated. Even if it were somehow evaluated, all of > the paths in CONFIG_PROTECT should be outside of /usr anyway. Exactly. CONFIG_PROTECT requires admins to (sort of) manually merge /etc changes. If you want all changes on the build server to be magically propagated to clients, then share it over nfs or whatever is your prefered mechanism. This is not a great idea though, FHS is quite clear on the matter: /etc : Host-specific system configuration usr-merge does not deal with that at all. usr-merge deals with the intracate dependencies of /usr onto /lib, /bin, etc. by simply removing them. If you want to share everything, then use nfsroot and mount local disks by label. ^ permalink raw reply [flat|nested] 89+ messages in thread
* Re: [gentoo-dev] usr merge 2016-04-06 16:57 ` Alexis Ballier @ 2016-04-06 17:06 ` Alexis Ballier 0 siblings, 0 replies; 89+ messages in thread From: Alexis Ballier @ 2016-04-06 17:06 UTC (permalink / raw To: gentoo-dev On Wednesday, April 6, 2016 6:57:20 PM CEST, Alexis Ballier wrote: > usr-merge does not deal with that at all. usr-merge deals with > the intracate dependencies of /usr onto /lib, /bin, etc. by now that I read this again: 'etc.' was the shortcut for 'et caetera' and has nothing to do with /etc in this sentence ^ permalink raw reply [flat|nested] 89+ messages in thread
* Re: [gentoo-dev] usr merge 2016-04-06 16:33 ` Richard Yao 2016-04-06 16:57 ` Alexis Ballier @ 2016-04-06 17:43 ` Richard Yao 1 sibling, 0 replies; 89+ messages in thread From: Richard Yao @ 2016-04-06 17:43 UTC (permalink / raw To: gentoo-dev On 04/06/2016 12:33 PM, Richard Yao wrote: > On 04/06/2016 12:20 PM, Alexis Ballier wrote: >> On Wednesday, April 6, 2016 6:06:35 PM CEST, Richard Yao wrote: >> >>> That is unless you put per-system state in /usr/local, do symlinks to it >>> in / and mount /usr/local as part of system boot, which is the other way >>> of doing this. I have seen a variant of this done in asuswrt-merlin on >>> routers. >> >> This doesnt seem to have anything to do with what I was describing. >> >> Another option I'm using a lot is nfsroot. This doesn't have the same >> level of flexibility: running multiple hosts with nfsroot and thus >> shared /etc/fstab tends to be annoying. >> >>>> See https://fedoraproject.org/wiki/Features/UsrMove for a more complete >>>> discussion. >>> >>> That does not address the problems of supporting this configuration in a >>> rolling release. >>> >>> Formats in /etc can fall out of sync with software in /usr. If boot >>> options change, the stuff in /etc/init.d is not updated. If you add >>> software, the update to /etc/init.d is omitted. If you have a baselayout >>> change, it is not propagated. >> >> Ever heard of CONFIG_PROTECT ? :) What you describe is already what >> happens and what most people want. > > Leveraging the /usr merge to enable easier updating of multiple systems > means that you are updating a Gentoo system image on a build server, > snapshotting /usr both before/after the update and then distributing the > delta on /usr to other systems without any of the changes that occurred > outside of /usr. A proper update requires finding all of those changes > and then applying them manually. That really is not the same thing that > RHEL and Solaris have, where the necessity of propagating changes > outside of /usr is minimized by having none to propagate. After thinking about it some more, maybe git can be (ab)used to do this by having the image's root be a git repository with /usr in .gitingore. When updates are done, you would run etc-update on the build host by committing the changes to / into git with ./usr in .gitignore. Then you would have the delta from /usr via whatever mechanism that the user wishes to use with a patch on / from the build system that can be merged into each target's / repository. The procedure would require more effort than what Solaris and RHEL do, but if documented, it should fix the partial update problem that occurs when you do a /usr merge in a rolling release and then do what people on Solaris and RHEL do to update systems configured for updates via deltas of /usr. To give an example, lets assume: 1. /path/to/build/image has a git repository with ./usr in .gitignore on the build host while the targets have the same (with paths being / and /usr of course). 2. Things are on ZFS on the build host and the targets. 3. There is a snapshot of the build environment that the targets have. 4. tank/BUILD/gentoo is the / and tank/BUILD/gentoo/usr is the /usr on the build host. 5. The targets have rpool/SYSTEM/USR/gentoo as their /usr and rpool/SYSTEM/USR is set readonly (so /usr is not modified due to inheritance of readonly). The delta generation would be something like this: # Setup build environment sudo -i /path/to/enter-container.sh /path/to/build/image https://gist.github.com/ryao/3c345f206b19c9795109) # Update portage emerge-webrsync / emerge-delta-webrsync / emerge --sync # Preferably one of the first two with PORTAGE_GPG_* configured # Install updates emerge -avDuN @world # Update config files etc-update # Exit build environment exit # Commit changes to / git -C /path/to/build/image commit -a # Snapshot the build environment sudo -i zfs snapshot -r tank/BUILD/gentoo@"$(date +%Y%m%d)" Then a child could be updated by something like: # First /usr zfs send -i tank/BUILD/gentoo@previous tank/BUILD/gentoo/usr@"$(date +%Y%m%d)" | ssh root@$CHILD zfs recv rpool/SYSTEM/USR/gentoo # Then / git -C /path/to/build/image diff HEAD^ | ssh root@$CHILD git -C /other/location apply # If conflicts occurred, fix whatever was broken ssh root@$CHILD # Reboot / restart services reboot This is intended to only be an example, but there are a few problems with this simple example that are worth mentioning: 1. You probably want to have a shell into the system in case the update to / does not go well, which makes the update to / somewhat hackish. 2. The way Solaris does things is to have boot environments where the change is in a different boot environment that only takes effect as part of a reboot. If you are doing a boot environment type thing, you could probably update / for the new reboot, although you would want to implement easy rollback if anything goes wrong. That step on / is somewhat hackish, but it is intended to be an example. Doing something similar to what Solaris did to make management of multiple systems easier is likely doable with some way of handling changes outside of /usr. > I do not understand how CONFIG_PROTECT is relevant here. Whatever > CONFIG_PROTECT did was done on the build system. The systems receiving > the updates via ZFS send/recv or some similar mechanism are not going to > have CONFIG_PROTECT evaluated. Even if it were somehow evaluated, all of > the paths in CONFIG_PROTECT should be outside of /usr anyway. ^ permalink raw reply [flat|nested] 89+ messages in thread
* Re: [gentoo-dev] usr merge 2016-04-06 16:06 ` Richard Yao 2016-04-06 16:12 ` M. J. Everitt 2016-04-06 16:20 ` Alexis Ballier @ 2016-04-06 16:24 ` Richard Yao 2 siblings, 0 replies; 89+ messages in thread From: Richard Yao @ 2016-04-06 16:24 UTC (permalink / raw To: gentoo-dev On 04/06/2016 12:06 PM, Richard Yao wrote: > On 04/06/2016 11:11 AM, Alexis Ballier wrote: >> On Wednesday, April 6, 2016 4:58:05 PM CEST, M. J. Everitt wrote: >>> What, if any, is the benefit of squashing /usr out of the equation? I >>> happen to have a few workstations that load their /usr off an NFS share >>> presently, >> >> >> This is precisely one case where I see benefits: no need to correlate / >> and /usr. >> >> With the current way, this setup is broken if you don't pay attention: >> glibc is not backwards compatible (that is, stuff built for glibc 2.22 >> is not guaranteed to work with 2.21 and less), but you have glibc in >> /lib and stuff in /usr linking and dynamically loading it. If your nfs >> server updates glibc, you have to update every / on every of your >> workstations or fear the consequences of running binaries built for 2.22 >> but running against an older version. >> > > That is unless you put per-system state in /usr/local, do symlinks to it > in / and mount /usr/local as part of system boot, which is the other way > of doing this. I have seen a variant of this done in asuswrt-merlin on > routers. > >> See https://fedoraproject.org/wiki/Features/UsrMove for a more complete >> discussion. > > That does not address the problems of supporting this configuration in a > rolling release. > > Formats in /etc can fall out of sync with software in /usr. If boot > options change, the stuff in /etc/init.d is not updated. If you add > software, the update to /etc/init.d is omitted. If you have a baselayout > change, it is not propagated. Whether or not the package manager can be > used is not discussed. It definitely can be in Solaris when this feature > is used in Solaris zones, although I am not sure how that interacts with > updates as I never looked. I do not have a VM with a member of the > OpenSolaris family handy to check. > > Solaris and RHEL will see the benefits described on the Fedora page > because they handled many of those problems. In most cases, they handled > it by being stale non-rolling releases that do not support major version > upgrades. Fedora handled it by having a disclaimer that things should be > expected to break across Fedora versions. Neither are things that I > expect us to do, so if we adopt this, we will need to do something > entirely new to be able to gain these benefits. To say it clearly, lets not claim that the /usr merge will give us any of the benefits mentioned in the Fedora wiki unless we have a plan to handle the complications that being a rolling distribution poses for doing atomic updates via the mechanism invented in Solaris on a post-/usr merge system. On a non-rolling system release like Solaris or RHEL, you need to install information on how to boot daemons with default configurations in /usr and let users override things in /etc in addition to the /usr merge to gain the capabilities cited on the Fedora wiki page. You get bonus points if a clone of a snapshot can be used to make a container with a working package manager, but could otherwise define that configuration as being unsupported or write a script to install it. On a rolling release like Gentoo, rearranging the system baselayout is also insufficient, but there are many more problems than those that occur on a non-rolling system release. I listed most of those in my previous email. ^ permalink raw reply [flat|nested] 89+ messages in thread
* Re: [gentoo-dev] usr merge 2016-04-06 14:58 ` M. J. Everitt 2016-04-06 15:11 ` Alexis Ballier @ 2016-04-06 15:52 ` Richard Yao 2016-04-06 20:43 ` William Hubbs 2016-04-07 9:03 ` Alexis Ballier 1 sibling, 2 replies; 89+ messages in thread From: Richard Yao @ 2016-04-06 15:52 UTC (permalink / raw To: gentoo-dev On 04/06/2016 10:58 AM, M. J. Everitt wrote: > What, if any, is the benefit of squashing /usr out of the equation? I > happen to have a few workstations that load their /usr off an NFS share > presently, with some bodgery-workarounds I did pre the udev notification > about initramfs's which I have never got around to implementing > (although I'm pretty sure I have the tools now to do, along with UUIDs > for boot media). The idea in Solaris is to enable atomic updates via the /usr mount without touching data files in /etc or temporary files in /var. Usually, this would be done on reboot and could be propagated to many systems either via /usr on NFS or ZFS send/recv. This works well on Solaris because both software versions are pegged (such that file formats in /etc are stable) in favor of backported fixes and the FHS does not change across major OS versions. The same goes for RHEL. Gentoo systems managed this way will suffer from multiple problems: * Software updates that change the configuration file format without supporting the older format will break. * Software updates that change the boot scripts will break. * Future baselayout updates will not be able to touch anything outside of /usr and anything requiring such things be touched will break. * An update to /usr that adds new software will fail to include things outside of /usr, like the boot scripts and configuration files. * The package database will fall out of sync with /usr (or be broken period). Presumably, if you are updating this way, you should expect the package database to be broken. These are likely to be mostly fixable, but I do not think we have a plan in place to fix them right now. The general staleness of Solaris and RHEL handle the first 3 issues for them for free. I have not looked at the specifics of how Solaris handles the 4th, but I know that SMF in OpenSolaris descendents will update manifests on first boot into a new boot environment. That suggests to me that the Solaris boot scripts handle it by comparing /etc with /usr. As for the 5th, the package database is not broken in Solaris zones where the /usr merge is leveraged to enable easy updates. However, I do not know how updating all zones works when zones have independently installed software. It might be that the software is installed in /usr/local inside the zone and conflicts are the user's problem, but it has been so long since I used an illumos distribution (which is descended from OpenSolaris) that I do not remember. > Whilst these aren't currently scheduled for upgrade, I don't personally > see any merit, given discussions here about work needed to 'shore up' a > change to match some particular use case. I would therefore definitely > agree with those that have proposed that this is an Option and not a > standard gentoo install item unless there are some specific caveats that > this solves. The original purpose of the /usr merge in Solaris was to make managing updates easier. Redhat realized that and copied it. Copying it too without doing the enabling work necessary for a rolling distribution would be setting a trap for users who would think that they can manage deployments of Gentoo like they can manage deployments Solaris and/or RHEL. > Michael/veremit. > > On 05/04/16 02:19, William Hubbs wrote: >> All, >> >> I thought that since the usr merge is coming up again, and since I lost >> track of the message where it was brought up, I would open a >> new thread to discuss it. >> >> When it came up before, some were saying that the /usr merge violates >> the fhs. I don't remember the specifics of what the claim was at the >> time, (I'm sure someone will point it out if it is still a concern). >> >> I don't think creating usr merged stages would be that difficult. I >> think it would just be a matter of creating a new version of baselayout >> that puts these symlinks in place: >> >> /bin->usr/bin >> /lib->usr/lib >> /lib32->usr/lib32 >> /lib64->usr/lib64 >> /sbin->usr/bin >> /usr/sbin->bin >> >> Once that is in place in a new baselayout, I think portage's colission >> detection would be able to catch files that had the same names and were >> originally in different paths when building the new stages. >> >> I put some thought also in how to nigrate live systems, and I'm not sure >> what the best way to do that is. I wrote a script, which would do it in >> theory, but I haven't tested because I only have one system and if >> it breaks it the only way back would be to reinstall. >> >> The script is attached. >> >> >> Thoughts on any of this? >> >> William >> > > ^ permalink raw reply [flat|nested] 89+ messages in thread
* Re: [gentoo-dev] usr merge 2016-04-06 15:52 ` Richard Yao @ 2016-04-06 20:43 ` William Hubbs 2016-04-06 21:36 ` Richard Yao 2016-04-07 9:03 ` Alexis Ballier 1 sibling, 1 reply; 89+ messages in thread From: William Hubbs @ 2016-04-06 20:43 UTC (permalink / raw To: gentoo-dev [-- Attachment #1: Type: text/plain, Size: 4179 bytes --] On Wed, Apr 06, 2016 at 11:52:52AM -0400, Richard Yao wrote: > On 04/06/2016 10:58 AM, M. J. Everitt wrote: > > What, if any, is the benefit of squashing /usr out of the equation? I > > happen to have a few workstations that load their /usr off an NFS share > > presently, with some bodgery-workarounds I did pre the udev notification > > about initramfs's which I have never got around to implementing > > (although I'm pretty sure I have the tools now to do, along with UUIDs > > for boot media). > > The idea in Solaris is to enable atomic updates via the /usr mount > without touching data files in /etc or temporary files in /var. Usually, > this would be done on reboot and could be propagated to many systems > either via /usr on NFS or ZFS send/recv. > > This works well on Solaris because both software versions are pegged > (such that file formats in /etc are stable) in favor of backported fixes > and the FHS does not change across major OS versions. The same goes for > RHEL. Also, there are other benefits to the /usr merge [1]. Note, we are not talking about squashing /usr out of the equasion, but merging /bin, /sbin and /lib* into their counterparts in /usr and creating symlinks in the root directory pointing to the counterparts in /usr. > > Gentoo systems managed this way will suffer from multiple problems: > > * Software updates that change the configuration file format without > supporting the older format will break. > > * Software updates that change the boot scripts will break. > > * Future baselayout updates will not be able to touch anything outside > of /usr and anything requiring such things be touched will break. > > * An update to /usr that adds new software will fail to include things > outside of /usr, like the boot scripts and configuration files. > > * The package database will fall out of sync with /usr (or be broken > period). Presumably, if you are updating this way, you should expect the > package database to be broken. > > These are likely to be mostly fixable, but I do not think we have a plan > in place to fix them right now. The general staleness of Solaris and > RHEL handle the first 3 issues for them for free. > > I have not looked at the specifics of how Solaris handles the 4th, but I > know that SMF in OpenSolaris descendents will update manifests on first > boot into a new boot environment. That suggests to me that the Solaris > boot scripts handle it by comparing /etc with /usr. > > As for the 5th, the package database is not broken in Solaris zones > where the /usr merge is leveraged to enable easy updates. However, I do > not know how updating all zones works when zones have independently > installed software. It might be that the software is installed in > /usr/local inside the zone and conflicts are the user's problem, but it > has been so long since I used an illumos distribution (which is > descended from OpenSolaris) that I do not remember. I don't think any of these issues are issues that Gentoo systems managed like this do not already have. If you are mounting /usr from nfs right now, for example, things are worse, because you also have to worry about whether packages split their installations between /usr/lib*->/lib* and /usr/{,s}bin->/{s,}bin. > > Whilst these aren't currently scheduled for upgrade, I don't personally > > see any merit, given discussions here about work needed to 'shore up' a > > change to match some particular use case. I would therefore definitely > > agree with those that have proposed that this is an Option and not a > > standard gentoo install item unless there are some specific caveats that > > this solves. > > The original purpose of the /usr merge in Solaris was to make managing > updates easier. Redhat realized that and copied it. Copying it too > without doing the enabling work necessary for a rolling distribution > would be setting a trap for users who would think that they can manage > deployments of Gentoo like they can manage deployments Solaris and/or RHEL. [1] https://www.freedesktop.org/wiki/Software/systemd/TheCaseForTheUsrMerge/ [-- Attachment #2: Digital signature --] [-- Type: application/pgp-signature, Size: 181 bytes --] ^ permalink raw reply [flat|nested] 89+ messages in thread
* Re: [gentoo-dev] usr merge 2016-04-06 20:43 ` William Hubbs @ 2016-04-06 21:36 ` Richard Yao 2016-04-07 0:44 ` William Hubbs 2016-04-07 9:12 ` Alexis Ballier 0 siblings, 2 replies; 89+ messages in thread From: Richard Yao @ 2016-04-06 21:36 UTC (permalink / raw To: gentoo-dev >> On Apr 6, 2016, at 4:43 PM, William Hubbs <williamh@gentoo.org> wrote: >> >>> On Wed, Apr 06, 2016 at 11:52:52AM -0400, Richard Yao wrote: >>> On 04/06/2016 10:58 AM, M. J. Everitt wrote: >>> What, if any, is the benefit of squashing /usr out of the equation? I >>> happen to have a few workstations that load their /usr off an NFS share >>> presently, with some bodgery-workarounds I did pre the udev notification >>> about initramfs's which I have never got around to implementing >>> (although I'm pretty sure I have the tools now to do, along with UUIDs >>> for boot media). >> >> The idea in Solaris is to enable atomic updates via the /usr mount >> without touching data files in /etc or temporary files in /var. Usually, >> this would be done on reboot and could be propagated to many systems >> either via /usr on NFS or ZFS send/recv. >> >> This works well on Solaris because both software versions are pegged >> (such that file formats in /etc are stable) in favor of backported fixes >> and the FHS does not change across major OS versions. The same goes for >> RHEL. > > Also, there are other benefits to the /usr merge [1]. Are they worth breaking existing systems that are configured the one way we all know things will break if this is forced? If not, a USE flag would work. As for those benefits, they do little for {/usr,}/sbin vs {/usr,}/bin, which is where the incompatibilities tend to live. I encountered one of these in powertop the other day (patch pending). The benefits of being able to access things from both places are somewhat exaggerated given that compatibility among systems has long required searching $PATH and likely always will. > Note, we are not > talking about squashing /usr out of the equasion, but merging /bin, > /sbin and /lib* into their counterparts in /usr and creating symlinks in > the root directory pointing to the counterparts in /usr. While one guy did the reverse (and the reverse ought to be okay for those that want to do that), no one appears to think that adopting the reverse is what is being suggested. Having this sort of clarity on whether forcing this on everyone via baselayout update, just providing the option for those who want it or some combination of the two (e.g. a long transition period in which both are supported) is being discussed would be nice though. This is not a Boolean decision. >> Gentoo systems managed this way will suffer from multiple problems: >> >> * Software updates that change the configuration file format without >> supporting the older format will break. >> >> * Software updates that change the boot scripts will break. >> >> * Future baselayout updates will not be able to touch anything outside >> of /usr and anything requiring such things be touched will break. >> >> * An update to /usr that adds new software will fail to include things >> outside of /usr, like the boot scripts and configuration files. >> >> * The package database will fall out of sync with /usr (or be broken >> period). Presumably, if you are updating this way, you should expect the >> package database to be broken. >> >> These are likely to be mostly fixable, but I do not think we have a plan >> in place to fix them right now. The general staleness of Solaris and >> RHEL handle the first 3 issues for them for free. >> >> I have not looked at the specifics of how Solaris handles the 4th, but I >> know that SMF in OpenSolaris descendents will update manifests on first >> boot into a new boot environment. That suggests to me that the Solaris >> boot scripts handle it by comparing /etc with /usr. >> >> As for the 5th, the package database is not broken in Solaris zones >> where the /usr merge is leveraged to enable easy updates. However, I do >> not know how updating all zones works when zones have independently >> installed software. It might be that the software is installed in >> /usr/local inside the zone and conflicts are the user's problem, but it >> has been so long since I used an illumos distribution (which is >> descended from OpenSolaris) that I do not remember. > > I don't think any of these issues are issues that Gentoo systems > managed like this do not already have. That is my point. > If you are mounting /usr from nfs right now, for example, > things are worse, because you also have to worry about whether packages > split their installations between /usr/lib*->/lib* and > /usr/{,s}bin->/{s,}bin. Only a masochist would want to do this right now. There are saner ways of doing things with the legacy layout than the Solaris way that depends on the /usr merge. >>> Whilst these aren't currently scheduled for upgrade, I don't personally >>> see any merit, given discussions here about work needed to 'shore up' a >>> change to match some particular use case. I would therefore definitely >>> agree with those that have proposed that this is an Option and not a >>> standard gentoo install item unless there are some specific caveats that >>> this solves. >> >> The original purpose of the /usr merge in Solaris was to make managing >> updates easier. Redhat realized that and copied it. Copying it too >> without doing the enabling work necessary for a rolling distribution >> would be setting a trap for users who would think that they can manage >> deployments of Gentoo like they can manage deployments Solaris and/or RHEL. > > [1] > https://www.freedesktop.org/wiki/Software/systemd/TheCaseForTheUsrMerge/ ^ permalink raw reply [flat|nested] 89+ messages in thread
* Re: [gentoo-dev] usr merge 2016-04-06 21:36 ` Richard Yao @ 2016-04-07 0:44 ` William Hubbs 2016-04-07 9:12 ` Alexis Ballier 1 sibling, 0 replies; 89+ messages in thread From: William Hubbs @ 2016-04-07 0:44 UTC (permalink / raw To: gentoo-dev [-- Attachment #1: Type: text/plain, Size: 4856 bytes --] On Wed, Apr 06, 2016 at 05:36:09PM -0400, Richard Yao wrote: > > > >> On Apr 6, 2016, at 4:43 PM, William Hubbs <williamh@gentoo.org> wrote: > >> > >>> On Wed, Apr 06, 2016 at 11:52:52AM -0400, Richard Yao wrote: > >>> On 04/06/2016 10:58 AM, M. J. Everitt wrote: > >>> What, if any, is the benefit of squashing /usr out of the equation? I > >>> happen to have a few workstations that load their /usr off an NFS share > >>> presently, with some bodgery-workarounds I did pre the udev notification > >>> about initramfs's which I have never got around to implementing > >>> (although I'm pretty sure I have the tools now to do, along with UUIDs > >>> for boot media). > >> > >> The idea in Solaris is to enable atomic updates via the /usr mount > >> without touching data files in /etc or temporary files in /var. Usually, > >> this would be done on reboot and could be propagated to many systems > >> either via /usr on NFS or ZFS send/recv. > >> > >> This works well on Solaris because both software versions are pegged > >> (such that file formats in /etc are stable) in favor of backported fixes > >> and the FHS does not change across major OS versions. The same goes for > >> RHEL. > > > > Also, there are other benefits to the /usr merge [1]. > > Are they worth breaking existing systems that are configured the one way we all know things will break if this is forced? If not, a USE flag would work. Other than systems using separate /usr without initramfs (which we declared broken three years ago), I'm not following what "we all know" would break. > As for those benefits, they do little for {/usr,}/sbin vs {/usr,}/bin, which is where the incompatibilities tend to live. I encountered one of these in powertop the other day (patch pending). The benefits of being able to access things from both places are somewhat exaggerated given that compatibility among systems has long required searching $PATH and likely always will. > > > Note, we are not > > talking about squashing /usr out of the equasion, but merging /bin, > > /sbin and /lib* into their counterparts in /usr and creating symlinks in > > the root directory pointing to the counterparts in /usr. > > While one guy did the reverse (and the reverse ought to be okay for those that want to do that), no one appears to think that adopting the reverse is what is being suggested. Having this sort of clarity on whether forcing this on everyone via baselayout update, just providing the option for those who want it or some combination of the two (e.g. a long transition period in which both are supported) is being discussed would be nice though. This is not a Boolean decision. > > >> Gentoo systems managed this way will suffer from multiple problems: > >> > >> * Software updates that change the configuration file format without > >> supporting the older format will break. > >> > >> * Software updates that change the boot scripts will break. > >> > >> * Future baselayout updates will not be able to touch anything outside > >> of /usr and anything requiring such things be touched will break. > >> > >> * An update to /usr that adds new software will fail to include things > >> outside of /usr, like the boot scripts and configuration files. > >> > >> * The package database will fall out of sync with /usr (or be broken > >> period). Presumably, if you are updating this way, you should expect the > >> package database to be broken. > >> > >> These are likely to be mostly fixable, but I do not think we have a plan > >> in place to fix them right now. The general staleness of Solaris and > >> RHEL handle the first 3 issues for them for free. > >> > >> I have not looked at the specifics of how Solaris handles the 4th, but I > >> know that SMF in OpenSolaris descendents will update manifests on first > >> boot into a new boot environment. That suggests to me that the Solaris > >> boot scripts handle it by comparing /etc with /usr. > >> > >> As for the 5th, the package database is not broken in Solaris zones > >> where the /usr merge is leveraged to enable easy updates. However, I do > >> not know how updating all zones works when zones have independently > >> installed software. It might be that the software is installed in > >> /usr/local inside the zone and conflicts are the user's problem, but it > >> has been so long since I used an illumos distribution (which is > >> descended from OpenSolaris) that I do not remember. > > > > I don't think any of these issues are issues that Gentoo systems > > managed like this do not already have. > > That is my point. Then we agree that these issues are not regressions that the usr merge would cause. They are issues possibly, but imo not relevant to whether we go ahead with the /usr merge or not. William [-- Attachment #2: Digital signature --] [-- Type: application/pgp-signature, Size: 181 bytes --] ^ permalink raw reply [flat|nested] 89+ messages in thread
* Re: [gentoo-dev] usr merge 2016-04-06 21:36 ` Richard Yao 2016-04-07 0:44 ` William Hubbs @ 2016-04-07 9:12 ` Alexis Ballier 2016-04-07 14:40 ` William Hubbs 1 sibling, 1 reply; 89+ messages in thread From: Alexis Ballier @ 2016-04-07 9:12 UTC (permalink / raw To: gentoo-dev On Wednesday, April 6, 2016 11:36:09 PM CEST, Richard Yao wrote: > As for those benefits, they do little for {/usr,}/sbin vs > {/usr,}/bin, which is where the incompatibilities tend to live. > I encountered one of these in powertop the other day (patch > pending). The benefits of being able to access things from both > places are somewhat exaggerated given that compatibility among > systems has long required searching $PATH and likely always > will. PATH is a shell thing; some libc functions like execvp duplicate this functionality but that's all; you dont have PATH in shebangs nor in execv. >> Note, we are not >> talking about squashing /usr out of the equasion, but merging /bin, >> /sbin and /lib* into their counterparts in /usr and creating symlinks in >> the root directory pointing to the counterparts in /usr. > > While one guy did the reverse (and the reverse ought to be okay > for those that want to do that), no one appears to think that > adopting the reverse is what is being suggested. Having this > sort of clarity on whether forcing this on everyone via > baselayout update, just providing the option for those who want > it or some combination of the two (e.g. a long transition period > in which both are supported) is being discussed would be nice > though. This is not a Boolean decision. I've been under the impression since the beginning of the thread that it is what is being proposed: make it possible but support both. We can't force usr-merge without battle testing the migration process anyway, which means there needs to be such a long transition period. Alexis. ^ permalink raw reply [flat|nested] 89+ messages in thread
* Re: [gentoo-dev] usr merge 2016-04-07 9:12 ` Alexis Ballier @ 2016-04-07 14:40 ` William Hubbs 2016-04-07 15:12 ` [gentoo-dev] " Duncan 0 siblings, 1 reply; 89+ messages in thread From: William Hubbs @ 2016-04-07 14:40 UTC (permalink / raw To: gentoo-dev [-- Attachment #1: Type: text/plain, Size: 2626 bytes --] On Thu, Apr 07, 2016 at 11:12:13AM +0200, Alexis Ballier wrote: > On Wednesday, April 6, 2016 11:36:09 PM CEST, Richard Yao wrote: > > As for those benefits, they do little for {/usr,}/sbin vs > > {/usr,}/bin, which is where the incompatibilities tend to live. > > I encountered one of these in powertop the other day (patch > > pending). The benefits of being able to access things from both > > places are somewhat exaggerated given that compatibility among > > systems has long required searching $PATH and likely always > > will. > > PATH is a shell thing; some libc functions like execvp duplicate this > functionality but that's all; you dont have PATH in shebangs nor in execv. > > >> Note, we are not > >> talking about squashing /usr out of the equasion, but merging /bin, > >> /sbin and /lib* into their counterparts in /usr and creating symlinks in > >> the root directory pointing to the counterparts in /usr. > > > > While one guy did the reverse (and the reverse ought to be okay > > for those that want to do that), no one appears to think that > > adopting the reverse is what is being suggested. Having this > > sort of clarity on whether forcing this on everyone via > > baselayout update, just providing the option for those who want > > it or some combination of the two (e.g. a long transition period > > in which both are supported) is being discussed would be nice > > though. This is not a Boolean decision. > > I've been under the impression since the beginning of the thread that it is > what is being proposed: make it possible but support both. We can't force > usr-merge without battle testing the migration process anyway, which means > there needs to be such a long transition period. I do agree that we need a testing period to iron out the migration process. Like I said, I'm not quite comfortable even with running it here because I don't know if it will break my system, and once you do the migration, the only way to undo it is to wipe and re-install. I have thought about a way to roll back, but I don't see that as very feesable, so once you migrate to a /usr merged setup, there is no way to undo it. Also, the usr merge affects linux only; we aren't talking about messing with *bsd. After the testing period is over, I'm confused about why we should support both layouts. With separate usr without initramfs gone, the usr merge is transparent to end users because of the symbolic links in /, so there should be no reason to keep supporting both layouts once we are satisfied with the migration process. William [-- Attachment #2: Digital signature --] [-- Type: application/pgp-signature, Size: 181 bytes --] ^ permalink raw reply [flat|nested] 89+ messages in thread
* [gentoo-dev] Re: usr merge 2016-04-07 14:40 ` William Hubbs @ 2016-04-07 15:12 ` Duncan 2016-04-07 15:42 ` Rich Freeman 0 siblings, 1 reply; 89+ messages in thread From: Duncan @ 2016-04-07 15:12 UTC (permalink / raw To: gentoo-dev William Hubbs posted on Thu, 07 Apr 2016 09:40:49 -0500 as excerpted: > After the testing period is over, I'm confused about why we should > support both layouts. With separate usr without initramfs gone, the usr > merge is transparent to end users because of the symbolic links in /, so > there should be no reason to keep supporting both layouts once we are > satisfied with the migration process. Because we're Gentoo, and gentooers tend to have rather strong opinions on what sort of choices we should be able to make about things like that. Honestly, that's going to look to a lot of people like another of the systemd/RedHat alliance overreaches and they're not going to go for it, simple as that. People are likely to feel strongly enough about it that we'll be risking triggering a distro split. The ability to make that sort of choice is /why/ they are on gentoo. So by all means, present it as an option, document it in the handbooks, and even make it the default (and we all know how strongly people feel about even changing a default after the eudev debate), but I don't think it's a fight worth having to take away the other choices. At least not now. Maybe in five or ten years, after another generation of devs has come and gone, if the new one isn't interested in supporting the alternative any longer. (Again, I say this as one who has already done the merge here, if in reverse, and who fully intends to setup new systems the same way as well.) -- Duncan - List replies preferred. No HTML msgs. "Every nonfree program has a lord, a master -- and if you use the program, he is your master." Richard Stallman ^ permalink raw reply [flat|nested] 89+ messages in thread
* Re: [gentoo-dev] Re: usr merge 2016-04-07 15:12 ` [gentoo-dev] " Duncan @ 2016-04-07 15:42 ` Rich Freeman 2016-04-07 15:46 ` William Hubbs 0 siblings, 1 reply; 89+ messages in thread From: Rich Freeman @ 2016-04-07 15:42 UTC (permalink / raw To: gentoo-dev On Thu, Apr 7, 2016 at 11:12 AM, Duncan <1i5t5.duncan@cox.net> wrote: > William Hubbs posted on Thu, 07 Apr 2016 09:40:49 -0500 as excerpted: > >> After the testing period is over, I'm confused about why we should >> support both layouts. With separate usr without initramfs gone, the usr >> merge is transparent to end users because of the symbolic links in /, so >> there should be no reason to keep supporting both layouts once we are >> satisfied with the migration process. > > Because we're Gentoo, and gentooers tend to have rather strong opinions > on what sort of choices we should be able to make about things like that. > I'm trying to think of whether offering a choice really costs us anything. The main issue I see here is that the compatibility symlinks only go one way. #!/bin/bash will work whether you've done a usr merge or not #!/usr/bin/bash will probably only work if you've done the usr merge #!/usr/bin/python will work whether you've done a usr merge or not #!/bin/python will probably only work if you've done the usr merge It seems like a bit of a challenge to try to make sure that all your links are to wherever the original package tries to install files when on the system you are developing/testing on everything is in one place. We could of course require that maintainers accept patches to fix these kinds of broken links if they're offered, but users would be more likely to run into temporary breakage if they didn't use the merge unless we can come up with a way to offer compatibility in both directions. Unless there is a bigger problem lurking it probably still should be up to the user. -- Rich ^ permalink raw reply [flat|nested] 89+ messages in thread
* Re: [gentoo-dev] Re: usr merge 2016-04-07 15:42 ` Rich Freeman @ 2016-04-07 15:46 ` William Hubbs 2016-04-07 16:22 ` Rich Freeman 0 siblings, 1 reply; 89+ messages in thread From: William Hubbs @ 2016-04-07 15:46 UTC (permalink / raw To: gentoo-dev [-- Attachment #1: Type: text/plain, Size: 1529 bytes --] On Thu, Apr 07, 2016 at 11:42:01AM -0400, Rich Freeman wrote: > On Thu, Apr 7, 2016 at 11:12 AM, Duncan <1i5t5.duncan@cox.net> wrote: > > William Hubbs posted on Thu, 07 Apr 2016 09:40:49 -0500 as excerpted: > > > >> After the testing period is over, I'm confused about why we should > >> support both layouts. With separate usr without initramfs gone, the usr > >> merge is transparent to end users because of the symbolic links in /, so > >> there should be no reason to keep supporting both layouts once we are > >> satisfied with the migration process. > > > > Because we're Gentoo, and gentooers tend to have rather strong opinions > > on what sort of choices we should be able to make about things like that. > > > > I'm trying to think of whether offering a choice really costs us > anything. The main issue I see here is that the compatibility > symlinks only go one way. > > #!/bin/bash will work whether you've done a usr merge or not > #!/usr/bin/bash will probably only work if you've done the usr merge > #!/usr/bin/python will work whether you've done a usr merge or not > #!/bin/python will probably only work if you've done the usr merge That's correct, but you shouldn't be using shebangs like the second and fourth ones now either. The standard shebangs (the first and third ones) are fully compatible pre and post usr merge. If people decide to start using non-standard shebangs like your second and fourth ones above, that is wrong and should be stopped. William [-- Attachment #2: Digital signature --] [-- Type: application/pgp-signature, Size: 181 bytes --] ^ permalink raw reply [flat|nested] 89+ messages in thread
* Re: [gentoo-dev] Re: usr merge 2016-04-07 15:46 ` William Hubbs @ 2016-04-07 16:22 ` Rich Freeman 2016-04-07 16:36 ` [gentoo-dev] " Alexis Ballier 0 siblings, 1 reply; 89+ messages in thread From: Rich Freeman @ 2016-04-07 16:22 UTC (permalink / raw To: gentoo-dev On Thu, Apr 7, 2016 at 11:46 AM, William Hubbs <williamh@gentoo.org> wrote: >> #!/bin/bash will work whether you've done a usr merge or not >> #!/usr/bin/bash will probably only work if you've done the usr merge >> #!/usr/bin/python will work whether you've done a usr merge or not >> #!/bin/python will probably only work if you've done the usr merge > > That's correct, but you shouldn't be using shebangs like the second and > fourth ones now either. The standard shebangs (the first and third > ones) are fully compatible pre and post usr merge. > > If people decide to start using non-standard shebangs like your second > and fourth ones above, that is wrong and should be stopped. > Of course, but how will this be easily prevented? If a package (whether new or a routine update) changes a path somewhere it would work just fine in testing, due to the compatibility symlinks. I can't really think of any straightforward way to detect this in an automated fashion either, at least not 100% reliably. Upstream could introduce these paths without a developer noticing, or a developer might just not notice that netstat and bzip2 and more is in /bin, but lsof and xz and less are in /usr/bin. Since we do not have any policy as to what has to go in either path any of these are subject to change at any time as well. Heck, we struggle just to get people to stop using /etc/init.d/functions.sh. Again, I don't see this as a reason not to make it optional, but I suspect that we will find bugs here from time to time which users who run with the split /usr will have to report/fix. -- Rich ^ permalink raw reply [flat|nested] 89+ messages in thread
* Re: [gentoo-dev] usr merge 2016-04-07 16:22 ` Rich Freeman @ 2016-04-07 16:36 ` Alexis Ballier 2016-04-07 18:21 ` Raymond Jennings 2016-04-07 18:32 ` M. J. Everitt 0 siblings, 2 replies; 89+ messages in thread From: Alexis Ballier @ 2016-04-07 16:36 UTC (permalink / raw To: gentoo-dev On Thursday, April 7, 2016 6:22:16 PM CEST, Rich Freeman wrote: > Again, I don't see this as a reason not to make it optional, but I > suspect that we will find bugs here from time to time which users who > run with the split /usr will have to report/fix. Considering the advantages of usr-merge are rather specific IMHO but risks during the migration are high, I think you're optimistic on the user base of usr-merged systems :) Heck, it hasn't happened yet because there hasn't been such a big need for it. ^ permalink raw reply [flat|nested] 89+ messages in thread
* Re: [gentoo-dev] usr merge 2016-04-07 16:36 ` [gentoo-dev] " Alexis Ballier @ 2016-04-07 18:21 ` Raymond Jennings 2016-04-07 18:32 ` M. J. Everitt 1 sibling, 0 replies; 89+ messages in thread From: Raymond Jennings @ 2016-04-07 18:21 UTC (permalink / raw To: gentoo-dev [-- Attachment #1: Type: text/plain, Size: 1018 bytes --] May I suggest first moving everything into /usr one at a time, and for each file moved out of /bin or /sbin or whatever, replace it with a symlink? This will allow the /bin and /sbin directories themselves to atomically be replaced with symlinks later. Doing it all at once will leave a gap. For each file: 1. Install it in the new location 2. Delete the old file and replace it with a symlink On Thu, Apr 7, 2016 at 9:36 AM, Alexis Ballier <aballier@gentoo.org> wrote: > On Thursday, April 7, 2016 6:22:16 PM CEST, Rich Freeman wrote: > >> Again, I don't see this as a reason not to make it optional, but I >> suspect that we will find bugs here from time to time which users who >> run with the split /usr will have to report/fix. >> > > > Considering the advantages of usr-merge are rather specific IMHO but risks > during the migration are high, I think you're optimistic on the user base > of usr-merged systems :) > > Heck, it hasn't happened yet because there hasn't been such a big need for > it. > > [-- Attachment #2: Type: text/html, Size: 1578 bytes --] ^ permalink raw reply [flat|nested] 89+ messages in thread
* Re: [gentoo-dev] usr merge 2016-04-07 16:36 ` [gentoo-dev] " Alexis Ballier 2016-04-07 18:21 ` Raymond Jennings @ 2016-04-07 18:32 ` M. J. Everitt 2016-04-07 18:54 ` Rich Freeman 1 sibling, 1 reply; 89+ messages in thread From: M. J. Everitt @ 2016-04-07 18:32 UTC (permalink / raw To: gentoo-dev [-- Attachment #1: Type: text/plain, Size: 1035 bytes --] On 07/04/16 17:36, Alexis Ballier wrote: > On Thursday, April 7, 2016 6:22:16 PM CEST, Rich Freeman wrote: >> Again, I don't see this as a reason not to make it optional, but I >> suspect that we will find bugs here from time to time which users who >> run with the split /usr will have to report/fix. > > > Considering the advantages of usr-merge are rather specific IMHO but > risks during the migration are high, I think you're optimistic on the > user base of usr-merged systems :) > > Heck, it hasn't happened yet because there hasn't been such a big need > for it. > In the spirit of hearing arguments for/against .. could someone with the appropriate 'fu' throw up a quick survey for those on this ML (and/or possibly the g-users?) to indicate a preference for a change to a flattened-/usr system? I did think re: the eudev "debate" that it was really hard to quantify the opinion for and against a change, and take it away from the vocal people that obviously feel passionately about their cause :) . [-- Attachment #2: OpenPGP digital signature --] [-- Type: application/pgp-signature, Size: 901 bytes --] ^ permalink raw reply [flat|nested] 89+ messages in thread
* Re: [gentoo-dev] usr merge 2016-04-07 18:32 ` M. J. Everitt @ 2016-04-07 18:54 ` Rich Freeman 2016-04-07 20:18 ` Raymond Jennings 0 siblings, 1 reply; 89+ messages in thread From: Rich Freeman @ 2016-04-07 18:54 UTC (permalink / raw To: gentoo-dev On Thu, Apr 7, 2016 at 2:32 PM, M. J. Everitt <m.j.everitt@iee.org> wrote: > In the spirit of hearing arguments for/against .. could someone with the > appropriate 'fu' throw up a quick survey for those on this ML (and/or > possibly the g-users?) to indicate a preference for a change to a > flattened-/usr system? > > I did think re: the eudev "debate" that it was really hard to quantify > the opinion for and against a change, and take it away from the vocal > people that obviously feel passionately about their cause :) . > By all means do so, but we can probably save the trouble and assume that 95% of the respondents would prefer things remain as they are, and probably 80% would suggest that Gentoo should fully support systems without /usr mounted during early boot. Gentoo has become a fairly conservative distro, even more so when everybody else dropped support for not running systemd. I personally think the /usr merge is a cleaner approach (and I'd go a step further and merge sbin and bin), but it was rightly said that many of the benefits of a merge only come when you do a lot of other things as well. Of course, we could go ahead and do those things later. I think the main immediate benefit of a usr merge is that it actually reduces the risk of shebangs and such pointing to the wrong place (due to compat links, and there only being one right place in general), and it greatly consolidates the static stuff on the filesystem. -- Rich ^ permalink raw reply [flat|nested] 89+ messages in thread
* Re: [gentoo-dev] usr merge 2016-04-07 18:54 ` Rich Freeman @ 2016-04-07 20:18 ` Raymond Jennings 2016-04-08 1:39 ` William Hubbs 0 siblings, 1 reply; 89+ messages in thread From: Raymond Jennings @ 2016-04-07 20:18 UTC (permalink / raw To: gentoo-dev [-- Attachment #1: Type: text/plain, Size: 1777 bytes --] Personally I think that merging things into /usr is a major policy decision that is likely to contravene upstream installation locations. I wouldn't do it lightly, if at all. On Thu, Apr 7, 2016 at 11:54 AM, Rich Freeman <rich0@gentoo.org> wrote: > On Thu, Apr 7, 2016 at 2:32 PM, M. J. Everitt <m.j.everitt@iee.org> wrote: > > In the spirit of hearing arguments for/against .. could someone with the > > appropriate 'fu' throw up a quick survey for those on this ML (and/or > > possibly the g-users?) to indicate a preference for a change to a > > flattened-/usr system? > > > > I did think re: the eudev "debate" that it was really hard to quantify > > the opinion for and against a change, and take it away from the vocal > > people that obviously feel passionately about their cause :) . > > > > By all means do so, but we can probably save the trouble and assume > that 95% of the respondents would prefer things remain as they are, > and probably 80% would suggest that Gentoo should fully support > systems without /usr mounted during early boot. > > Gentoo has become a fairly conservative distro, even more so when > everybody else dropped support for not running systemd. > > I personally think the /usr merge is a cleaner approach (and I'd go a > step further and merge sbin and bin), but it was rightly said that > many of the benefits of a merge only come when you do a lot of other > things as well. Of course, we could go ahead and do those things > later. > > I think the main immediate benefit of a usr merge is that it actually > reduces the risk of shebangs and such pointing to the wrong place (due > to compat links, and there only being one right place in general), and > it greatly consolidates the static stuff on the filesystem. > > -- > Rich > > [-- Attachment #2: Type: text/html, Size: 2357 bytes --] ^ permalink raw reply [flat|nested] 89+ messages in thread
* Re: [gentoo-dev] usr merge 2016-04-07 20:18 ` Raymond Jennings @ 2016-04-08 1:39 ` William Hubbs 2016-04-08 1:42 ` William Hubbs 0 siblings, 1 reply; 89+ messages in thread From: William Hubbs @ 2016-04-08 1:39 UTC (permalink / raw To: gentoo-dev [-- Attachment #1: Type: text/plain, Size: 589 bytes --] On Thu, Apr 07, 2016 at 01:18:01PM -0700, Raymond Jennings wrote: > Personally I think that merging things into /usr is a major policy decision > that is likely to contravene upstream installation locations. I wouldn't > do it lightly, if at all. Actually, there are upstreams that already do this, and we are the ones that move things around. Specifically, one example is coreutils. The ebuild installs everything in /usr/bin, then we move all of the binaries around. Also, any time we run gen_usr_ldscript in an ebuild this is going against upstream installation locations. William [-- Attachment #2: Digital signature --] [-- Type: application/pgp-signature, Size: 181 bytes --] ^ permalink raw reply [flat|nested] 89+ messages in thread
* Re: [gentoo-dev] usr merge 2016-04-08 1:39 ` William Hubbs @ 2016-04-08 1:42 ` William Hubbs 2016-04-08 2:35 ` M. J. Everitt 2016-04-08 10:14 ` [gentoo-dev] " Rich Freeman 0 siblings, 2 replies; 89+ messages in thread From: William Hubbs @ 2016-04-08 1:42 UTC (permalink / raw To: gentoo-dev [-- Attachment #1: Type: text/plain, Size: 808 bytes --] On Thu, Apr 07, 2016 at 08:39:07PM -0500, William Hubbs wrote: > On Thu, Apr 07, 2016 at 01:18:01PM -0700, Raymond Jennings wrote: > > Personally I think that merging things into /usr is a major policy decision > > that is likely to contravene upstream installation locations. I wouldn't > > do it lightly, if at all. > > Actually, there are upstreams that already do this, and we are the ones > that move things around. > > Specifically, one example is coreutils. The ebuild installs everything > in /usr/bin, then we move all of the binaries around. There was a bypo here. "the ebuild" should be upstream. The default installation location of all coreutils binaries is /usr/bin, then we move everything around in the ebuild. We are deviating from upstream in this example. William [-- Attachment #2: Digital signature --] [-- Type: application/pgp-signature, Size: 181 bytes --] ^ permalink raw reply [flat|nested] 89+ messages in thread
* Re: [gentoo-dev] usr merge 2016-04-08 1:42 ` William Hubbs @ 2016-04-08 2:35 ` M. J. Everitt 2016-04-08 7:58 ` [gentoo-dev] " Duncan 2016-04-08 10:14 ` [gentoo-dev] " Rich Freeman 1 sibling, 1 reply; 89+ messages in thread From: M. J. Everitt @ 2016-04-08 2:35 UTC (permalink / raw To: gentoo-dev [-- Attachment #1: Type: text/plain, Size: 1693 bytes --] On 08/04/16 02:42, William Hubbs wrote: > On Thu, Apr 07, 2016 at 08:39:07PM -0500, William Hubbs wrote: >> On Thu, Apr 07, 2016 at 01:18:01PM -0700, Raymond Jennings wrote: >>> Personally I think that merging things into /usr is a major policy decision >>> that is likely to contravene upstream installation locations. I wouldn't >>> do it lightly, if at all. >> Actually, there are upstreams that already do this, and we are the ones >> that move things around. >> >> Specifically, one example is coreutils. The ebuild installs everything >> in /usr/bin, then we move all of the binaries around. > There was a bypo here. "the ebuild" should be upstream. The default > installation location of all coreutils binaries is /usr/bin, then we > move everything around in the ebuild. > We are deviating from upstream in this example. > > William > I would expect this isn't the only example of this in Gentoo .. we customise the packages to make sense to the Gentoo distro, not conform to a multitude of random "standards" applied by many developers. So, whilst I accept that its desirable to match 'upstream' - this isn't always going to be possible. I would also re-iterate, as I'm sure you're aware .. there ARE differences between sbin and bin .. unless of course you spend all your time in a Rooted VM where it doesn't matter if you accidentally trash your system. Some of us maintain a sensible user/superuser distinction for a variety of reasons, and simplifying your filesystem to suit some particular package style doesn't really sound like good reasoning for causing a lot of headaches for maintainers and a distro overall. *puts the paint can down* [-- Attachment #2: OpenPGP digital signature --] [-- Type: application/pgp-signature, Size: 819 bytes --] ^ permalink raw reply [flat|nested] 89+ messages in thread
* [gentoo-dev] Re: usr merge 2016-04-08 2:35 ` M. J. Everitt @ 2016-04-08 7:58 ` Duncan 2016-04-09 12:44 ` Nicolas Sebrecht 0 siblings, 1 reply; 89+ messages in thread From: Duncan @ 2016-04-08 7:58 UTC (permalink / raw To: gentoo-dev M. J. Everitt posted on Fri, 08 Apr 2016 03:35:33 +0100 as excerpted: > On 08/04/16 02:42, William Hubbs wrote: >> The default installation location of all coreutils binaries is >> /usr/bin, then we move everything around in the ebuild. >> We are deviating from upstream in this example. >> > I would expect this isn't the only example of this in Gentoo .. we > customise the packages to make sense to the Gentoo distro, not conform > to a multitude of random "standards" applied by many developers. So, > whilst I accept that its desirable to match 'upstream' - this isn't > always going to be possible. Agreed, and moreso, while gentoo does try to stay close to upstream in general, I'd argue that paths aren't the place to do that. Instead, for paths, we should be sticking as close to FHS, XDG/freedesktop.org, etc, as makes sense (and in general it's reasonable and /does/ make sense, tho when the specs are designed with for instance binary distros in mind, they don't always). Because there are specs, but upstreams may do all sorts of random things in terms of path, some of which aren't going to work particularly well on gentoo or other distros attempting to stay at least reasonably close to FHS and XDG/freedesktop.org specs. > I would also re-iterate, as I'm sure you're aware .. there ARE > differences between sbin and bin .. unless of course you spend all your > time in a Rooted VM where it doesn't matter if you accidentally trash > your system. Some of us maintain a sensible user/superuser distinction > for a variety of reasons, and simplifying your filesystem to suit some > particular package style doesn't really sound like good reasoning for > causing a lot of headaches for maintainers and a distro overall. But... the real important distinction in terms of user vs. superuser executables is file ownership and permissions, not the directory they're in. As long as the ownership and permissions are correct, the user can only run what they are supposed to, regardless of whether it's in bin or sbin. And as a user running the bin/sbin merge, I can tell you based on experience that tab-completion works differently for users vs. superusers with the merge just as it does without it, because again, it's based on ownership and permissions too, not just whether it's in the path (tho it must be that as well). Besides which, users unaccustomed to the CLI (and thus not knowing about tab completion) don't tend to know or care where binaries are anyway, as they prefer menu entries, complete with (graphical) sudo or these days, policy-kit integration. Which is ultimately what distros realized, bin/sbin didn't really matter to the general user any more, thus the bin/sbin merge, as having all installed executables in the same general location was easier to manage and didn't matter to (most) users anyway. But again, gentoo's about choice, and I'd hate to see that choice taken away from gentoo's users anyway, because many of them /do/ actually care, quite a bit in fact, which is why they're on gentoo in the first place. =:^) -- Duncan - List replies preferred. No HTML msgs. "Every nonfree program has a lord, a master -- and if you use the program, he is your master." Richard Stallman ^ permalink raw reply [flat|nested] 89+ messages in thread
* Re: [gentoo-dev] Re: usr merge 2016-04-08 7:58 ` [gentoo-dev] " Duncan @ 2016-04-09 12:44 ` Nicolas Sebrecht 2016-04-10 8:17 ` Duncan 0 siblings, 1 reply; 89+ messages in thread From: Nicolas Sebrecht @ 2016-04-09 12:44 UTC (permalink / raw To: gentoo-dev; +Cc: Nicolas Sebrecht On Fri, Apr 08, 2016 at 07:58:35AM +0000, Duncan wrote: > > I would also re-iterate, as I'm sure you're aware .. there ARE > > differences between sbin and bin .. unless of course you spend all your > > time in a Rooted VM where it doesn't matter if you accidentally trash > > your system. Some of us maintain a sensible user/superuser distinction > > for a variety of reasons, and simplifying your filesystem to suit some > > particular package style doesn't really sound like good reasoning for > > causing a lot of headaches for maintainers and a distro overall. > > But... the real important distinction in terms of user vs. superuser > executables is file ownership and permissions, not the directory they're > in. No. With a lightweight / I can install systems with two root filesystems that I rsync once I'm sure there's no regression. If one won't boot after an upgrade, I can just reboot and select the other root filesystem in grub. This is much more easy than anything else. -- Nicolas Sebrecht ^ permalink raw reply [flat|nested] 89+ messages in thread
* [gentoo-dev] Re: usr merge 2016-04-09 12:44 ` Nicolas Sebrecht @ 2016-04-10 8:17 ` Duncan 0 siblings, 0 replies; 89+ messages in thread From: Duncan @ 2016-04-10 8:17 UTC (permalink / raw To: gentoo-dev Nicolas Sebrecht posted on Sat, 09 Apr 2016 14:44:25 +0200 as excerpted: > On Fri, Apr 08, 2016 at 07:58:35AM +0000, Duncan wrote: > >> > I would also re-iterate, as I'm sure you're aware .. there ARE >> > differences between sbin and bin .. unless of course you spend all >> > your time in a Rooted VM where it doesn't matter if you accidentally >> > trash your system. Some of us maintain a sensible user/superuser >> > distinction for a variety of reasons, and simplifying your filesystem >> > to suit some particular package style doesn't really sound like good >> > reasoning for causing a lot of headaches for maintainers and a distro >> > overall. >> >> But... the real important distinction in terms of user vs. superuser >> executables is file ownership and permissions, not the directory >> they're in. > > No. With a lightweight / I can install systems with two root filesystems > that I rsync once I'm sure there's no regression. If one won't boot > after an upgrade, I can just reboot and select the other root filesystem > in grub. > > This is much more easy than anything else. Actually I do precisely that, with / itself, which here includes pretty much everything the package manager installs (with the exception of a few things in /var that need to be writable in normal operation, that are symlinked to /home/var as my / is normally read-only mounted), including the package database itself, so everything stays in sync with the package database tracking it, on the same filesystem. =:^) $ df / Filesystem 1M-blocks Used Available Use% Mounted on /dev/sda5 8192M 2819M 5132M 36% / That's the working-copy. It's actually a two-device btrfs raid1, 8 GiB per device (8 GiB partition on each of two SSDs). That already gives me device redundancy. The first backup is another 8 GiB, again actually two-device btrfs raid1, 8 GiB per device, on another partition on each of those ssds. That gives me fat-finger and broken update redundancy. The second backup is an 8 GiB reiserfs on spinning rust, giving me both filesystem type redundancy because btrfs is still stabilizing, and a second backup in case disaster strikes when I'm actually updating the primary backup, taking out both it and the working copy. All three are independently bootable from grub2 as installed on all three devices, using the working copy /boot on one of the ssds, the primary backup /boot on the other ssd, or the secondary backup /boot on the spinning rust. /home similarly has a working copy raid1 btrfs on the ssds, a primary backup on the ssds, and a reiserfs secondary backup on spinning rust. There are further backups on USB tho I don't keep them as current so if I actually had to fall back to them I'd have some work ahead of me. Actually, I don't even have to switch to the grub2 commandline to switch between the three, either. They're all selectable directly from my customized grub2 menu, as is init=/bin/bash, systemd emergency and rescue mode, etc. Of course I can go grub commandline if needed, but it's not needed for those entries as they're already available in the menu I've setup. And of course the grub installation and corresponding /boot I use is selectable directly from the BIOS. What's nice about this is that the 8 GiB root is plenty big enough to hold the entire working system, including all manpages, the X and KDE install, etc, so not only do I have full documentation to work with while I'm recovering my broken root, but I have a full X and kde-plasma, which with /home or one of its backups gives me a fully customized kde install as well. So I can load up X/kde-plasma, and fire up youtube for full- monitor viewing say some club music videos to keep me awake on the 42- inch, while I work from one of the backups to recover the working copy with multiple konsole terminals on the 48-inch below it, and the system performance graphs display on the 21-inch off to the side. Try doing all /that/ in recovery mode from your "lightweight" / backup. =:^) -- Duncan - List replies preferred. No HTML msgs. "Every nonfree program has a lord, a master -- and if you use the program, he is your master." Richard Stallman ^ permalink raw reply [flat|nested] 89+ messages in thread
* Re: [gentoo-dev] usr merge 2016-04-08 1:42 ` William Hubbs 2016-04-08 2:35 ` M. J. Everitt @ 2016-04-08 10:14 ` Rich Freeman 2016-04-08 11:31 ` Anthony G. Basile 1 sibling, 1 reply; 89+ messages in thread From: Rich Freeman @ 2016-04-08 10:14 UTC (permalink / raw To: gentoo-dev On Thu, Apr 7, 2016 at 9:42 PM, William Hubbs <williamh@gentoo.org> wrote: > > There was a bypo here. "the ebuild" should be upstream. The default > installation location of all coreutils binaries is /usr/bin, then we > move everything around in the ebuild. > We are deviating from upstream in this example. > Keep in mind that following upstream and the /usr merge are somewhat orthogonal. You can just install those binaries in /usr without merging everything over. The only issue is that without the merge anybody embedding a path to these binaries would have to fix their packages. Presumably to aid the transition a symlink (at the file level) would be needed for some period of time. -- Rich ^ permalink raw reply [flat|nested] 89+ messages in thread
* Re: [gentoo-dev] usr merge 2016-04-08 10:14 ` [gentoo-dev] " Rich Freeman @ 2016-04-08 11:31 ` Anthony G. Basile 2016-04-08 11:41 ` James Le Cuirot 2016-04-08 22:20 ` Daniel Campbell 0 siblings, 2 replies; 89+ messages in thread From: Anthony G. Basile @ 2016-04-08 11:31 UTC (permalink / raw To: gentoo-dev On 4/8/16 6:14 AM, Rich Freeman wrote: > On Thu, Apr 7, 2016 at 9:42 PM, William Hubbs <williamh@gentoo.org> wrote: >> >> There was a bypo here. "the ebuild" should be upstream. The default >> installation location of all coreutils binaries is /usr/bin, then we >> move everything around in the ebuild. >> We are deviating from upstream in this example. >> > > Keep in mind that following upstream and the /usr merge are somewhat > orthogonal. You can just install those binaries in /usr without > merging everything over. > > The only issue is that without the merge anybody embedding a path to > these binaries would have to fix their packages. Presumably to aid > the transition a symlink (at the file level) would be needed for some > period of time. > @anyone, can you list the reasons we're doing this (I'm sure there's more than one). If systemd if one of them, then I'm confused because debian has switched to systemd and yet has not merged usr. -- Anthony G. Basile, Ph.D. Gentoo Linux Developer [Hardened] E-Mail : blueness@gentoo.org GnuPG FP : 1FED FAD9 D82C 52A5 3BAB DC79 9384 FA6E F52D 4BBA GnuPG ID : F52D4BBA ^ permalink raw reply [flat|nested] 89+ messages in thread
* Re: [gentoo-dev] usr merge 2016-04-08 11:31 ` Anthony G. Basile @ 2016-04-08 11:41 ` James Le Cuirot 2016-04-08 11:52 ` Rich Freeman 2016-04-08 11:54 ` Anthony G. Basile 2016-04-08 22:20 ` Daniel Campbell 1 sibling, 2 replies; 89+ messages in thread From: James Le Cuirot @ 2016-04-08 11:41 UTC (permalink / raw To: gentoo-dev On Fri, 8 Apr 2016 07:31:03 -0400 "Anthony G. Basile" <blueness@gentoo.org> wrote: > On 4/8/16 6:14 AM, Rich Freeman wrote: > > On Thu, Apr 7, 2016 at 9:42 PM, William Hubbs <williamh@gentoo.org> > > wrote: > >> > >> There was a bypo here. "the ebuild" should be upstream. The default > >> installation location of all coreutils binaries is /usr/bin, then > >> we move everything around in the ebuild. > >> We are deviating from upstream in this example. > >> > > > > Keep in mind that following upstream and the /usr merge are somewhat > > orthogonal. You can just install those binaries in /usr without > > merging everything over. > > > > The only issue is that without the merge anybody embedding a path to > > these binaries would have to fix their packages. Presumably to aid > > the transition a symlink (at the file level) would be needed for > > some period of time. > > @anyone, can you list the reasons we're doing this (I'm sure there's > more than one). If systemd if one of them, then I'm confused because > debian has switched to systemd and yet has not merged usr. Not that I'm for or against the merge but note that openSUSE, which has also switched to systemd, hasn't done the merge either. -- James Le Cuirot (chewi) Gentoo Linux Developer ^ permalink raw reply [flat|nested] 89+ messages in thread
* Re: [gentoo-dev] usr merge 2016-04-08 11:41 ` James Le Cuirot @ 2016-04-08 11:52 ` Rich Freeman 2016-04-08 11:54 ` Anthony G. Basile 1 sibling, 0 replies; 89+ messages in thread From: Rich Freeman @ 2016-04-08 11:52 UTC (permalink / raw To: gentoo-dev On Fri, Apr 8, 2016 at 7:41 AM, James Le Cuirot <chewi@gentoo.org> wrote: > On Fri, 8 Apr 2016 07:31:03 -0400 > "Anthony G. Basile" <blueness@gentoo.org> wrote: >> >> @anyone, can you list the reasons we're doing this (I'm sure there's >> more than one). If systemd if one of them, then I'm confused because >> debian has switched to systemd and yet has not merged usr. > > Not that I'm for or against the merge but note that openSUSE, which has > also switched to systemd, hasn't done the merge either. > systemd and /usr merge are also fairly orthogonal. There are many reasons for a /usr merge, but most tend to revolve around getting all the relatively static distro-supplied content into a single place. That makes it easier to make a separate filesystem, or read-only, or signature-checked, or atomically updated, or shared, and so on. Some of those derivative benefits may be harder to realize on Gentoo, but I think the principle is worth considering. -- Rich ^ permalink raw reply [flat|nested] 89+ messages in thread
* Re: [gentoo-dev] usr merge 2016-04-08 11:41 ` James Le Cuirot 2016-04-08 11:52 ` Rich Freeman @ 2016-04-08 11:54 ` Anthony G. Basile 2016-04-08 12:55 ` Rich Freeman 1 sibling, 1 reply; 89+ messages in thread From: Anthony G. Basile @ 2016-04-08 11:54 UTC (permalink / raw To: gentoo-dev On 4/8/16 7:41 AM, James Le Cuirot wrote: > On Fri, 8 Apr 2016 07:31:03 -0400 > "Anthony G. Basile" <blueness@gentoo.org> wrote: > >> On 4/8/16 6:14 AM, Rich Freeman wrote: >>> On Thu, Apr 7, 2016 at 9:42 PM, William Hubbs <williamh@gentoo.org> >>> wrote: >>>> >>>> There was a bypo here. "the ebuild" should be upstream. The default >>>> installation location of all coreutils binaries is /usr/bin, then >>>> we move everything around in the ebuild. >>>> We are deviating from upstream in this example. >>>> >>> >>> Keep in mind that following upstream and the /usr merge are somewhat >>> orthogonal. You can just install those binaries in /usr without >>> merging everything over. >>> >>> The only issue is that without the merge anybody embedding a path to >>> these binaries would have to fix their packages. Presumably to aid >>> the transition a symlink (at the file level) would be needed for >>> some period of time. >> >> @anyone, can you list the reasons we're doing this (I'm sure there's >> more than one). If systemd if one of them, then I'm confused because >> debian has switched to systemd and yet has not merged usr. > > Not that I'm for or against the merge but note that openSUSE, which has > also switched to systemd, hasn't done the merge either. > As I'm getting into this thread, I'm looking at debian, fedora and I'll add openSUSE. I just don't get why a usr merge is as good as that fedora page says. -- Anthony G. Basile, Ph.D. Gentoo Linux Developer [Hardened] E-Mail : blueness@gentoo.org GnuPG FP : 1FED FAD9 D82C 52A5 3BAB DC79 9384 FA6E F52D 4BBA GnuPG ID : F52D4BBA ^ permalink raw reply [flat|nested] 89+ messages in thread
* Re: [gentoo-dev] usr merge 2016-04-08 11:54 ` Anthony G. Basile @ 2016-04-08 12:55 ` Rich Freeman 2016-04-09 12:52 ` Luca Barbato 0 siblings, 1 reply; 89+ messages in thread From: Rich Freeman @ 2016-04-08 12:55 UTC (permalink / raw To: gentoo-dev On Fri, Apr 8, 2016 at 7:54 AM, Anthony G. Basile <blueness@gentoo.org> wrote: > > As I'm getting into this thread, I'm looking at debian, fedora and I'll > add openSUSE. I just don't get why a usr merge is as good as that > fedora page says. > Keep in mind Fedora's purposes here: 1. It is a feeder where experimental technologies are previewed/developed. 2. It is feeding into RHEL, which is targeted at infrequently-updating users who run in a release-based atmosphere. The purpose of a /usr merge is to get all the stateless stuff into one place. Some of the ultimate goals include: 1. A read-only /usr 2. Having /usr signature-verified at boot 3. Having everything that runs signature-checked before it is run 4. Having /usr shared across many containers/etc. 5. Stateless systems - boot with a /usr and it creates the rest dynamically, and they're lost when the container is shut down. Any of these COULD be implemented on Gentoo, though whether it will happen is questionable. Some of these like #5 would require more invasive changes to how we do things. However, the principle of having everything that is static in one place does make sense. Put it this way, if you were designing a new OS from scratch today, would it make more sense to put all the distro-supplied binaries/libraries under a single path off the root, or off of many paths from the root? The main driver for having a split /usr is legacy, IMO. Apparently even the unix authors said that they originally did it only because of the size of one of their disks and they wanted root to be a secondary bootloader. -- Rich ^ permalink raw reply [flat|nested] 89+ messages in thread
* Re: [gentoo-dev] usr merge 2016-04-08 12:55 ` Rich Freeman @ 2016-04-09 12:52 ` Luca Barbato 0 siblings, 0 replies; 89+ messages in thread From: Luca Barbato @ 2016-04-09 12:52 UTC (permalink / raw To: gentoo-dev On 08/04/16 14:55, Rich Freeman wrote: > The purpose of a /usr merge is to get all the stateless stuff into one place. beside what you have in /etc ... usr-merge, in practice just moves early-boot/core tools where the rest of the userspace lives. > Some of the ultimate goals include: > 1. A read-only /usr And mixing early-boot tools with post-boot userspace would help how? > 2. Having /usr signature-verified at boot Because /etc is totally unimportant. > 3. Having everything that runs signature-checked before it is run Because obviously you do not need to signature-check per executable. > 4. Having /usr shared across many containers/etc. Because obviously it is the early-boot userspace spoiling this. > 5. Stateless systems - boot with a /usr and it creates the rest > dynamically, and they're lost when the container is shut down. Sounds backwards in many different ways. > Put it this way, if you were designing a new OS from scratch today, > would it make more sense to put all the distro-supplied > binaries/libraries under a single path off the root, or off of many > paths from the root? You mean /usr/local ? The whole thing ceases to be important once you have bind-mount and PATH imho. There is the specific need to have all the tools needed to boot in a single place that can be accessed with ease. It being /bin or initramfs or /boot/bin is completely cosmetic. But you need a easy and reliable way to get it. The idea of having / just holding the mount points and then have all the other paths mounted by the early boot is fun only on paper I'm afraid. (and we aren't even getting there since I bet /etc will stay in the root partition for ages). lu ^ permalink raw reply [flat|nested] 89+ messages in thread
* Re: [gentoo-dev] usr merge 2016-04-08 11:31 ` Anthony G. Basile 2016-04-08 11:41 ` James Le Cuirot @ 2016-04-08 22:20 ` Daniel Campbell 2016-04-09 0:42 ` William Hubbs 1 sibling, 1 reply; 89+ messages in thread From: Daniel Campbell @ 2016-04-08 22:20 UTC (permalink / raw To: gentoo-dev -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 On 04/08/2016 04:31 AM, Anthony G. Basile wrote: > On 4/8/16 6:14 AM, Rich Freeman wrote: >> On Thu, Apr 7, 2016 at 9:42 PM, William Hubbs >> <williamh@gentoo.org> wrote: >>> >>> There was a bypo here. "the ebuild" should be upstream. The >>> default installation location of all coreutils binaries is >>> /usr/bin, then we move everything around in the ebuild. We are >>> deviating from upstream in this example. >>> >> >> Keep in mind that following upstream and the /usr merge are >> somewhat orthogonal. You can just install those binaries in /usr >> without merging everything over. >> >> The only issue is that without the merge anybody embedding a path >> to these binaries would have to fix their packages. Presumably >> to aid the transition a symlink (at the file level) would be >> needed for some period of time. >> > > > @anyone, can you list the reasons we're doing this (I'm sure > there's more than one). If systemd if one of them, then I'm > confused because debian has switched to systemd and yet has not > merged usr. > Based on what I've read here in the thread, merging /bin and /sbin into /usr/{sbin,bin} is a matter of convenience by putting most of the static parts of a running system into a single path. As mentioned by some people, however, that's not enough to make deployment across multiple machines super simple. The distros that focus on that aren't rolling release like we are, and thus don't face the same difficulties that we do. In addition, Gentoo supports a broad number of choices for users and some are advocating for an option. At a higher level, I'm not really sure why we're discussing it. Perhaps I missed it, but I didn't see an actual problem that someone was having mentioned anywhere. The /usr merge seems to me as a partial "solution" for a different type of environment; one that, arguably, is better suited for a distro that's designed for such deployments. I personally think sharing /usr over a network and deploying it to multiple machines could be a recipe for disaster. It seems like a business case scenario that would involve multiple other system changes. It sounds like a great case for adding another profile or something rather than changing things tree-wide. Maybe it's a case for making profiles more powerful and flexible. Regardless, I'd hate to see choice diminished here for the sake of a single set of rather narrow use-cases. Just my 2¢. - -- Daniel Campbell - Gentoo Developer OpenPGP Key: 0x1EA055D6 @ hkp://keys.gnupg.net fpr: AE03 9064 AE00 053C 270C 1DE4 6F7A 9091 1EA0 55D6 -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iQIcBAEBCAAGBQJXCC6jAAoJEAEkDpRQOeFwD+kP/j1EYtnbgh/s4i4lTn14vuoY Fj57eCsCBLNbHlEEGesLEU3EEExtJgVeSC9emuI80/nynOM8dhqUUKjtoZwwBW5R 9P5QmMQdT+ScJHPQ6CL5IKh9UeAF4IgDbrJI9rdHnrRLtlE40xWDRp8NcB5fPAc2 EfLMFRZs3HpJpVMirIVIMTHEckDlMRmYzO9aqKCnmSCx3M/nKR/SWSSQ94Acet9C DPN6nLH42vosJv1+syNUHGqf4DLn6xTREx7DEP8fBUJuQi/wDpHbbRn8PON3WkCo FkDqjxd3AhahUpa2LaD4t/sRmvs+tjIXfgFJ8iYzJwjRQKKZvSHRlQzUwQNnI6mS fkunumIhcJdeWCBXegaSxouAtaua7pk+AXDLx+2ZjhxDmv/BbmC6RSPWkHo3wsMl TVbZSB4IrDrp8l3kYd+baBEtZicNgoma4jak1MFn1COFWR2/ZqtGaOVDErW92aX2 jVQkTGTUGrFNx24ZPcvPB0OVNleoOfhh97O3EFjeOl0TnbAziGX9Z75HPegLFlLR 5dXewskG6iACR2FATlhhOgscfIVwE2LxEWw/N6lm0NbZHKTOObzP39fdu0CisAck pXtd1WBYX349Z0ympl9PCdLd68SobBROsFHKE6rwkSLkCBadzOCxgjpEGIQOPKHu qpXF8Z8y/Hssr4SAsxb0 =4UKO -----END PGP SIGNATURE----- ^ permalink raw reply [flat|nested] 89+ messages in thread
* Re: [gentoo-dev] usr merge 2016-04-08 22:20 ` Daniel Campbell @ 2016-04-09 0:42 ` William Hubbs 2016-04-09 1:11 ` Anthony G. Basile 2016-04-09 14:11 ` Ian Stakenvicius 0 siblings, 2 replies; 89+ messages in thread From: William Hubbs @ 2016-04-09 0:42 UTC (permalink / raw To: gentoo-dev [-- Attachment #1: Type: text/plain, Size: 2487 bytes --] On Fri, Apr 08, 2016 at 03:20:24PM -0700, Daniel Campbell wrote: > Based on what I've read here in the thread, merging /bin and /sbin > into /usr/{sbin,bin} is a matter of convenience by putting most of the > static parts of a running system into a single path. As mentioned by > some people, however, that's not enough to make deployment across > multiple machines super simple. The distros that focus on that aren't > rolling release like we are, and thus don't face the same difficulties > that we do. In addition, Gentoo supports a broad number of choices for > users and some are advocating for an option. It is true that we offer a high degree of choice to users, but one of those choices is not which paths to install binaries and libraries into. We install some binaries/libraries in /{bin,sbin,lib*} and others in /usr/{bin,sbin,lib*}; the users don't get to choose which binaries and libraries go where. > At a higher level, I'm not really sure why we're discussing it. > Perhaps I missed it, but I didn't see an actual problem that someone > was having mentioned anywhere. The /usr merge seems to me as a partial > "solution" for a different type of environment; one that, arguably, is > better suited for a distro that's designed for such deployments. It would, for us, eliminate a lot of customization in the base-system ebuilds, for example, all of the rearrangement of binaries in coreutils, splitting of the binaries between / and /usr in procps, all calls to gen_usr_ldscript in any ebuilds, among other things. In short, it would make packaging simpler, and maintain backward compatibility at the same time since the symlinks in / would exist. > I personally think sharing /usr over a network and deploying it to > multiple machines could be a recipe for disaster. It seems like a > business case scenario that would involve multiple other system > changes. It sounds like a great case for adding another profile or > something rather than changing things tree-wide. Maybe it's a case for > making profiles more powerful and flexible. Regardless, I'd hate to > see choice diminished here for the sake of a single set of rather > narrow use-cases. Based on what I said above, I don't see what choice is being diminished by the /usr merge, since we do not give users a choice about how their file system is laid out, or where packages are installed. If I'm honestly missing something, enlighten me. :-) William [-- Attachment #2: Digital signature --] [-- Type: application/pgp-signature, Size: 181 bytes --] ^ permalink raw reply [flat|nested] 89+ messages in thread
* Re: [gentoo-dev] usr merge 2016-04-09 0:42 ` William Hubbs @ 2016-04-09 1:11 ` Anthony G. Basile 2016-04-09 1:36 ` William Hubbs 2016-04-09 14:11 ` Ian Stakenvicius 1 sibling, 1 reply; 89+ messages in thread From: Anthony G. Basile @ 2016-04-09 1:11 UTC (permalink / raw To: gentoo-dev On 4/8/16 8:42 PM, William Hubbs wrote: > > It is true that we offer a high degree of choice to users, but one of > those choices is not which paths to install binaries and libraries > into. I thought vapier was introducing a switch USE=usr-sep which allowed us to keep an unmerged /usr, or are we completely eliminating this choice? -- Anthony G. Basile, Ph.D. Gentoo Linux Developer [Hardened] E-Mail : blueness@gentoo.org GnuPG FP : 1FED FAD9 D82C 52A5 3BAB DC79 9384 FA6E F52D 4BBA GnuPG ID : F52D4BBA ^ permalink raw reply [flat|nested] 89+ messages in thread
* Re: [gentoo-dev] usr merge 2016-04-09 1:11 ` Anthony G. Basile @ 2016-04-09 1:36 ` William Hubbs 2016-04-09 1:51 ` Anthony G. Basile 0 siblings, 1 reply; 89+ messages in thread From: William Hubbs @ 2016-04-09 1:36 UTC (permalink / raw To: gentoo-dev [-- Attachment #1: Type: text/plain, Size: 810 bytes --] On Fri, Apr 08, 2016 at 09:11:48PM -0400, Anthony G. Basile wrote: > On 4/8/16 8:42 PM, William Hubbs wrote: > > > > > It is true that we offer a high degree of choice to users, but one of > > those choices is not which paths to install binaries and libraries > > into. > > I thought vapier was introducing a switch USE=usr-sep which allowed us > to keep an unmerged /usr, or are we completely eliminating this choice? This use flag, sep-usr, has nothing to do with the /usr merge. It was added as a way to allow a few more people to use separate /usr configurations (this means/ and /usr on separate filesystems) without initramfs, before the council decided that all who have separate /usr should be using an initramfs. Separate /usr does not preclude merging / into /usr. William [-- Attachment #2: Digital signature --] [-- Type: application/pgp-signature, Size: 181 bytes --] ^ permalink raw reply [flat|nested] 89+ messages in thread
* Re: [gentoo-dev] usr merge 2016-04-09 1:36 ` William Hubbs @ 2016-04-09 1:51 ` Anthony G. Basile 2016-04-09 3:03 ` Rich Freeman 0 siblings, 1 reply; 89+ messages in thread From: Anthony G. Basile @ 2016-04-09 1:51 UTC (permalink / raw To: gentoo-dev On 4/8/16 9:36 PM, William Hubbs wrote: > On Fri, Apr 08, 2016 at 09:11:48PM -0400, Anthony G. Basile wrote: >> On 4/8/16 8:42 PM, William Hubbs wrote: >> >>> >>> It is true that we offer a high degree of choice to users, but one of >>> those choices is not which paths to install binaries and libraries >>> into. >> >> I thought vapier was introducing a switch USE=usr-sep which allowed us >> to keep an unmerged /usr, or are we completely eliminating this choice? > > This use flag, sep-usr, has nothing to do with the /usr merge. It was > added as a way to allow a few more people to use separate /usr > configurations (this means/ and /usr on separate > filesystems) without initramfs, before the council decided that all who > have separate /usr should be using an initramfs. > > Separate /usr does not preclude merging / into /usr. > > William > So I'm still not seeing a great gain from this merger. It seems like you think the linker scripts are something bad. Why? And you don't seem to like that we move some things around between / and /usr for pkgs like coreutils. But other than coreutils, I don't know many pkgs where we do that. Alternatively, this may introduce problems. So it seems like we're fixing something that isn't broken. -- Anthony G. Basile, Ph.D. Gentoo Linux Developer [Hardened] E-Mail : blueness@gentoo.org GnuPG FP : 1FED FAD9 D82C 52A5 3BAB DC79 9384 FA6E F52D 4BBA GnuPG ID : F52D4BBA ^ permalink raw reply [flat|nested] 89+ messages in thread
* Re: [gentoo-dev] usr merge 2016-04-09 1:51 ` Anthony G. Basile @ 2016-04-09 3:03 ` Rich Freeman 2016-04-09 4:06 ` Anthony G. Basile 0 siblings, 1 reply; 89+ messages in thread From: Rich Freeman @ 2016-04-09 3:03 UTC (permalink / raw To: gentoo-dev On Fri, Apr 8, 2016 at 9:51 PM, Anthony G. Basile <blueness@gentoo.org> wrote: > > Alternatively, this may introduce problems. So it seems like we're > fixing something that isn't broken. > What problems are you anticipating, especially in light of the fact that many distros actually do it this way already? I don't really have a problem with making it optional or the default. It can also be left up to the maintainers, and of course somebody could even fork baselayout/etc if they wish and virtualize it in @system. Most things in Gentoo don't actually require a consensus to move forward, especially if they aren't defaults. In any case, what is the point of this thread? If somebody wants to implement a merged /usr what exactly is stopping them from doing so? -- Rich ^ permalink raw reply [flat|nested] 89+ messages in thread
* Re: [gentoo-dev] usr merge 2016-04-09 3:03 ` Rich Freeman @ 2016-04-09 4:06 ` Anthony G. Basile 2016-04-09 10:56 ` Rich Freeman 2016-04-09 15:11 ` William Hubbs 0 siblings, 2 replies; 89+ messages in thread From: Anthony G. Basile @ 2016-04-09 4:06 UTC (permalink / raw To: gentoo-dev On 4/8/16 11:03 PM, Rich Freeman wrote: > On Fri, Apr 8, 2016 at 9:51 PM, Anthony G. Basile <blueness@gentoo.org> wrote: >> >> Alternatively, this may introduce problems. So it seems like we're >> fixing something that isn't broken. >> > > What problems are you anticipating, especially in light of the fact > that many distros actually do it this way already? RBAC policy files for one. You'll probably break every single hardened gentoo server out there. scripts and programs that assume different executables with the same name at different points along the path, eg I know a company where they've set up an ssh wrapper at /usr/local/bin/ssh which wrap /usr/bin/ssh. security measures where you don't dereference sym links along $PATH because sym links can be used in various types of exploits. really, it doesn't take much imagination to come up with scenarios where you'll break people systems. > > I don't really have a problem with making it optional or the default. if we don't make it optional we're going to cause some serious headaches for people who are invested in the current status quo. > > It can also be left up to the maintainers, and of course somebody > could even fork baselayout/etc if they wish and virtualize it in > @system. Most things in Gentoo don't actually require a consensus to > move forward, especially if they aren't defaults. if we deprecate the linker scripts in /usr/lib by stubbing out gen_usr_ldscript, then its not as simple as "maintainer's choice". > > In any case, what is the point of this thread? If somebody wants to > implement a merged /usr what exactly is stopping them from doing so? i'm against something that doesn't maintain backwards compat. -- Anthony G. Basile, Ph.D. Gentoo Linux Developer [Hardened] E-Mail : blueness@gentoo.org GnuPG FP : 1FED FAD9 D82C 52A5 3BAB DC79 9384 FA6E F52D 4BBA GnuPG ID : F52D4BBA ^ permalink raw reply [flat|nested] 89+ messages in thread
* Re: [gentoo-dev] usr merge 2016-04-09 4:06 ` Anthony G. Basile @ 2016-04-09 10:56 ` Rich Freeman 2016-04-09 11:16 ` Anthony G. Basile 2016-04-09 15:11 ` William Hubbs 1 sibling, 1 reply; 89+ messages in thread From: Rich Freeman @ 2016-04-09 10:56 UTC (permalink / raw To: gentoo-dev On Sat, Apr 9, 2016 at 12:06 AM, Anthony G. Basile <blueness@gentoo.org> wrote: > On 4/8/16 11:03 PM, Rich Freeman wrote: >> >> What problems are you anticipating, especially in light of the fact >> that many distros actually do it this way already? > > RBAC policy files for one. You'll probably break every single hardened > gentoo server out there. I wasn't suggesting that some adjustments to packages wouldn't be needed to accommodate the change. I was talking about the long-term, after any necessary changes are made? > > scripts and programs that assume different executables with the same > name at different points along the path, eg I know a company where > they've set up an ssh wrapper at /usr/local/bin/ssh which wrap /usr/bin/ssh. I get your point, but the actual case you cited wouldn't be affected by a /usr merge. I appreciate that there are cases where something might be affected (though users shouldn't be sticking wrappers in /usr/bin anyway without packaging them). >> >> I don't really have a problem with making it optional or the default. > > if we don't make it optional we're going to cause some serious headaches > for people who are invested in the current status quo. If you want to use a distro where you can heavily invest in the status quo and not expect it to change I think you'd be better off with a distro like RHEL, which targets this niche almost explicitly. But, as I've said, I see no reason not to make it optional. A big part of why we CAN get stuff like this done is that we let people migrate themselves at their own pace. >> >> It can also be left up to the maintainers, and of course somebody >> could even fork baselayout/etc if they wish and virtualize it in >> @system. Most things in Gentoo don't actually require a consensus to >> move forward, especially if they aren't defaults. > > if we deprecate the linker scripts in /usr/lib by stubbing out > gen_usr_ldscript, then its not as simple as "maintainer's choice". Well, I don't hear toolchain asking to retire that function. If it is their desire to stop maintaining it, then somebody else could of course take over for them and preserve a choice. > >> >> In any case, what is the point of this thread? If somebody wants to >> implement a merged /usr what exactly is stopping them from doing so? > > i'm against something that doesn't maintain backwards compat. > Well, we all have the freedom to fork baselayout if it isn't maintained the way we want it to be. Currently, no policy exists that says the baselayout maintainers can't just maintain the package however they want to (other than the general QA practice of announcing/coordinating changes in advance with trackers/etc). I suppose if somebody wants to propose a policy that says otherwise it is their freedom to do so. Personally, I think our users would be better-served by making it a choice. That might be a choice that comes with some pros/cons, just like the choice to use an initramfs, or the choice to run systemd, or any other choice that we trust our users to make. -- Rich ^ permalink raw reply [flat|nested] 89+ messages in thread
* Re: [gentoo-dev] usr merge 2016-04-09 10:56 ` Rich Freeman @ 2016-04-09 11:16 ` Anthony G. Basile 2016-04-09 12:39 ` Anthony G. Basile 0 siblings, 1 reply; 89+ messages in thread From: Anthony G. Basile @ 2016-04-09 11:16 UTC (permalink / raw To: gentoo-dev On 4/9/16 6:56 AM, Rich Freeman wrote: > Personally, I think our users would be better-served by making it a > choice. Rich, we can bike shed for days. It would just be nice to hear from base-layout people whether it will be a choice or not. We need to know that so we can plan accordingly. -- Anthony G. Basile, Ph.D. Gentoo Linux Developer [Hardened] E-Mail : blueness@gentoo.org GnuPG FP : 1FED FAD9 D82C 52A5 3BAB DC79 9384 FA6E F52D 4BBA GnuPG ID : F52D4BBA ^ permalink raw reply [flat|nested] 89+ messages in thread
* Re: [gentoo-dev] usr merge 2016-04-09 11:16 ` Anthony G. Basile @ 2016-04-09 12:39 ` Anthony G. Basile 0 siblings, 0 replies; 89+ messages in thread From: Anthony G. Basile @ 2016-04-09 12:39 UTC (permalink / raw To: gentoo-dev On 4/9/16 7:16 AM, Anthony G. Basile wrote: > On 4/9/16 6:56 AM, Rich Freeman wrote: >> Personally, I think our users would be better-served by making it a >> choice. > > Rich, we can bike shed for days. It would just be nice to hear from > base-layout people whether it will be a choice or not. We need to know > that so we can plan accordingly. > @williamh Is there a plan here that I can read? -- Anthony G. Basile, Ph.D. Gentoo Linux Developer [Hardened] E-Mail : blueness@gentoo.org GnuPG FP : 1FED FAD9 D82C 52A5 3BAB DC79 9384 FA6E F52D 4BBA GnuPG ID : F52D4BBA ^ permalink raw reply [flat|nested] 89+ messages in thread
* Re: [gentoo-dev] usr merge 2016-04-09 4:06 ` Anthony G. Basile 2016-04-09 10:56 ` Rich Freeman @ 2016-04-09 15:11 ` William Hubbs 2016-04-09 17:25 ` Alexis Ballier 1 sibling, 1 reply; 89+ messages in thread From: William Hubbs @ 2016-04-09 15:11 UTC (permalink / raw To: gentoo-dev [-- Attachment #1: Type: text/plain, Size: 3270 bytes --] On Sat, Apr 09, 2016 at 12:06:47AM -0400, Anthony G. Basile wrote: > On 4/8/16 11:03 PM, Rich Freeman wrote: > > On Fri, Apr 8, 2016 at 9:51 PM, Anthony G. Basile <blueness@gentoo.org> wrote: > >> > >> Alternatively, this may introduce problems. So it seems like we're > >> fixing something that isn't broken. > >> > > > > What problems are you anticipating, especially in light of the fact > > that many distros actually do it this way already? > > RBAC policy files for one. You'll probably break every single hardened > gentoo server out there. Tell me more about this; I don't know a lot about what would break. Also, are you sure it would break, or are you just thinking that it would? > scripts and programs that assume different executables with the same > name at different points along the path, eg I know a company where > they've set up an ssh wrapper at /usr/local/bin/ssh which wrap /usr/bin/ssh. This won't break, because /usr/bin/ssh would still exist as it does and /usr/local is not touched. File colissions between the directories that are being merged would definitely be an issue that needs to be worked out before this could happen, and I understand that. I know of at least one, and we would need to find out if there are others. Forgetting about /usr/local since we don't control that, this type of file name collision across the bin directories is not good whether or not you merge /usr. It would cause issues in path name resolution. > security measures where you don't dereference sym links along $PATH > because sym links can be used in various types of exploits. Every amd64 gentoo system already has one of these, the "lib" symlink, both in / and /usr, so if you aren't dereferencing symlinks, aren't you already broken? > > > > I don't really have a problem with making it optional or the default. > > if we don't make it optional we're going to cause some serious headaches > for people who are invested in the current status quo. > > > > > It can also be left up to the maintainers, and of course somebody > > could even fork baselayout/etc if they wish and virtualize it in > > @system. Most things in Gentoo don't actually require a consensus to > > move forward, especially if they aren't defaults. > > if we deprecate the linker scripts in /usr/lib by stubbing out >gen_usr_ldscript, then its not as simple as "maintainer's choice". gen_usr_ldscript is only needed if you are using separate /usr without an initramfs. This is unsupported and orthogonal to the usr merge. > > > > In any case, what is the point of this thread? If somebody wants to > > implement a merged /usr what exactly is stopping them from doing so? > > i'm against something that doesn't maintain backwards compat. If there are ways that merging / into /usr does not maintain backward compatibility, I want to know what they are. This is not directed at anyone specifically, it is just a general comment. I've seen a lot of speculation on this thread about what might break, and a lot of comments about a perceived removal of choice. Can someone help get a summary together? let's get a single message summarizing everything. Thanks, William [-- Attachment #2: Digital signature --] [-- Type: application/pgp-signature, Size: 181 bytes --] ^ permalink raw reply [flat|nested] 89+ messages in thread
* Re: [gentoo-dev] usr merge 2016-04-09 15:11 ` William Hubbs @ 2016-04-09 17:25 ` Alexis Ballier 0 siblings, 0 replies; 89+ messages in thread From: Alexis Ballier @ 2016-04-09 17:25 UTC (permalink / raw To: gentoo-dev On Saturday, April 9, 2016 5:11:30 PM CEST, William Hubbs wrote: >>> ... >> if we don't make it optional we're going to cause some serious headaches >> for people who are invested in the current status quo. >> ... > > gen_usr_ldscript is only needed if you are using separate /usr without > an initramfs. This is unsupported and orthogonal to the usr merge. just one addition: for having self-contained / with /usr on another partition, you just need to move libfoo.so* to /lib; the ldscript is only useful because the linker searches /usr/lib first then /lib, and if libfoo.a is in /usr/lib, it picks that one. IOW: gen_usr_ldscript is not needed without static libs, even for the case you describe ^ permalink raw reply [flat|nested] 89+ messages in thread
* Re: [gentoo-dev] usr merge 2016-04-09 0:42 ` William Hubbs 2016-04-09 1:11 ` Anthony G. Basile @ 2016-04-09 14:11 ` Ian Stakenvicius 1 sibling, 0 replies; 89+ messages in thread From: Ian Stakenvicius @ 2016-04-09 14:11 UTC (permalink / raw To: gentoo-dev > On Apr 8, 2016, at 8:42 PM, William Hubbs <williamh@gentoo.org> wrote: > >> On Fri, Apr 08, 2016 at 03:20:24PM -0700, Daniel Campbell wrote: >> Based on what I've read here in the thread, merging /bin and /sbin >> into /usr/{sbin,bin} is a matter of convenience by putting most of the >> static parts of a running system into a single path. As mentioned by >> some people, however, that's not enough to make deployment across >> multiple machines super simple. The distros that focus on that aren't >> rolling release like we are, and thus don't face the same difficulties >> that we do. In addition, Gentoo supports a broad number of choices for >> users and some are advocating for an option. > > It is true that we offer a high degree of choice to users, but one of > those choices is not which paths to install binaries and libraries > into. > Sure we do. Users can do pretty much whatever convoluted file system hierarchy layout they want prior to unpacking the stage3 -- multiple volumes, symlinks or bind-mounts to combine dirs, etc etc. IMO support for this usr-merge should be left to that level of system configuration, as long as portage/other PMs support installing packages in such a way that the contents of /bin and /usr/bin don't collide with each other at merge time. In other words, we should not drop any form of support at all for non-usr-merged systems. Which means all of that ebuild cleanup WilliamH wants to do cannot happen. Which, IMO, makes the whole thing moot. ^ permalink raw reply [flat|nested] 89+ messages in thread
* Re: [gentoo-dev] usr merge 2016-04-06 15:52 ` Richard Yao 2016-04-06 20:43 ` William Hubbs @ 2016-04-07 9:03 ` Alexis Ballier 1 sibling, 0 replies; 89+ messages in thread From: Alexis Ballier @ 2016-04-07 9:03 UTC (permalink / raw To: gentoo-dev On Wednesday, April 6, 2016 5:52:52 PM CEST, Richard Yao wrote: > The original purpose of the /usr merge in Solaris was to make managing > updates easier. Redhat realized that and copied it. Copying it too > without doing the enabling work necessary for a rolling distribution > would be setting a trap for users who would think that they can manage > deployments of Gentoo like they can manage deployments Solaris and/or RHEL. You're tying the whole thing too much to solaris/rh ways. The benefits for us are far less than for them and managing updates via everything in /usr is certainly out of scope of the current proposal. ^ permalink raw reply [flat|nested] 89+ messages in thread
* Re: [gentoo-dev] usr merge 2016-04-05 1:19 [gentoo-dev] usr merge William Hubbs ` (2 preceding siblings ...) 2016-04-06 14:58 ` M. J. Everitt @ 2016-04-09 11:41 ` Luca Barbato 2016-04-09 11:53 ` Rich Freeman 2016-04-10 11:55 ` [gentoo-dev] " Joshua Kinard 4 siblings, 1 reply; 89+ messages in thread From: Luca Barbato @ 2016-04-09 11:41 UTC (permalink / raw To: gentoo-dev On 05/04/16 03:19, William Hubbs wrote: > Thoughts on any of this? The whole usr-merge moves the problem of putting stuff in / to putting the very same stuff in the initrd when something different from busybox (or equivalent) is needed to get the early boot mounting. Do we have a reliable way to address this now? If the answer is no, maybe we should focus on solving it first and then think how to move stuff around. lu ^ permalink raw reply [flat|nested] 89+ messages in thread
* Re: [gentoo-dev] usr merge 2016-04-09 11:41 ` Luca Barbato @ 2016-04-09 11:53 ` Rich Freeman 2016-04-09 12:27 ` Luca Barbato 0 siblings, 1 reply; 89+ messages in thread From: Rich Freeman @ 2016-04-09 11:53 UTC (permalink / raw To: gentoo-dev On Sat, Apr 9, 2016 at 7:41 AM, Luca Barbato <lu_zero@gentoo.org> wrote: > On 05/04/16 03:19, William Hubbs wrote: >> Thoughts on any of this? > > The whole usr-merge moves the problem of putting stuff in / to putting > the very same stuff in the initrd when something different from busybox > (or equivalent) is needed to get the early boot mounting. > > Do we have a reliable way to address this now? > Put the very same stuff in the initramfs? Most initramfs creation scripts should already do this automatically, and with compat symlinks even those that don't probably will still end up doing it anyway.. Apologies if I missed the point of your question. Are you looking for a solution OTHER than an initramfs? I imagine somebody could stick some kind of wrapper on /, but in general if you want /usr not on the root filesystem with a /usr merge you're going to have to jump through hoops if you're not using an initramfs. If you want a more traditional configuration where / is used to mount /usr then a merged /usr probably isn't for you. -- Rich ^ permalink raw reply [flat|nested] 89+ messages in thread
* Re: [gentoo-dev] usr merge 2016-04-09 11:53 ` Rich Freeman @ 2016-04-09 12:27 ` Luca Barbato 2016-04-09 12:37 ` Rich Freeman 0 siblings, 1 reply; 89+ messages in thread From: Luca Barbato @ 2016-04-09 12:27 UTC (permalink / raw To: gentoo-dev On 09/04/16 13:53, Rich Freeman wrote: > On Sat, Apr 9, 2016 at 7:41 AM, Luca Barbato <lu_zero@gentoo.org> wrote: >> On 05/04/16 03:19, William Hubbs wrote: >>> Thoughts on any of this? >> >> The whole usr-merge moves the problem of putting stuff in / to putting >> the very same stuff in the initrd when something different from busybox >> (or equivalent) is needed to get the early boot mounting. >> >> Do we have a reliable way to address this now? >> > > Put the very same stuff in the initramfs? Most initramfs creation > scripts should already do this automatically, and with compat symlinks > even those that don't probably will still end up doing it anyway.. The question is different: do they work reliably? usr-merge does not solve any problem in itself (and it is totally backwards, if somebody wants to simplify would do /usr -> /), but makes more evident that you might need lots of the userspace to successfully complete your early boot. lu ^ permalink raw reply [flat|nested] 89+ messages in thread
* Re: [gentoo-dev] usr merge 2016-04-09 12:27 ` Luca Barbato @ 2016-04-09 12:37 ` Rich Freeman 2016-04-09 13:03 ` Luca Barbato 0 siblings, 1 reply; 89+ messages in thread From: Rich Freeman @ 2016-04-09 12:37 UTC (permalink / raw To: gentoo-dev On Sat, Apr 9, 2016 at 8:27 AM, Luca Barbato <lu_zero@gentoo.org> wrote: > On 09/04/16 13:53, Rich Freeman wrote: >> Put the very same stuff in the initramfs? Most initramfs creation >> scripts should already do this automatically, and with compat symlinks >> even those that don't probably will still end up doing it anyway.. > > The question is different: do they work reliably? > I've certainly haven't had many problems with dracut. When it fails it is usually because I'm doing something ELSE that is off-the-wall and it just doesn't have a plugin for it yet. (And in those cases it isn't like the kernel tends to get it right without an initramfs.) I'd certainly want to test it on a merged /usr, but I'd be surprised if it doesn't work, since it was designed to run on distros that are using a merged /usr. > usr-merge does not solve any problem in itself (and it is totally > backwards, if somebody wants to simplify would do /usr -> /) I don't really have any devotion to the particular design, but half the point of the merge is to allow /usr to be read-only, on a filesystem remotely mounted, and so on. In an ideal world, you might argue that / should just be a tmpfs or something almost as ephemeral. It is just a place you hang everything else off of. But, of course moving all of /usr to / solves the early boot issue. But, if you're going to do that you might as well just put /usr on your root filesystem and have the same thing. The thing I like about the merge is that it basically puts all your distro-supplied stuff in one place. /usr basically becomes the OS minus state. If things started out that way and you just had a short stub loader that gets things initialized, and I were arguing that instead of that little initialization stub you should break up /usr so that the root count mount /usr, would that sound all that compelling? I think having it all in one mountpoint seems a lot more compelling. -- Rich ^ permalink raw reply [flat|nested] 89+ messages in thread
* Re: [gentoo-dev] usr merge 2016-04-09 12:37 ` Rich Freeman @ 2016-04-09 13:03 ` Luca Barbato 2016-04-10 7:38 ` [gentoo-dev] " Duncan 0 siblings, 1 reply; 89+ messages in thread From: Luca Barbato @ 2016-04-09 13:03 UTC (permalink / raw To: gentoo-dev On 09/04/16 14:37, Rich Freeman wrote: > I've certainly haven't had many problems with dracut. When it fails > it is usually because I'm doing something ELSE that is off-the-wall > and it just doesn't have a plugin for it yet. (And in those cases it > isn't like the kernel tends to get it right without an initramfs.) > > I'd certainly want to test it on a merged /usr, but I'd be surprised > if it doesn't work, since it was designed to run on distros that are > using a merged /usr. I think that should be the first thing to do not the last one =) > In an ideal world, you might argue that / should just be a tmpfs or > something almost as ephemeral. It is just a place you hang everything > else off of. That would be the core concept, but then you can just not have /bin /sbin /lib . > The thing I like about the merge is that it basically puts all your > distro-supplied stuff in one place. /usr basically becomes the OS > minus state. If things started out that way and you just had a short > stub loader that gets things initialized, and I were arguing that > instead of that little initialization stub you should break up /usr so > that the root count mount /usr, would that sound all that compelling? > I think having it all in one mountpoint seems a lot more compelling. you cannot ever have everything in 1 mount point, you just move the problem somewhere else you notice less (initramfs), but the problem remains and either is solved or not. having everything in /usr and then copy it over ${somewhere} is there, it can be debated if /bin or initramfs is the best place to put it. lu ^ permalink raw reply [flat|nested] 89+ messages in thread
* [gentoo-dev] Re: usr merge 2016-04-09 13:03 ` Luca Barbato @ 2016-04-10 7:38 ` Duncan 0 siblings, 0 replies; 89+ messages in thread From: Duncan @ 2016-04-10 7:38 UTC (permalink / raw To: gentoo-dev Luca Barbato posted on Sat, 09 Apr 2016 15:03:15 +0200 as excerpted: > On 09/04/16 14:37, Rich Freeman wrote: >> I've certainly haven't had many problems with dracut. When it fails it >> is usually because I'm doing something ELSE that is off-the-wall and it >> just doesn't have a plugin for it yet. (And in those cases it isn't >> like the kernel tends to get it right without an initramfs.) >> >> I'd certainly want to test it on a merged /usr, but I'd be surprised if >> it doesn't work, since it was designed to run on distros that are using >> a merged /usr. > > I think that should be the first thing to do not the last one =) FWIW, dracut works just fine with a "reverse-merged" /usr (usr -> .), as well as the bin/sbin merge. And if it works with that, it'll certainly work with (normal) merged usr, as AFAIK upstream's fedora/rh sponsored. >> In an ideal world, you might argue that / should just be a tmpfs or >> something almost as ephemeral. It is just a place you hang everything >> else off of. > That would be the core concept, but then you can just not have /bin > /sbin /lib That's in the context of (forward) /usr merge, which would make all those symlinks to the appropriate /usr location anyway. Those symlinks could of course be created dynamically, as could the various mountpoint directories. Of course /etc would have to be dynamically mounted in that scenario as well, but that's not a big issue as long as there's an initr* I actually thought about doing a tmpfs-based / here, or effectively just never doing the pivotroot off the initramfs, with everything else dynamically mounted over top the initramfs dirs, but decided to go a different way instead, putting (almost) everything installed by the PM on /, and doing a reverse-/usr-merge, with /usr -> . , with / then ro- mounted by default. Seemed simpler for what I wanted to do than the tmpfs or stay-on-initramfs / route. >> The thing I like about the merge is that it basically puts all your >> distro-supplied stuff in one place. /usr basically becomes the OS >> minus state. If things started out that way and you just had a short >> stub loader that gets things initialized, and I were arguing that >> instead of that little initialization stub you should break up /usr so >> that the root count mount /usr, would that sound all that compelling? I >> think having it all in one mountpoint seems a lot more compelling. > > you cannot ever have everything in 1 mount point, you just move the > problem somewhere else you notice less (initramfs), but the problem > remains and either is solved or not. > > having everything in /usr and then copy it over ${somewhere} is there, > it can be debated if /bin or initramfs is the best place to put it. I suppose many of us have made that point at least to ourselves, at some point. =:^) -- Duncan - List replies preferred. No HTML msgs. "Every nonfree program has a lord, a master -- and if you use the program, he is your master." Richard Stallman ^ permalink raw reply [flat|nested] 89+ messages in thread
* Re: [gentoo-dev] usr merge 2016-04-05 1:19 [gentoo-dev] usr merge William Hubbs ` (3 preceding siblings ...) 2016-04-09 11:41 ` Luca Barbato @ 2016-04-10 11:55 ` Joshua Kinard 2016-04-10 12:14 ` Rich Freeman 2016-04-10 12:26 ` Anthony G. Basile 4 siblings, 2 replies; 89+ messages in thread From: Joshua Kinard @ 2016-04-10 11:55 UTC (permalink / raw To: gentoo-dev On 04/04/2016 21:19, William Hubbs wrote: > All, > > I thought that since the usr merge is coming up again, and since I lost > track of the message where it was brought up, I would open a > new thread to discuss it. > > When it came up before, some were saying that the /usr merge violates > the fhs. I don't remember the specifics of what the claim was at the > time, (I'm sure someone will point it out if it is still a concern). > > I don't think creating usr merged stages would be that difficult. I > think it would just be a matter of creating a new version of baselayout > that puts these symlinks in place: > > /bin->usr/bin > /lib->usr/lib > /lib32->usr/lib32 > /lib64->usr/lib64 > /sbin->usr/bin > /usr/sbin->bin > > Once that is in place in a new baselayout, I think portage's colission > detection would be able to catch files that had the same names and were > originally in different paths when building the new stages. > > I put some thought also in how to nigrate live systems, and I'm not sure > what the best way to do that is. I wrote a script, which would do it in > theory, but I haven't tested because I only have one system and if > it breaks it the only way back would be to reinstall. > > The script is attached. > > > Thoughts on any of this? > > William I looked at Thunderbird, at my folder labeled "gentoo-dev", and it had "666" unread messages. I should've done the smart thing and closed my mail program. But noooo, I had to look inside the folder. I am now regretting this decision. *sigh*, I can see the thread has gone clongie 'round the blonger, so all I'll have to say is we should still try to maintain the choice for users. But, in order to evaluate what amount of effort is needed to maintain that choice, we need to know what packages break on such a setup, what the level of effort needed to fix them is, and do those fixes impact the non-split crowd. Create like, a table on the Wiki or some kind of metadata property per-package that can contain a boolean or tri-state flag indicating whether it works or doesn't work (or kinda works) on split-usr. Or a tracker on Bugzie. Something. Once we know this, then we can work out what's the minimum system that can be successfully run on split-usr. Then, knowing *that*, we can see if that system can be supported by our @system target or some minimal subset of @world. As new package versions come out of upstream, we update this metadata with changes to the split-usr status, and this then provides a history of the more or less amount of difficulty needed to maintain support for split-usr, and *then*, we can make an objective decision to continue supporting or not supporting the capability. As for me, I am flat out ruling out a full-reinstall of all my systems. I have fixed disk partition layouts on all of them that cannot be re-arranged unless I tar up each filesystem and temporarily move it off, then rebuild the MD-RAID and reformat the filesystems. I am simply not going to do that on my many SGI systems, and whatever facet of upstream, whether it's some core GNU package or RH itself, can go pound sand for all I care. I'll go back to a static /dev and I'll manually mknod any missing devices if I have to. You know it's getting ridiculous when you can maintain a Windows/NTFS partition layout easier than a Linux one. -- Joshua Kinard Gentoo/MIPS kumba@gentoo.org 6144R/F5C6C943 2015-04-27 177C 1972 1FB8 F254 BAD0 3E72 5C63 F4E3 F5C6 C943 "The past tempts us, the present confuses us, the future frightens us. And our lives slip away, moment by moment, lost in that vast, terrible in-between." --Emperor Turhan, Centauri Republic ^ permalink raw reply [flat|nested] 89+ messages in thread
* Re: [gentoo-dev] usr merge 2016-04-10 11:55 ` [gentoo-dev] " Joshua Kinard @ 2016-04-10 12:14 ` Rich Freeman 2016-04-10 12:34 ` Anthony G. Basile 2016-04-11 1:59 ` Joshua Kinard 2016-04-10 12:26 ` Anthony G. Basile 1 sibling, 2 replies; 89+ messages in thread From: Rich Freeman @ 2016-04-10 12:14 UTC (permalink / raw To: gentoo-dev On Sun, Apr 10, 2016 at 7:55 AM, Joshua Kinard <kumba@gentoo.org> wrote: > > Create like, a table on the Wiki or some kind of metadata property per-package > that can contain a boolean or tri-state flag indicating whether it works or > doesn't work (or kinda works) on split-usr. Or a tracker on Bugzie. Something. > I'm sure there will be a tracker for packages that don't work on a merged /usr. (We are already on a split /usr.) Honestly, I'm still not quite sure why we're even having this discussion. I don't think anybody actually intends to make any changes at all. If they do, they should issue some kind of plan and indicate what they're looking for from everybody else. Such as: "Hello, I help maintain baselayout and I intend to change /(s)lib and /(s)bin and /usr/sbin into symlinks to /usr/bin and move all the files into those directories there. To test this out now please do xyz, and report any bugs against tracker #123456." Or: "Hello, I help maintain baselayout and I just introduced a new USE flag which does ... I think it is something you should try out. Bugs can be reported at..." Or: "Hello, I think the baselayout maintainers are idiots and I just introduced librelayout which does ... You should definitely switch because only losers run with a split /usr. Bugs can be reported at... Oh, and my fancy librelayout doesn't need gen_usr_ldscript so I select_one_of('won't lift a finger to keep it working', 'will just laugh at the folks who are wasting their time keeping it working')." It seems to me that we're just having a general discussion about the pros/cons of a /usr merge. That is nice, but people are getting worked up because they think that somehow whoever "loses" this "discussion" will get something shoved down their throats or won't be able to have something nice. Almost every big change that has become popular in Gentoo just started out as another alternative, and support grew organically. I don't really see the need to reach some kind of consensus here. I'd love to have an option of a /usr merge and a migration path. I'd love to see it as the default, but that is a different discussion, and if it is optional then it is also a less contentious discussion whichever way it goes. -- Rich ^ permalink raw reply [flat|nested] 89+ messages in thread
* Re: [gentoo-dev] usr merge 2016-04-10 12:14 ` Rich Freeman @ 2016-04-10 12:34 ` Anthony G. Basile 2016-04-11 1:59 ` Joshua Kinard 1 sibling, 0 replies; 89+ messages in thread From: Anthony G. Basile @ 2016-04-10 12:34 UTC (permalink / raw To: gentoo-dev On 4/10/16 8:14 AM, Rich Freeman wrote: > > Honestly, I'm still not quite sure why we're even having this > discussion. I don't think anybody actually intends to make any > changes at all. If they do, they should issue some kind of plan and > indicate what they're looking for from everybody else. > Because William started this thread, http://www.mail-archive.com/gentoo-dev@lists.gentoo.org/msg73010.html and because the passive voice in the sentence "I thought that since the usr merge is coming up again" begs the question: Who is bringing it up and why? So ... questioning minds want to know. -- Anthony G. Basile, Ph.D. Gentoo Linux Developer [Hardened] E-Mail : blueness@gentoo.org GnuPG FP : 1FED FAD9 D82C 52A5 3BAB DC79 9384 FA6E F52D 4BBA GnuPG ID : F52D4BBA ^ permalink raw reply [flat|nested] 89+ messages in thread
* Re: [gentoo-dev] usr merge 2016-04-10 12:14 ` Rich Freeman 2016-04-10 12:34 ` Anthony G. Basile @ 2016-04-11 1:59 ` Joshua Kinard 1 sibling, 0 replies; 89+ messages in thread From: Joshua Kinard @ 2016-04-11 1:59 UTC (permalink / raw To: gentoo-dev On 04/10/2016 08:14, Rich Freeman wrote: > On Sun, Apr 10, 2016 at 7:55 AM, Joshua Kinard <kumba@gentoo.org> wrote: >> >> Create like, a table on the Wiki or some kind of metadata property per-package >> that can contain a boolean or tri-state flag indicating whether it works or >> doesn't work (or kinda works) on split-usr. Or a tracker on Bugzie. Something. >> > > I'm sure there will be a tracker for packages that don't work on a > merged /usr. (We are already on a split /usr.) > > Honestly, I'm still not quite sure why we're even having this > discussion. I don't think anybody actually intends to make any > changes at all. If they do, they should issue some kind of plan and > indicate what they're looking for from everybody else. Agreed. A plan is most definitely needed if we ever choose to pursue a policy of only supporting non-split-/usr installs, especially if we want to handle cases like mine where we try to make migration of existing installs possible or not. [snip] > It seems to me that we're just having a general discussion about the > pros/cons of a /usr merge. That is nice, but people are getting > worked up because they think that somehow whoever "loses" this > "discussion" will get something shoved down their throats or won't be > able to have something nice. "General discussion" -- hah! Maybe it's the way Thunderbird is displaying it, but I have five distinct, top-level threads in my gentoo-dev folder for this discussion. I think we left "general" back there after we broke through the plaid barrier. That said, I don't really think there are any pros or cons of split or merged. Largely, what's being discussed is the after-effects of what once was a common approach to filesystem layout. Myself, I only went the split-usr route because of habit, itself which started because when I set up my first Gentoo system. I studied both the then-Gentoo Security and Debian Security manuals, which indicated a split-/usr layout had certain advantages in that you could limit capabilities via mount options. Mainly, /usr didn't need devices, so "nodev" was common there. Along with /var having "noexec" and "nodev". I've pretty much stuck to that layout approach since then out of habit. Certainly, I've got a few VMs where I have just /, /boot, and /tmp as my only partitions, and there's no real noticeable difference except what's in /etc/fstab. --- I think where the problem ultimately arises is a subtle conflict between filesystem semantics and system-design philosophy against the Linux kernel's device architecture and management. It's long regarded that /bin and /sbin contain system-level binaries, and /usr/bin and /usr/sbin being for almost everything else. A.k.a., a two-level binary installation hierarchy (and the BSD's extend this with a third level under /usr/local). From the kernel angle, you have a monolithic kernel design where device drivers run in kernel space most of the time. This worked okay with traditional, static devices that didn't change a whole lot and whose resources could be determined at boot-time, before userspace is brought up. But once the Linux ecosystem needed devices that can come online from userspace or need their resource determination to be dynamic (e.g., for hotplugging), we went to the approach that the kernel needs to get out of managing devices altogether, and thus came up with udev for device management from userspace. Since udev is supposed to run from userspace, but kinda needs to interface with the kernel a lot, the split between what's system-specific and what's not clashes with the two-level file system layout. This wasn't anything new...this conflict has existed elsewhere for a long time (namely in X11), but it came to a head when udev maintainers (and later, systemd maintainers) questioned the approach, and largely decided it wasn't worth it, and a merged filesystem, with /usr not separate, was simpler and more elegant. --- But really, does it matter where the binaries all live? It's just a design decision. Every OS had a different approach, such as NetWare running virtually everything out of SYS:SYSTEM, and Windows out of C:\WINDOWS. Heck, for the longest time, you could *not* install Windows on anything other than the first partition of the first drive, because software literally hardcoded filespec strings as "C:\WINDOWS\...". And even why C:, the third letter of the alphabet, for the first fixed disk? Because A: and B: were reserved for systems that needed two floppy drives. Yay MS-DOS! If it were possible to give every binary and file out there a unique name, you wouldn't even need directories. You could have a totally-flat namespace with all files of any kind under /, and let security models handle who sees/accesses what. But in that setup, would you even need "/"? > Almost every big change that has become popular in Gentoo just started > out as another alternative, and support grew organically. I don't > really see the need to reach some kind of consensus here. I'd love to > have an option of a /usr merge and a migration path. I'd love to see > it as the default, but that is a different discussion, and if it is > optional then it is also a less contentious discussion whichever way > it goes. I don't think there is a viable migration path that can cover all scenarios. I have one of the simpler disk layouts, with MD-RAID to merge separate physical disks into /dev/md* and then my unencrypted filesystems live there. Certainly, my /usr partition on those systems is big enough that I could cp /* into /usr, then change my root= parameter to the kernel to have it treat my /usr partition as /. That'd be the simplest method for me, but it would leave me with a ~3GB block of dead diskspace on those systems. The only real way forward for migration would literally be to back the / and /usr filesystems up (minimum), blow away the filesystem (and RAID or whatever else) layouts, redesign it, then restore. For some people, this would only take a few hours. Others, it'd be days, if not longer. Some probably wouldn't mind the time investment. Others (like me), just don't have the time (or motivation) to do such a task right now. That is really my only issue here. I guess the saving grace is UNIX filesystems in general are trivially easy to backup/restore (or migrate, etc). I migrated a Windows install once and discovered that Windows records disk serial numbers in the Registry, which it used to quickly find the disk with C:. Took me several hours to figure that out, then to build a BartPE disk that I could use to edit the specific registry hive and delete those serials. Then, unsurprisingly, Windows just shrugs and uses the new disk serials as defaults and records them in place of the old ones. -- Joshua Kinard Gentoo/MIPS kumba@gentoo.org 6144R/F5C6C943 2015-04-27 177C 1972 1FB8 F254 BAD0 3E72 5C63 F4E3 F5C6 C943 "The past tempts us, the present confuses us, the future frightens us. And our lives slip away, moment by moment, lost in that vast, terrible in-between." --Emperor Turhan, Centauri Republic ^ permalink raw reply [flat|nested] 89+ messages in thread
* Re: [gentoo-dev] usr merge 2016-04-10 11:55 ` [gentoo-dev] " Joshua Kinard 2016-04-10 12:14 ` Rich Freeman @ 2016-04-10 12:26 ` Anthony G. Basile 1 sibling, 0 replies; 89+ messages in thread From: Anthony G. Basile @ 2016-04-10 12:26 UTC (permalink / raw To: gentoo-dev On 4/10/16 7:55 AM, Joshua Kinard wrote: > On 04/04/2016 21:19, William Hubbs wrote: >> All, >> >> I thought that since the usr merge is coming up again, and since I lost >> track of the message where it was brought up, I would open a >> new thread to discuss it. Why is this coming up? What problem(s) are we trying to solve with the usr merge. I'm still not clear on this. @William can you please answer this? >> >> When it came up before, some were saying that the /usr merge violates >> the fhs. I don't remember the specifics of what the claim was at the >> time, (I'm sure someone will point it out if it is still a concern). >> >> I don't think creating usr merged stages would be that difficult. I >> think it would just be a matter of creating a new version of baselayout >> that puts these symlinks in place: If you give me a prototype baselayout (or I can write one myself but I'd rather see what you have in mind) I could test some catalyst runs and see. I could put these on the /experimental for others to look at. >> >> /bin->usr/bin >> /lib->usr/lib >> /lib32->usr/lib32 >> /lib64->usr/lib64 >> /sbin->usr/bin >> /usr/sbin->bin >> >> Once that is in place in a new baselayout, I think portage's colission >> detection would be able to catch files that had the same names and were >> originally in different paths when building the new stages. >> >> I put some thought also in how to nigrate live systems, and I'm not sure >> what the best way to do that is. I wrote a script, which would do it in >> theory, but I haven't tested because I only have one system and if >> it breaks it the only way back would be to reinstall. >> >> The script is attached. >> >> >> Thoughts on any of this? >> >> William > > I looked at Thunderbird, at my folder labeled "gentoo-dev", and it had "666" > unread messages. I should've done the smart thing and closed my mail program. > But noooo, I had to look inside the folder. I am now regretting this decision. Did you round off to the nearest evil? > > *sigh*, I can see the thread has gone clongie 'round the blonger, so all I'll > have to say is we should still try to maintain the choice for users. But, in > order to evaluate what amount of effort is needed to maintain that choice, we > need to know what packages break on such a setup, what the level of effort > needed to fix them is, and do those fixes impact the non-split crowd. I tested and it will affect systems using RBAC. So if we force people to migrate, then users of hardened-sources using RBAC will have to update their policy file. To be clear, I'm not against moving forwards with this, but we can't these people hanging because hardened-sources+RBAC is one reason people in the industry consider gentoo at all. I'm willing to help with backwards compat. > > Create like, a table on the Wiki or some kind of metadata property per-package > that can contain a boolean or tri-state flag indicating whether it works or > doesn't work (or kinda works) on split-usr. Or a tracker on Bugzie. Something. +1 > > Once we know this, then we can work out what's the minimum system that can be > successfully run on split-usr. Then, knowing *that*, we can see if that system > can be supported by our @system target or some minimal subset of @world. As > new package versions come out of upstream, we update this metadata with changes > to the split-usr status, and this then provides a history of the more or less > amount of difficulty needed to maintain support for split-usr, and *then*, we > can make an objective decision to continue supporting or not supporting the > capability. > > As for me, I am flat out ruling out a full-reinstall of all my systems. I have > fixed disk partition layouts on all of them that cannot be re-arranged unless I > tar up each filesystem and temporarily move it off, then rebuild the MD-RAID > and reformat the filesystems. I am simply not going to do that on my many SGI > systems, and whatever facet of upstream, whether it's some core GNU package or > RH itself, can go pound sand for all I care. I'll go back to a static /dev and > I'll manually mknod any missing devices if I have to. Not just you, but many sys admins out there in the real world are in similar situation. Again we can move forward on this but not without backwards compat planning. > > You know it's getting ridiculous when you can maintain a Windows/NTFS partition > layout easier than a Linux one. > Has it come to that? -- Anthony G. Basile, Ph.D. Gentoo Linux Developer [Hardened] E-Mail : blueness@gentoo.org GnuPG FP : 1FED FAD9 D82C 52A5 3BAB DC79 9384 FA6E F52D 4BBA GnuPG ID : F52D4BBA ^ permalink raw reply [flat|nested] 89+ messages in thread
* Re: [gentoo-dev] usr merge @ 2016-04-08 2:36 Damien Levac 2016-04-08 2:44 ` M. J. Everitt 0 siblings, 1 reply; 89+ messages in thread From: Damien Levac @ 2016-04-08 2:36 UTC (permalink / raw To: gentoo-dev Anybody who have this kind of misconception about 'usr merge' should read this: https://www.freedesktop.org/wiki/Software/systemd/TheCaseForTheUsrMerge/ Signed, a user who got scared by this thread and documented myself before freaking out too much... >> Personally I think that merging things into /usr is a major policy >> decision that is likely to contravene upstream installation >> locations. I wouldn't do it lightly, if at all. -- Damien Levac ^ permalink raw reply [flat|nested] 89+ messages in thread
* Re: [gentoo-dev] usr merge 2016-04-08 2:36 Damien Levac @ 2016-04-08 2:44 ` M. J. Everitt 2016-04-08 10:36 ` Rich Freeman 2016-04-08 14:20 ` [gentoo-dev] " William Hubbs 0 siblings, 2 replies; 89+ messages in thread From: M. J. Everitt @ 2016-04-08 2:44 UTC (permalink / raw To: gentoo-dev On 08/04/16 03:36, Damien Levac wrote: > Anybody who have this kind of misconception about 'usr merge' should > read this: > > https://www.freedesktop.org/wiki/Software/systemd/TheCaseForTheUsrMerge/ > > Signed, > > a user who got scared by this thread and documented myself before > freaking out too much... > >>> Personally I think that merging things into /usr is a major policy >> > decision that is likely to contravene upstream installation >>> locations. I wouldn't do it lightly, if at all. > Three points :- 1) systemd - not all gentoo users subscribe to this 'philosophy' .. but no, I don't want get drawn into debates of yes/no of systemd .. 2) "Today, a separate /usr partition already must be mounted by the initramfs during early boot, thus making the justification for a split-off moot." - no, not all gentoo users have an initramfs and need/want one .. so this is a false assumption. 3) I still believe there is merit in distinguishing between binaries that can/should be run as root, and those that can/should not. Those that run as root 100% of the time, or use VMs, don't really 'use' linux in the original sense of the OS .. *hides in his bike-shed, awaiting the flaming torches....* ^ permalink raw reply [flat|nested] 89+ messages in thread
* Re: [gentoo-dev] usr merge 2016-04-08 2:44 ` M. J. Everitt @ 2016-04-08 10:36 ` Rich Freeman 2016-04-09 5:20 ` [gentoo-dev] " Duncan 2016-04-08 14:20 ` [gentoo-dev] " William Hubbs 1 sibling, 1 reply; 89+ messages in thread From: Rich Freeman @ 2016-04-08 10:36 UTC (permalink / raw To: gentoo-dev On Thu, Apr 7, 2016 at 10:44 PM, M. J. Everitt <m.j.everitt@iee.org> wrote: > 2) "Today, a separate /usr partition already must be mounted by the > initramfs during early boot, thus making the justification for a > split-off moot." - no, not all gentoo users have an initramfs and > need/want one .. so this is a false assumption. You only need an initramfs (or some other mechanism to mount /usr during early boot) if /usr is on a different filesystem than /. If /usr is a separate filesystem, then Gentoo does require that it be mounted during early boot, at least as a supported configuration. While it is true today that with some configurations you can probably get away with not mounting it during early boot, there is no requirement that package maintainers support this. That includes system packages. So, #2 applies to Gentoo as much as to any other distro. That was a topic of some debate a few years ago now. > 3) I still believe there is merit in distinguishing between binaries > that can/should be run as root, and those that can/should not. Those > that run as root 100% of the time, or use VMs, don't really 'use' linux > in the original sense of the OS .. Duncan already explained much of this, but if you're relying on a user's PATH setting to prevent security issues you're doing it wrong. There are a number of binaries in /sbin which are completely appropriate for a non-privileged user to execute. Besides non-privileged operations of binaries like btrfs or rpcinfo, there are a bunch of misc binaries in there like usleep or zdump. Really though the main point of merging these paths into /usr is to get all the static content of a distro into a single path, which can then be maintained as a read-only filesystem, mounted across multiple systems, protected using tripwire or signature checking, and so on. As has been pointed out the rolling release nature of Gentoo reduces some of these benefits somewhat. To truly get these benefits we would also need to rethink how post-install configuration gets managed as was already pointed out. However, the principle is still a potentially useful one even if we never follow-up with some of the things Fedora/etc are doing. After a merge the package manager has free rein over /usr, full config management is the policy in /etc, and /var is a place for persistent state that generally belongs to the applications themselves (but management of this is a bit of a mix still with stuff like /var/www and /var/bind alongside mail spools and mysql database files). -- Rich ^ permalink raw reply [flat|nested] 89+ messages in thread
* [gentoo-dev] Re: usr merge 2016-04-08 10:36 ` Rich Freeman @ 2016-04-09 5:20 ` Duncan 0 siblings, 0 replies; 89+ messages in thread From: Duncan @ 2016-04-09 5:20 UTC (permalink / raw To: gentoo-dev Rich Freeman posted on Fri, 08 Apr 2016 06:36:48 -0400 as excerpted: > Really though the main point of merging these paths into /usr is to get > all the static content of a distro into a single path, which can then be > maintained as a read-only filesystem, mounted across multiple systems, > protected using tripwire or signature checking, and so on. As has been > pointed out the rolling release nature of Gentoo reduces some of these > benefits somewhat. To truly get these benefits we would also need to > rethink how post-install configuration gets managed as was already > pointed out. Somewhat unrelated to the /usr or bin/sbin merge here, as (nearly) everything the package manager installs to any of its paths (including /usr, FWIW, but that's easy because my is a /usr -> . symlink) is on /, here, but FWIW, I actually do keep my / read-only mounted by default. So / is only mounted writable to update and/or change configuration. That includes /etc/ and of course my /usr -> . symlink, as well as parts of /var. The parts of /var that system services need to write into during normal operation (well, the ones that need to be permanent, those like /var/run that should be temporary are already on tmpfs mounts) are symlinked into subdirs under /home/var, with /home of course being mounted writable by default, so they can be written into during normal operation despite / being mounted read-only. Works out pretty well, actually, improving reliability of /, since it's normally mounted ro and thus is fully stable in the event of a system crash. Not having to worry about being unable to get to my system recovery tools on / in the event of a bad crash because / was mounted read-only and thus wasn't susceptible to the damage that writable-mounted filesystems can sustain in the event of a hard shutdown is nice. =:^) -- Duncan - List replies preferred. No HTML msgs. "Every nonfree program has a lord, a master -- and if you use the program, he is your master." Richard Stallman ^ permalink raw reply [flat|nested] 89+ messages in thread
* Re: [gentoo-dev] usr merge 2016-04-08 2:44 ` M. J. Everitt 2016-04-08 10:36 ` Rich Freeman @ 2016-04-08 14:20 ` William Hubbs 2016-04-08 20:07 ` waltdnes 1 sibling, 1 reply; 89+ messages in thread From: William Hubbs @ 2016-04-08 14:20 UTC (permalink / raw To: gentoo-dev [-- Attachment #1: Type: text/plain, Size: 749 bytes --] On Fri, Apr 08, 2016 at 03:44:06AM +0100, M. J. Everitt wrote: > 3) I still believe there is merit in distinguishing between binaries > that can/should be run as root, and those that can/should not. Those > that run as root 100% of the time, or use VMs, don't really 'use' linux > in the original sense of the OS .. Here is more info about the split and why it exists. It turns out it hs nothing to do with system admininistration or permissions. http://lists.busybox.net/pipermail/busybox/2010-December/074114.html http://www.osnews.com/story/25556/Understanding_the_bin_sbin_usr_bin_usr_sbin_Split/ https://news.ycombinator.com/item?id=3519952 In short, this is all a historical artifact with justifications thought up after the fact. William [-- Attachment #2: Digital signature --] [-- Type: application/pgp-signature, Size: 181 bytes --] ^ permalink raw reply [flat|nested] 89+ messages in thread
* Re: [gentoo-dev] usr merge 2016-04-08 14:20 ` [gentoo-dev] " William Hubbs @ 2016-04-08 20:07 ` waltdnes 2016-04-08 20:18 ` Joseph Booker 0 siblings, 1 reply; 89+ messages in thread From: waltdnes @ 2016-04-08 20:07 UTC (permalink / raw To: gentoo-dev On Fri, Apr 08, 2016 at 09:20:19AM -0500, William Hubbs wrote > > Here is more info about the split and why it exists. It turns out it hs > nothing to do with system admininistration or permissions. > > http://lists.busybox.net/pipermail/busybox/2010-December/074114.html > http://www.osnews.com/story/25556/Understanding_the_bin_sbin_usr_bin_usr_sbin_Split/ > https://news.ycombinator.com/item?id=3519952 > > In short, this is all a historical artifact with justifications thought > up after the fact. The historical reasons may or may not exist any longer. The question is "what is the current situation?". The current situation is that there are 3 classes of software... 1) system software that is required for bootup (mount, init, etcetera) 2) system software that is usually used by root for admin purposes 3) regular applications that users use Question... do we really want "GIMP", "Firefox", etcetera, in the same directory as "mount", "chroot", "login", "passwd", "ifconfig", etcetera? I don't think so. I want separate "system progs" versus "user progs" directories. There may be an argument for merging /bin and /sbin directories (items 1 and 2 above), but user applications should be separate. If we move /bin and /sbin into /usr/bin, I suggest moving all user programs to /usr/local/binuser applications should be separate. If we move /bin and /sbin into /usr/bin, I suggest moving all user programs to /usr/local/bin. -- Walter Dnes <waltdnes@waltdnes.org> I don't run "desktop environments"; I run useful applications ^ permalink raw reply [flat|nested] 89+ messages in thread
* Re: [gentoo-dev] usr merge 2016-04-08 20:07 ` waltdnes @ 2016-04-08 20:18 ` Joseph Booker 2016-04-08 20:30 ` Rich Freeman 0 siblings, 1 reply; 89+ messages in thread From: Joseph Booker @ 2016-04-08 20:18 UTC (permalink / raw To: gentoo-dev [-- Attachment #1: Type: text/plain, Size: 1985 bytes --] On Fri, Apr 8, 2016 at 4:07 PM, <waltdnes@waltdnes.org> wrote: > On Fri, Apr 08, 2016 at 09:20:19AM -0500, William Hubbs wrote > > > > Here is more info about the split and why it exists. It turns out it hs > > nothing to do with system admininistration or permissions. > > > > http://lists.busybox.net/pipermail/busybox/2010-December/074114.html > > > http://www.osnews.com/story/25556/Understanding_the_bin_sbin_usr_bin_usr_sbin_Split/ > > https://news.ycombinator.com/item?id=3519952 > > > > In short, this is all a historical artifact with justifications thought > > up after the fact. > > The historical reasons may or may not exist any longer. The question > is "what is the current situation?". The current situation is that > there are 3 classes of software... > 1) system software that is required for bootup (mount, init, etcetera) > 2) system software that is usually used by root for admin purposes > 3) regular applications that users use > > Question... do we really want "GIMP", "Firefox", etcetera, in the same > directory as "mount", "chroot", "login", "passwd", "ifconfig", etcetera? > From my own experience, it is useful to run "ifconfig" or "mount" as a regular user, same as the gimp or firefox commands. Given that all the commands you listed are in /usr/bin or /bin, I think I'm not the only one. The difference between "system software" and "regular applications" isn't clear-cut. > I don't think so. I want separate "system progs" versus "user progs" > directories. There may be an argument for merging /bin and /sbin > directories (items 1 and 2 above), but user applications should be > separate. If we move /bin and /sbin into /usr/bin, I suggest moving all > user programs to /usr/local/binuser applications should be separate. If > we move /bin and /sbin into /usr/bin, I suggest moving all user programs > to /usr/local/bin. > > -- > Walter Dnes <waltdnes@waltdnes.org> > I don't run "desktop environments"; I run useful applications > > [-- Attachment #2: Type: text/html, Size: 3197 bytes --] ^ permalink raw reply [flat|nested] 89+ messages in thread
* Re: [gentoo-dev] usr merge 2016-04-08 20:18 ` Joseph Booker @ 2016-04-08 20:30 ` Rich Freeman 2016-04-09 1:18 ` waltdnes 0 siblings, 1 reply; 89+ messages in thread From: Rich Freeman @ 2016-04-08 20:30 UTC (permalink / raw To: gentoo-dev On Fri, Apr 8, 2016 at 4:18 PM, Joseph Booker <joe@neoturbine.net> wrote: > The difference between "system software" and "regular applications" isn't > clear-cut. > This. Half the reason we don't officially support running without /usr mounted during early boot is that if we actually put everything in / that could conceivably be needed during early boot we'd end up with everything there. Bluetooth keyboards is a common example. The console should work during early boot, right? -- Rich ^ permalink raw reply [flat|nested] 89+ messages in thread
* Re: [gentoo-dev] usr merge 2016-04-08 20:30 ` Rich Freeman @ 2016-04-09 1:18 ` waltdnes 2016-04-10 17:51 ` [gentoo-dev] " »Q« 0 siblings, 1 reply; 89+ messages in thread From: waltdnes @ 2016-04-09 1:18 UTC (permalink / raw To: gentoo-dev On Fri, Apr 08, 2016 at 04:30:04PM -0400, Rich Freeman wrote > Half the reason we don't officially support running without /usr > mounted during early boot is that if we actually put everything in / > that could conceivably be needed during early boot we'd end up with > everything there. Bluetooth keyboards is a common example. The > console should work during early boot, right? Seriously... how many people run Bluetooth keyboards on Gentoo anyways? -- Walter Dnes <waltdnes@waltdnes.org> I don't run "desktop environments"; I run useful applications ^ permalink raw reply [flat|nested] 89+ messages in thread
* [gentoo-dev] Re: usr merge 2016-04-09 1:18 ` waltdnes @ 2016-04-10 17:51 ` »Q« 0 siblings, 0 replies; 89+ messages in thread From: »Q« @ 2016-04-10 17:51 UTC (permalink / raw To: gentoo-dev On Fri, 8 Apr 2016 21:18:37 -0400 waltdnes@waltdnes.org wrote: > On Fri, Apr 08, 2016 at 04:30:04PM -0400, Rich Freeman wrote > > > Half the reason we don't officially support running without /usr > > mounted during early boot is that if we actually put everything in / > > that could conceivably be needed during early boot we'd end up with > > everything there. Bluetooth keyboards is a common example. The > > console should work during early boot, right? > > Seriously... how many people run Bluetooth keyboards on Gentoo > anyways? The sarcasm and the rhetorical questions make it tough to tell -- are you trying here to make an argument that Gentoo should commit to supporting boot without /usr mounted early? ^ permalink raw reply [flat|nested] 89+ messages in thread
* Re: [gentoo-dev] usr merge
@ 2016-04-09 3:59 Damien Levac
2016-04-09 5:32 ` waltdnes
0 siblings, 1 reply; 89+ messages in thread
From: Damien Levac @ 2016-04-09 3:59 UTC (permalink / raw
To: gentoo-dev@lists.gentoo.org >> gentoo-dev
> Seriously... how many people run Bluetooth keyboards on Gentoo >anyways?
That you ask such a question is concerning to me. Are we discriminating
against normal desktop users now?
--
Damien Levac
^ permalink raw reply [flat|nested] 89+ messages in thread
* Re: [gentoo-dev] usr merge 2016-04-09 3:59 [gentoo-dev] " Damien Levac @ 2016-04-09 5:32 ` waltdnes 2016-04-09 11:11 ` Rich Freeman 0 siblings, 1 reply; 89+ messages in thread From: waltdnes @ 2016-04-09 5:32 UTC (permalink / raw To: gentoo-dev On Fri, Apr 08, 2016 at 11:59:09PM -0400, Damien Levac wrote > > > Seriously... how many people run Bluetooth keyboards on Gentoo > > anyways? > > That you ask such a question is concerning to me. Are we > discriminating against normal desktop users now? Here's the item that really bugs me... before - many people successfully used separate /usr, without initramfs. A few edge cases, e.g. people with bluetooth keyboards, had to use initramfs if they wanted a separate /usr. The poor darlings felt left out because they had to do extra setup work, versus the other 95%. now - an arbitrary decree comes down that *EVERYBODY* who wants a separate /usr needs to have initramfs. * IT DOES NOT MAKE THINGS ANY EASIER FOR THE ORIGINAL 5% EDGE CASES *. But the other 95% who could run separate /usr are now being told they must run initramfs "just because". What does it accomplish? BTW, I'm still running a separate /usr without initramfs, and no related problems; thank you. If I decided to go to an edge-case setup (e.g. Bluetooth keyboard, or ell partitions encrypted) then I could understand being asked to run initramfs. This is reminiscent of the "Mozilla Mentality", where everybody is forced to the lowest common denominator. Yes, a desktop GUI sucks on a tablet/smartphone; I get it. So Firefox was saddled with the smartphone-oriented Atrocious^H^H^H^H^H^H Australis GUI, which sucks on a desktop. That was the last straw that drove me to Pale Moon. -- Walter Dnes <waltdnes@waltdnes.org> I don't run "desktop environments"; I run useful applications ^ permalink raw reply [flat|nested] 89+ messages in thread
* Re: [gentoo-dev] usr merge 2016-04-09 5:32 ` waltdnes @ 2016-04-09 11:11 ` Rich Freeman 2016-04-09 16:09 ` waltdnes 0 siblings, 1 reply; 89+ messages in thread From: Rich Freeman @ 2016-04-09 11:11 UTC (permalink / raw To: gentoo-dev On Sat, Apr 9, 2016 at 1:32 AM, <waltdnes@waltdnes.org> wrote: > > now - an arbitrary decree comes down that *EVERYBODY* who wants a > separate /usr needs to have initramfs. > The "decree" wasn't some kind of law that the Gentoo police will come out to your house and arrest you for violating. It was simply a recognition that we were already in a state where booting a system without /usr mounted early can cause problems. There isn't really any solution to these problems (other than moving most of /usr into /, which I doubt is the desire of anybody who puts /usr on a separate filesystem), and it probably will only get worse. The intent of the resolution was to not burden package maintainers to have to cater to a use case that was already failing. And the wording of the resolution doesn't mention the word "initramfs" at all, precisely because we recognized that there were many ways to work around the problem. If you have concerns about the decision being arbitrary you might want to read the original summary: https://projects.gentoo.org/council/meeting-logs/20130813-summary.txt and log: https://projects.gentoo.org/council/meeting-logs/20130813.txt And of course you can read the list archives from the time where the issue was extensively discussed. > * IT DOES NOT MAKE THINGS ANY EASIER FOR THE ORIGINAL 5% EDGE CASES *. > But the other 95% who could run separate /usr are now being told they > must run initramfs "just because". What does it accomplish? I never really got the mentality that using an initramfs is a burden. You can boot a kernel as an EFI program, but the reality is that many if not most users of linux on EFI use a secondary bootloader. Heck, back in the old days you could actually boot linux directly from the BIOS without any secondary bootloader, but this was so impractical that even Linus now tells people to: bugger_off_msg: .ascii "Use a boot loader.\r\n" .ascii "\n" .ascii "Remove disk and press any key to reboot...\r\n" .byte 0 (and I must say that I admire the man with the guts to not insert a carriage return when the carriage is already on the first column) An initramfs is just a secondary bootloader for userspace. I almost always use them even if I'm just booting a VM with a single partition on it. If something goes wrong you can fall back to a shell in the initramfs and it is like having a rescue disk built into your system disk. For a more complex setup it is much more robust than relying on the kernel to find your root, and it also lets you build with a more module-based kernel, which has some benefits as well even if you build kernels tailored to each host. -- Rich ^ permalink raw reply [flat|nested] 89+ messages in thread
* Re: [gentoo-dev] usr merge 2016-04-09 11:11 ` Rich Freeman @ 2016-04-09 16:09 ` waltdnes 2016-04-09 16:15 ` James Le Cuirot 2016-04-09 17:18 ` »Q« 0 siblings, 2 replies; 89+ messages in thread From: waltdnes @ 2016-04-09 16:09 UTC (permalink / raw To: gentoo-dev On Sat, Apr 09, 2016 at 07:11:31AM -0400, Rich Freeman wrote > It was simply a recognition that we were already in a state where > booting a system without /usr mounted early can cause problems. For certain edge cases... yes. But they were already using initramfs or merging /usr into /. I'm talking about the 95% who don't really need it. > I never really got the mentality that using an initramfs is a burden. One more piece of software that can go wrong. You have to maintain+configure it; e.g. sync software and library versions with what's on the rest of the system. > An initramfs is just a secondary bootloader for userspace. I almost > always use them even if I'm just booting a VM with a single partition > on it. If something goes wrong you can fall back to a shell in the > initramfs and it is like having a rescue disk built into your system > disk. There is single-user mode for rescue. > For a more complex setup it is much more robust than relying on > the kernel to find your root, and it also lets you build with a more > module-based kernel, which has some benefits as well even if you build > kernels tailored to each host. I have "Production" and "Experimental" entries in my LILO menu. A new kernel is always set up as the "Experimental" entry. After running several days without problems, I run a script which copies the data from the "Experimental" portion to "Production". The only time my system had problems "finding root" was years ago when the switch from /dev/hd* to /dev/sd* took place. The "Experimental" boot with the new kernel died. I booted "Production", read the mailing list, changed "hd" to "sd" for the "Experimental" entry, and rebooted. After several days without problems, I made the same change to the "Production" entry, and copied the "Experimental" portion to "Production". -- Walter Dnes <waltdnes@waltdnes.org> I don't run "desktop environments"; I run useful applications ^ permalink raw reply [flat|nested] 89+ messages in thread
* Re: [gentoo-dev] usr merge 2016-04-09 16:09 ` waltdnes @ 2016-04-09 16:15 ` James Le Cuirot 2016-04-09 18:42 ` Dale 2016-04-10 0:09 ` [gentoo-dev] " J. Roeleveld 2016-04-09 17:18 ` »Q« 1 sibling, 2 replies; 89+ messages in thread From: James Le Cuirot @ 2016-04-09 16:15 UTC (permalink / raw To: gentoo-dev [-- Attachment #1: Type: text/plain, Size: 685 bytes --] On Sat, 9 Apr 2016 12:09:38 -0400 waltdnes@waltdnes.org wrote: > > I never really got the mentality that using an initramfs is a > > burden. > > One more piece of software that can go wrong. You have to > maintain+configure it; e.g. sync software and library versions with > what's on the rest of the system. Errm, have you ever actually used dracut? dracut --kver 4.5 Wow, that was hard! It requires zero configuration and that's true even if you've got LVM on top of LUKS on top of RAID or something equally complex. If you're already running that kernel version, you don't even need to specify it. -- James Le Cuirot (chewi) Gentoo Linux Developer [-- Attachment #2: OpenPGP digital signature --] [-- Type: application/pgp-signature, Size: 951 bytes --] ^ permalink raw reply [flat|nested] 89+ messages in thread
* Re: [gentoo-dev] usr merge 2016-04-09 16:15 ` James Le Cuirot @ 2016-04-09 18:42 ` Dale 2016-04-10 9:14 ` [gentoo-dev] " Duncan 2016-04-10 0:09 ` [gentoo-dev] " J. Roeleveld 1 sibling, 1 reply; 89+ messages in thread From: Dale @ 2016-04-09 18:42 UTC (permalink / raw To: gentoo-dev James Le Cuirot wrote: > On Sat, 9 Apr 2016 12:09:38 -0400 > waltdnes@waltdnes.org wrote: > >>> I never really got the mentality that using an initramfs is a >>> burden. >> One more piece of software that can go wrong. You have to >> maintain+configure it; e.g. sync software and library versions with >> what's on the rest of the system. > Errm, have you ever actually used dracut? > > dracut --kver 4.5 > > Wow, that was hard! It requires zero configuration and that's true even > if you've got LVM on top of LUKS on top of RAID or something equally > complex. If you're already running that kernel version, you don't even > need to specify it. > FYI. I've had those to fail too. As Walt said, just one more thing to fail. Dale :-) :-) ^ permalink raw reply [flat|nested] 89+ messages in thread
* [gentoo-dev] Re: usr merge 2016-04-09 18:42 ` Dale @ 2016-04-10 9:14 ` Duncan 0 siblings, 0 replies; 89+ messages in thread From: Duncan @ 2016-04-10 9:14 UTC (permalink / raw To: gentoo-dev Dale posted on Sat, 09 Apr 2016 13:42:37 -0500 as excerpted: > James Le Cuirot wrote: >> On Sat, 9 Apr 2016 12:09:38 -0400 waltdnes@waltdnes.org wrote: >> >>>> I never really got the mentality that using an initramfs is a burden. >>> One more piece of software that can go wrong. You have to >>> maintain+configure it; e.g. sync software and library versions with >>> what's on the rest of the system. >> Errm, have you ever actually used dracut? >> >> dracut --kver 4.5 >> >> Wow, that was hard! It requires zero configuration and that's true even >> if you've got LVM on top of LUKS on top of RAID or something equally >> complex. If you're already running that kernel version, you don't even >> need to specify it. >> > FYI. I've had those to fail too. As Walt said, just one more thing to > fail. And more to the point, if all you know about dracut is dracut --kver 4.5, then you're not going to be able to _fix_ those failures. Some years ago I tried lvm2 as well as mdraid. I quickly ejected lvm2 from my system and future plans (keeping mdraid), because it was simply too complex for me as an admin to be confident in my ability to work with it without fat-fingering something, under the extreme pressure of a disaster recovery situation, possibly without proper access to manpages and other documentation due to the disaster recovery I was working thru. Waltdnes has a point, the same point I learned then. As a responsible admin, if the system's too complex to be understood well enough to be confident in one's ability to restore in a disaster recovery situation with limited or no access to manpages and similar documentation, it's too complex. A reasonable system is one you understand well enough to confidently manage it in those sorts of situations. Otherwise it's simply too complex for your skill level as an admin. And understanding an initr* well enough to confidently deal with a disaster recovery situation, possibly/likely without access to documentation (it's a disaster recovery, after all!) is definitely *well* beyond "dracut --kver 4.5" level, so he's very right to be worried about it. -- Duncan - List replies preferred. No HTML msgs. "Every nonfree program has a lord, a master -- and if you use the program, he is your master." Richard Stallman ^ permalink raw reply [flat|nested] 89+ messages in thread
* Re: [gentoo-dev] usr merge 2016-04-09 16:15 ` James Le Cuirot 2016-04-09 18:42 ` Dale @ 2016-04-10 0:09 ` J. Roeleveld 2016-04-10 1:07 ` Rich Freeman 1 sibling, 1 reply; 89+ messages in thread From: J. Roeleveld @ 2016-04-10 0:09 UTC (permalink / raw To: gentoo-dev [-- Attachment #1: Type: text/plain, Size: 1168 bytes --] On Saturday, April 09, 2016 05:15:08 PM James Le Cuirot wrote: > On Sat, 9 Apr 2016 12:09:38 -0400 > > waltdnes@waltdnes.org wrote: > > > I never really got the mentality that using an initramfs is a > > > burden. > > > > > One more piece of software that can go wrong. You have to > > > > maintain+configure it; e.g. sync software and library versions with > > what's on the rest of the system. > > Errm, have you ever actually used dracut? > > dracut --kver 4.5 > > Wow, that was hard! It requires zero configuration and that's true even > if you've got LVM on top of LUKS on top of RAID or something equally > complex. If you're already running that kernel version, you don't even > need to specify it. I actually write my own initramfs because neither dracut not genkernel end up with a convenient boot system. I have 2 disks, both encrypted. I prefer only to enter the decryption password once. Both Dracut and Genkernel insist on asking for the password/key for every single disk. The ONLY reason why I feel an initramfs is warranted is because of the encryption. Without that, it should not be necessary. -- Joost [-- Attachment #2: This is a digitally signed message part. --] [-- Type: application/pgp-signature, Size: 473 bytes --] ^ permalink raw reply [flat|nested] 89+ messages in thread
* Re: [gentoo-dev] usr merge 2016-04-10 0:09 ` [gentoo-dev] " J. Roeleveld @ 2016-04-10 1:07 ` Rich Freeman 2016-04-10 9:37 ` [gentoo-dev] " Duncan 0 siblings, 1 reply; 89+ messages in thread From: Rich Freeman @ 2016-04-10 1:07 UTC (permalink / raw To: gentoo-dev On Sat, Apr 9, 2016 at 8:09 PM, J. Roeleveld <joost@antarean.org> wrote: > > I actually write my own initramfs because neither dracut not genkernel end up > with a convenient boot system. > > I have 2 disks, both encrypted. > I prefer only to enter the decryption password once. Both Dracut and Genkernel > insist on asking for the password/key for every single disk. > You can of course roll your own, but I imagine that it would be more straightforward to just write your own dracut plugin. They're basically just scripts that run at whatever boot stage you define. You might also just be able to modify the existing plugin. -- Rich ^ permalink raw reply [flat|nested] 89+ messages in thread
* [gentoo-dev] Re: usr merge 2016-04-10 1:07 ` Rich Freeman @ 2016-04-10 9:37 ` Duncan 2016-04-10 11:32 ` Rich Freeman 0 siblings, 1 reply; 89+ messages in thread From: Duncan @ 2016-04-10 9:37 UTC (permalink / raw To: gentoo-dev Rich Freeman posted on Sat, 09 Apr 2016 21:07:46 -0400 as excerpted: > On Sat, Apr 9, 2016 at 8:09 PM, J. Roeleveld <joost@antarean.org> wrote: >> >> I actually write my own initramfs because neither dracut not genkernel >> end up with a convenient boot system. >> >> I have 2 disks, both encrypted. >> I prefer only to enter the decryption password once. Both Dracut and >> Genkernel insist on asking for the password/key for every single disk. >> >> > You can of course roll your own, but I imagine that it would be more > straightforward to just write your own dracut plugin. They're basically > just scripts that run at whatever boot stage you define. > You might also just be able to modify the existing plugin. The problem with writing your own plugin, is that then you have to understand /both/ what you want to do, and the system you're writing a plugin for, in sufficient depth to actually do so. Whereas if you write the functionality directly, you only have to, by the end, understand the functionality you're implementing, nothing else. True, you'll probably be implementing something a bit broader than the plugin, and will probably learn a bit more about the topic as you do it, but if that was the intent in the first place, you'll definitely have achieved it when you're done, while if you write the plugin, you'll only have a so-so knowledge of two things, instead of a more in depth knowledge of the one you were actually interested in. This is actually one of the problems I had trying to learn firewalling on Linux, back when I switched from MS. I knew the general concepts, and indeed had played with them on my DSL router, back in the day, well enough to be the reference person from which many others learned. But I just couldn't get the hang of the various firewall helper scripts, shorewall and the like, because they were simply too far from the action, and I would have had to effectively learn both them and Linux IPTables, particularly when the helper script didn't cover what I wanted to do in its simplified model. When I finally gave up on the so-called helper scripts and decided to try IPTables directly, it was *MUCH* simpler, because now I was just learning ONE thing! Similarly, I learned kernel suspend/hibernate and resume by reading the kernel docs and writing my own script. Once I was part way done, I was able to actually understand the complex suspend-tools or whatever script far better, and could actually borrow a few tricks from it, whereas when I started I tried reading it and it was just trying to do far too much fancy stuff, far too far from the real action, for me to properly master the real kernel action that the fancy script covered. Grub2 same story. I ended up learning the advanced scripting stuff and doing my own thing, eventually install.masking the normal user-side config stuff, which wouldn't let me do what I wanted to do, without having to learn BOTH the advanced scripting and the user-side layout, because the userside stuff was too dumbed down to expose what I needed to do what I wanted to do. Tho with the initr*, I did go the dracut route myself. But I'm still not entirely convinced that I wouldn't have been better off rolling my own, as I'm still not entirely comfortable with the level to which I understand, or more accurately don't understand, dracut. Tho I do understand it well enough to have cut it down to the modules I need, only, but I still don't understand the scripts at the level I know I would had I created them myself... -- Duncan - List replies preferred. No HTML msgs. "Every nonfree program has a lord, a master -- and if you use the program, he is your master." Richard Stallman ^ permalink raw reply [flat|nested] 89+ messages in thread
* Re: [gentoo-dev] Re: usr merge 2016-04-10 9:37 ` [gentoo-dev] " Duncan @ 2016-04-10 11:32 ` Rich Freeman 0 siblings, 0 replies; 89+ messages in thread From: Rich Freeman @ 2016-04-10 11:32 UTC (permalink / raw To: gentoo-dev On Sun, Apr 10, 2016 at 5:37 AM, Duncan <1i5t5.duncan@cox.net> wrote: > > Tho with the initr*, I did go the dracut route myself. But I'm still not > entirely convinced that I wouldn't have been better off rolling my own, > as I'm still not entirely comfortable with the level to which I > understand, or more accurately don't understand, dracut. Tho I do > understand it well enough to have cut it down to the modules I need, > only, but I still don't understand the scripts at the level I know I > would had I created them myself... > Well, by this logic you ought to be writing your own kernel as well. How else are you going to deal with a kernel panic? :) Dracut is pretty straightforward though and fairly well-documented. It runs through a series of stages like detecting devices, mounting filesystems, and so on. Plugins can run at any stage, and they also can add files to the initramfs at time of creation. The advantage of using something standard is that somebody else has probably already thought of all the edge cases already, and they'll fix the bugs as well. If you contribute your plugin upstream they'll take care of it for you as well. So, you can just put one line in a config file to add /usr/bin/btrfs to your initramfs and the built-in code will figure out that you'll need /usr/lib64/liblzo2.so.2 installed there as well. Of course, that is just an example as somebody else has already written the btrfs module. I was running into some kind of strange md-raid behavior ages ago and worked around it by writing a quick module: https://rich0gentoo.wordpress.com/2012/01/21/a-quick-dracut-module/ That one is a bit of a hack - I suspect there was some underlying bug in udev or something preventing the raid from being set up by the normal module. I suspect that module would no longer be needed (I've since moved on to btrfs). However, it demonstrated how dracut plugins work - you just define hooks that get run at the appropriate phase. -- Rich ^ permalink raw reply [flat|nested] 89+ messages in thread
* [gentoo-dev] Re: usr merge 2016-04-09 16:09 ` waltdnes 2016-04-09 16:15 ` James Le Cuirot @ 2016-04-09 17:18 ` »Q« 2016-04-09 18:34 ` waltdnes 2016-04-11 14:21 ` Ian Stakenvicius 1 sibling, 2 replies; 89+ messages in thread From: »Q« @ 2016-04-09 17:18 UTC (permalink / raw To: gentoo-dev On Sat, 9 Apr 2016 12:09:38 -0400 waltdnes@waltdnes.org wrote: > On Sat, Apr 09, 2016 at 07:11:31AM -0400, Rich Freeman wrote > > > It was simply a recognition that we were already in a state where > > booting a system without /usr mounted early can cause problems. > > For certain edge cases... yes. But they were already using > initramfs or merging /usr into /. I'm talking about the 95% who > don't really need it. Booting without /usr mounted early is something Gentoo already doesn't support and can't support, right? ^ permalink raw reply [flat|nested] 89+ messages in thread
* Re: [gentoo-dev] Re: usr merge 2016-04-09 17:18 ` »Q« @ 2016-04-09 18:34 ` waltdnes 2016-04-09 19:05 ` Canek Peláez Valdés 2016-04-09 19:22 ` Alan McKinnon 2016-04-11 14:21 ` Ian Stakenvicius 1 sibling, 2 replies; 89+ messages in thread From: waltdnes @ 2016-04-09 18:34 UTC (permalink / raw To: gentoo-dev On Sat, Apr 09, 2016 at 12:18:25PM -0500, »Q« wrote > On Sat, 9 Apr 2016 12:09:38 -0400 > waltdnes@waltdnes.org wrote: > > > On Sat, Apr 09, 2016 at 07:11:31AM -0400, Rich Freeman wrote > > > > > It was simply a recognition that we were already in a state where > > > booting a system without /usr mounted early can cause problems. > > > > For certain edge cases... yes. But they were already using > > initramfs or merging /usr into /. I'm talking about the 95% who > > don't really need it. > > Booting without /usr mounted early is something Gentoo already doesn't > support and can't support, right? If you can read this post, you've got a mighty powerful imagination. Because we all know that Gentoo can't boot, let alone send emails, from a machine with separate /usr and no initramfs... just like I'm using right now. -- Walter Dnes <waltdnes@waltdnes.org> I don't run "desktop environments"; I run useful applications ^ permalink raw reply [flat|nested] 89+ messages in thread
* Re: [gentoo-dev] Re: usr merge 2016-04-09 18:34 ` waltdnes @ 2016-04-09 19:05 ` Canek Peláez Valdés 2016-04-09 19:22 ` Alan McKinnon 1 sibling, 0 replies; 89+ messages in thread From: Canek Peláez Valdés @ 2016-04-09 19:05 UTC (permalink / raw To: gentoo-dev On Sat, Apr 9, 2016 at 1:34 PM, <waltdnes@waltdnes.org> wrote: > On Sat, Apr 09, 2016 at 12:18:25PM -0500, »Q« wrote >> On Sat, 9 Apr 2016 12:09:38 -0400 >> waltdnes@waltdnes.org wrote: >> >> > On Sat, Apr 09, 2016 at 07:11:31AM -0400, Rich Freeman wrote >> > >> > > It was simply a recognition that we were already in a state where >> > > booting a system without /usr mounted early can cause problems. >> > >> > For certain edge cases... yes. But they were already using >> > initramfs or merging /usr into /. I'm talking about the 95% who >> > don't really need it. >> >> Booting without /usr mounted early is something Gentoo already doesn't >> support and can't support, right? > > If you can read this post, you've got a mighty powerful imagination. > Because we all know that Gentoo can't boot, let alone send emails, from > a machine with separate /usr and no initramfs... just like I'm using > right now. Nobody said it was not possible; Q said that it was not supported, and it cannot be because, in the general case (not *YOU* specific case), someone somewhere may require something from /usr to boot. Regards. -- Dr. Canek Peláez Valdés Profesor de Carrera Asociado C Departamento de Matemáticas Facultad de Ciencias Universidad Nacional Autónoma de México ^ permalink raw reply [flat|nested] 89+ messages in thread
* Re: [gentoo-dev] Re: usr merge 2016-04-09 18:34 ` waltdnes 2016-04-09 19:05 ` Canek Peláez Valdés @ 2016-04-09 19:22 ` Alan McKinnon 1 sibling, 0 replies; 89+ messages in thread From: Alan McKinnon @ 2016-04-09 19:22 UTC (permalink / raw To: gentoo-dev On 09/04/2016 20:34, waltdnes@waltdnes.org wrote: > On Sat, Apr 09, 2016 at 12:18:25PM -0500, »Q« wrote >> On Sat, 9 Apr 2016 12:09:38 -0400 >> waltdnes@waltdnes.org wrote: >> >>> On Sat, Apr 09, 2016 at 07:11:31AM -0400, Rich Freeman wrote >>> >>>> It was simply a recognition that we were already in a state where >>>> booting a system without /usr mounted early can cause problems. >>> >>> For certain edge cases... yes. But they were already using >>> initramfs or merging /usr into /. I'm talking about the 95% who >>> don't really need it. >> >> Booting without /usr mounted early is something Gentoo already doesn't >> support and can't support, right? > > If you can read this post, you've got a mighty powerful imagination. > Because we all know that Gentoo can't boot, let alone send emails, from > a machine with separate /usr and no initramfs... just like I'm using > right now. > That's not what he said. He said gentoo doesn't, and can't support it, because it's a world of pain to provide proper support to everyone who wants it. If you want it, as you do, you get to do it yourself. While it still works, grat. When it stops working, you fix it. He did not say, as you imply, that it cannot work right now. -- Alan McKinnon alan.mckinnon@gmail.com ^ permalink raw reply [flat|nested] 89+ messages in thread
* Re: [gentoo-dev] Re: usr merge 2016-04-09 17:18 ` »Q« 2016-04-09 18:34 ` waltdnes @ 2016-04-11 14:21 ` Ian Stakenvicius 1 sibling, 0 replies; 89+ messages in thread From: Ian Stakenvicius @ 2016-04-11 14:21 UTC (permalink / raw To: gentoo-dev -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 On 09/04/16 01:18 PM, »Q« wrote: > On Sat, 9 Apr 2016 12:09:38 -0400 waltdnes@waltdnes.org wrote: > >> On Sat, Apr 09, 2016 at 07:11:31AM -0400, Rich Freeman wrote >> >>> It was simply a recognition that we were already in a state >>> where booting a system without /usr mounted early can cause >>> problems. >> >> For certain edge cases... yes. But they were already using >> initramfs or merging /usr into /. I'm talking about the 95% >> who don't really need it. > > Booting without /usr mounted early is something Gentoo already > doesn't support and can't support, right? > > It still works for a number of basic cases, but per the Council's decision quite a while ago, dev's have zero obligation to ensure it continues to work -- in effect this means it can break at any time. -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iF4EAREIAAYFAlcLstQACgkQAJxUfCtlWe2I+AEAyVX2Zh8YDbbCfTtCJ8C3Y1Gk 8OX3on1uKDCFeybThAgA/3g/uI2WyPhdayARsNNGQuX8tD+ejv/mpZjY2UUJPSpv =3e1w -----END PGP SIGNATURE----- ^ permalink raw reply [flat|nested] 89+ messages in thread
end of thread, other threads:[~2016-04-11 14:21 UTC | newest] Thread overview: 89+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2016-04-05 1:19 [gentoo-dev] usr merge William Hubbs 2016-04-05 10:10 ` Alexis Ballier 2016-04-05 12:26 ` [gentoo-dev] " Duncan 2016-04-05 16:53 ` [gentoo-dev] " Alexis Ballier 2016-04-06 0:06 ` [gentoo-dev] " Jonathan Callen 2016-04-06 4:15 ` [gentoo-dev] " Richard Yao 2016-04-06 5:34 ` [gentoo-dev] " Duncan 2016-04-06 14:32 ` Richard Yao 2016-04-06 7:42 ` [gentoo-dev] " Alexis Ballier 2016-04-06 8:55 ` James Le Cuirot 2016-04-06 14:06 ` Richard Yao 2016-04-06 14:04 ` Richard Yao 2016-04-06 14:48 ` Alexis Ballier 2016-04-06 22:01 ` [gentoo-dev] " Duncan 2016-04-07 11:27 ` [gentoo-dev] " Tom H 2016-04-06 17:06 ` waltdnes 2016-04-06 14:58 ` M. J. Everitt 2016-04-06 15:11 ` Alexis Ballier 2016-04-06 16:06 ` Richard Yao 2016-04-06 16:12 ` M. J. Everitt 2016-04-06 16:20 ` Alexis Ballier 2016-04-06 16:33 ` Richard Yao 2016-04-06 16:57 ` Alexis Ballier 2016-04-06 17:06 ` Alexis Ballier 2016-04-06 17:43 ` Richard Yao 2016-04-06 16:24 ` Richard Yao 2016-04-06 15:52 ` Richard Yao 2016-04-06 20:43 ` William Hubbs 2016-04-06 21:36 ` Richard Yao 2016-04-07 0:44 ` William Hubbs 2016-04-07 9:12 ` Alexis Ballier 2016-04-07 14:40 ` William Hubbs 2016-04-07 15:12 ` [gentoo-dev] " Duncan 2016-04-07 15:42 ` Rich Freeman 2016-04-07 15:46 ` William Hubbs 2016-04-07 16:22 ` Rich Freeman 2016-04-07 16:36 ` [gentoo-dev] " Alexis Ballier 2016-04-07 18:21 ` Raymond Jennings 2016-04-07 18:32 ` M. J. Everitt 2016-04-07 18:54 ` Rich Freeman 2016-04-07 20:18 ` Raymond Jennings 2016-04-08 1:39 ` William Hubbs 2016-04-08 1:42 ` William Hubbs 2016-04-08 2:35 ` M. J. Everitt 2016-04-08 7:58 ` [gentoo-dev] " Duncan 2016-04-09 12:44 ` Nicolas Sebrecht 2016-04-10 8:17 ` Duncan 2016-04-08 10:14 ` [gentoo-dev] " Rich Freeman 2016-04-08 11:31 ` Anthony G. Basile 2016-04-08 11:41 ` James Le Cuirot 2016-04-08 11:52 ` Rich Freeman 2016-04-08 11:54 ` Anthony G. Basile 2016-04-08 12:55 ` Rich Freeman 2016-04-09 12:52 ` Luca Barbato 2016-04-08 22:20 ` Daniel Campbell 2016-04-09 0:42 ` William Hubbs 2016-04-09 1:11 ` Anthony G. Basile 2016-04-09 1:36 ` William Hubbs 2016-04-09 1:51 ` Anthony G. Basile 2016-04-09 3:03 ` Rich Freeman 2016-04-09 4:06 ` Anthony G. Basile 2016-04-09 10:56 ` Rich Freeman 2016-04-09 11:16 ` Anthony G. Basile 2016-04-09 12:39 ` Anthony G. Basile 2016-04-09 15:11 ` William Hubbs 2016-04-09 17:25 ` Alexis Ballier 2016-04-09 14:11 ` Ian Stakenvicius 2016-04-07 9:03 ` Alexis Ballier 2016-04-09 11:41 ` Luca Barbato 2016-04-09 11:53 ` Rich Freeman 2016-04-09 12:27 ` Luca Barbato 2016-04-09 12:37 ` Rich Freeman 2016-04-09 13:03 ` Luca Barbato 2016-04-10 7:38 ` [gentoo-dev] " Duncan 2016-04-10 11:55 ` [gentoo-dev] " Joshua Kinard 2016-04-10 12:14 ` Rich Freeman 2016-04-10 12:34 ` Anthony G. Basile 2016-04-11 1:59 ` Joshua Kinard 2016-04-10 12:26 ` Anthony G. Basile -- strict thread matches above, loose matches on Subject: below -- 2016-04-08 2:36 Damien Levac 2016-04-08 2:44 ` M. J. Everitt 2016-04-08 10:36 ` Rich Freeman 2016-04-09 5:20 ` [gentoo-dev] " Duncan 2016-04-08 14:20 ` [gentoo-dev] " William Hubbs 2016-04-08 20:07 ` waltdnes 2016-04-08 20:18 ` Joseph Booker 2016-04-08 20:30 ` Rich Freeman 2016-04-09 1:18 ` waltdnes 2016-04-10 17:51 ` [gentoo-dev] " »Q« 2016-04-09 3:59 [gentoo-dev] " Damien Levac 2016-04-09 5:32 ` waltdnes 2016-04-09 11:11 ` Rich Freeman 2016-04-09 16:09 ` waltdnes 2016-04-09 16:15 ` James Le Cuirot 2016-04-09 18:42 ` Dale 2016-04-10 9:14 ` [gentoo-dev] " Duncan 2016-04-10 0:09 ` [gentoo-dev] " J. Roeleveld 2016-04-10 1:07 ` Rich Freeman 2016-04-10 9:37 ` [gentoo-dev] " Duncan 2016-04-10 11:32 ` Rich Freeman 2016-04-09 17:18 ` »Q« 2016-04-09 18:34 ` waltdnes 2016-04-09 19:05 ` Canek Peláez Valdés 2016-04-09 19:22 ` Alan McKinnon 2016-04-11 14:21 ` Ian Stakenvicius
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox