[gentoo-user] Optional /usr merge in Gentoo

[gentoo-user] Optional /usr merge in Gentoo

[Alessio Ababilov]

[-- Attachment #1: Type: text/plain, Size: 312 bytes --]

Hi!

I wrote a script that allows /usr merge in Gentoo without changes to
ebuilds.

I described it in an article
http://aababilov.wordpress.com/2013/07/17/usr-merge-in-gentoo/

Are there any volunteers to test it? I use it on my computers for two
months.

Alessio Ababilov
Senior Software Engineer
Grid Dynamics

[-- Attachment #2: Type: text/html, Size: 529 bytes --]

Re: [gentoo-user] Optional /usr merge in Gentoo

[东方巽雷]

[-- Attachment #1: Type: text/plain, Size: 412 bytes --]

more information?


2013/8/13 Alessio Ababilov <ilovegnulinux@gmail.com>

> Hi!
>
> I wrote a script that allows /usr merge in Gentoo without changes to
> ebuilds.
>
> I described it in an article
> http://aababilov.wordpress.com/2013/07/17/usr-merge-in-gentoo/
>
> Are there any volunteers to test it? I use it on my computers for two
> months.
>
> Alessio Ababilov
> Senior Software Engineer
> Grid Dynamics
>

[-- Attachment #2: Type: text/html, Size: 983 bytes --]

Re: [gentoo-user] Optional /usr merge in Gentoo

[Alessio Ababilov]

[-- Attachment #1: Type: text/plain, Size: 1120 bytes --]

"/usr merge" is the process of making /bin, /sbin, and /lib to be symlinks
to corresponding directories in /usr. It is done in Fedora and several
other distros now, and also in Solaris 15 years ago.
Benefits from /usr merge are described here:
http://www.freedesktop.org/wiki/Software/systemd/TheCaseForTheUsrMerge/
Technical details are here:
http://aababilov.wordpress.com/2013/07/17/usr-merge-in-gentoo/

In few words, the script is run once to merge /usr on a running system.
Also, the script is installed in post_src_install hook to perform /usr
merge during package updates or installations.

2013/8/13 东方巽雷 <dongfangxunlei@gmail.com>

> more information?
>
>
> 2013/8/13 Alessio Ababilov <ilovegnulinux@gmail.com>
>
>> Hi!
>>
>> I wrote a script that allows /usr merge in Gentoo without changes to
>> ebuilds.
>>
>> I described it in an article
>> http://aababilov.wordpress.com/2013/07/17/usr-merge-in-gentoo/
>>
>> Are there any volunteers to test it? I use it on my computers for two
>> months.
>>
>> Alessio Ababilov
>> Senior Software Engineer
>> Grid Dynamics
>>
>
>

[-- Attachment #2: Type: text/html, Size: 2648 bytes --]

Re: [gentoo-user] Optional /usr merge in Gentoo

[pk]

On 2013-08-13 16:05, Alessio Ababilov wrote:
> "/usr merge" is the process of making /bin, /sbin, and /lib to be symlinks
> to corresponding directories in /usr. It is done in Fedora and several
> other distros now, and also in Solaris 15 years ago.
> Benefits from /usr merge are described here:
> http://www.freedesktop.org/wiki/Software/systemd/TheCaseForTheUsrMerge/
> Technical details are here:
> http://aababilov.wordpress.com/2013/07/17/usr-merge-in-gentoo/
> 
> In few words, the script is run once to merge /usr on a running system.
> Also, the script is installed in post_src_install hook to perform /usr
> merge during package updates or installations.

So, how would this work for me who have /usr on a separate harddrive?
And what would be the benefit? To me, mentioning Fedora, makes the alarm
bells go off...

Best regards

Peter K

Re: [gentoo-user] Optional /usr merge in Gentoo

[the]

On 08/13/13 18:05, Alessio Ababilov wrote:
> "/usr merge" is the process of making /bin, /sbin, and /lib to be
> symlinks to corresponding directories in /usr. It is done in Fedora and
> several other distros now, and also in Solaris 15 years ago.
> Benefits from /usr merge are described here:
> http://www.freedesktop.org/wiki/Software/systemd/TheCaseForTheUsrMerge/
> Technical details are here:
> http://aababilov.wordpress.com/2013/07/17/usr-merge-in-gentoo/
>
> In few words, the script is run once to merge /usr on a running system.
> Also, the script is installed in post_src_install hook to perform /usr
> merge during package updates or installations.

The site doesn't describe any real problems.

Also I don't see how the current dir tree is not compatible
with gnu autoconf/automake.

-- 
Stop talking and start compiling.
Linux user #557897

Re: [gentoo-user] Optional /usr merge in Gentoo

[Alessio Ababilov]

[-- Attachment #1: Type: text/plain, Size: 1464 bytes --]

2013/8/13 the <the.guard@mail.ru>

> The site doesn't describe any real problems.
>
Well, it is a question to discuss.
I am not going to begin a holy war, I would like just to provide a
possibility to perform a harmless /usr merge for those who share
FreeDesktop's opinion.

>
> Also I don't see how the current dir tree is not compatible
> with gnu autoconf/automake.
>
In a simple way: please look at coreutils-8.20.ebuild that has to move a
lot of binaries from /usr/bin to /bin:

                cd "${D}"/usr/bin
                dodir /bin
                # move critical binaries into /bin (required by FHS)
                local fhs="cat chgrp chmod chown cp date dd df echo false
ln ls
                           mkdir mknod mv pwd rm rmdir stty sync true uname"
                mv ${fhs} ../../bin/ || die "could not move fhs bins"

2013/8/13 pk <peterk2@coolmail.se>

> So, how would this work for me who have /usr on a separate harddrive?
>
If you have an initrd, it will work.
Anyway, I just look for people that are interested in /usr merge.

And what would be the benefit? To me, mentioning Fedora, makes the alarm
> bells go off...
>
Yes. it does. Fedora is a big distro sponsored by Red Hat and its /usr
merge will be in RHEL-7. That's not a great idea to fight against upstream
if it will do /usr merge. Remember, /bin/mail now is moved to /usr/bin/mail
- what will be the next?

Sincerely,
Alessio Ababilov
Senior Software Engineer
Grid Dynamics

[-- Attachment #2: Type: text/html, Size: 3227 bytes --]

Re: [gentoo-user] Optional /usr merge in Gentoo

[Canek Peláez Valdés]

On Tue, Aug 13, 2013 at 4:08 AM, Alessio Ababilov
<ilovegnulinux@gmail.com> wrote:
> Hi!

Hi Alessio.

> I wrote a script that allows /usr merge in Gentoo without changes to
> ebuilds.
>
> I described it in an article
> http://aababilov.wordpress.com/2013/07/17/usr-merge-in-gentoo/
>
> Are there any volunteers to test it? I use it on my computers for two
> months.

I think it's a great experiment, but perhaps too much work for little
gain, at least currently.

The next council meeting will vote if separated /usr without and
initramfs is officially supported by Gentoo; I hope this time around
finally is officially and unequivocally stated by the council that a
separated /usr without an initramfs is *NOT* supported.

When that is out of the way, several packages will start to naturally
move to /usr, since most upstreams are doing that, and eventually we
will have empty /bin, /sbin, and /lib directories. Then there will be
no need for a script to move everything to /usr; which is good: I
believe in Gentoo a flag-day doesn't really work.

The usr-merge will be a slow, gradual change; it will probably take
years. The systemd package entered the tree in June 2011, after more
than a year in an overlay, and then it took more than two years to
make it an official alternative to OpenRC. The /usr merge will take a
similar amount of time, if not longer.

But it's good to know that you can do the merge now; thanks for
sharing your experiment.

Regards.
-- 
Canek Peláez Valdés
Posgrado en Ciencia e Ingeniería de la Computación
Universidad Nacional Autónoma de México

Re: [gentoo-user] Optional /usr merge in Gentoo

[Daniel Campbell]

On 08/13/2013 01:08 PM, Alessio Ababilov wrote:
> 
> 2013/8/13 the <the.guard@mail.ru <mailto:the.guard@mail.ru>>
> 
>     The site doesn't describe any real problems.
> 
> Well, it is a question to discuss.
> I am not going to begin a holy war, I would like just to provide a
> possibility to perform a harmless /usr merge for those who share
> FreeDesktop's opinion.
> 
> 
>     Also I don't see how the current dir tree is not compatible
>     with gnu autoconf/automake. 
> 
> In a simple way: please look at coreutils-8.20.ebuild that has to move a
> lot of binaries from /usr/bin to /bin:
> 
>                 cd "${D}"/usr/bin
>                 dodir /bin
>                 # move critical binaries into /bin (required by FHS)
>                 local fhs="cat chgrp chmod chown cp date dd df echo
> false ln ls
>                            mkdir mknod mv pwd rm rmdir stty sync true uname"
>                 mv ${fhs} ../../bin/ || die "could not move fhs bins"
> 
> 2013/8/13 pk <peterk2@coolmail.se <mailto:peterk2@coolmail.se>>
> 
>     So, how would this work for me who have /usr on a separate harddrive?
> 
> If you have an initrd, it will work.
> Anyway, I just look for people that are interested in /usr merge.
> 
>     And what would be the benefit? To me, mentioning Fedora, makes the alarm
>     bells go off...
> 
> Yes. it does. Fedora is a big distro sponsored by Red Hat and its /usr
> merge will be in RHEL-7. That's not a great idea to fight against
> upstream if it will do /usr merge. Remember, /bin/mail now is moved to
> /usr/bin/mail - what will be the next?
> 
> Sincerely,
> Alessio Ababilov
> Senior Software Engineer
> Grid Dynamics
Red Hat is only upstream for GNOME and systemd. What they choose to do
with their distro should not affect the choices of any other distro. I
see no reason for a /usr merge unless one is using Fedora or wants to
turn their Gentoo installation into a makeshift Fedora installation.
This merge should not be forced on Gentoo whatsoever.

Re: [gentoo-user] Optional /usr merge in Gentoo

[Alessio Ababilov]

[-- Attachment #1: Type: text/plain, Size: 2027 bytes --]

2013/8/13 Canek Peláez Valdés <caneko@gmail.com>

> I think it's a great experiment, but perhaps too much work for little
> gain, at least currently.
>
> Thank you!

> The next council meeting will vote if separated /usr without and
> initramfs is officially supported by Gentoo; I hope this time around
> finally is officially and unequivocally stated by the council that a
> separated /usr without an initramfs is *NOT* supported.
>
As I see from
http://www.gentoo.org/proj/en/council/meeting-logs/20130813.txt, the
council has stated that it is not supported anymore.

The usr-merge will be a slow, gradual change; it will probably take
> years. The systemd package entered the tree in June 2011, after more
> than a year in an overlay, and then it took more than two years to
> make it an official alternative to OpenRC. The /usr merge will take a
> similar amount of time, if not longer.
>
> Yes, but systemd is a large important package and it requires changes to
startup files in other packages, so, it took a lot of time.

As the opposite, /usr merge is easier and, IMHO, it doesn't introduce any
_obvious_ problems to Gentoo.

2013/8/16 Daniel Campbell <lists@sporkbox.us>

>
> Red Hat is only upstream for GNOME and systemd. What they choose to do
> with their distro should not affect the choices of any other distro. I
> see no reason for a /usr merge unless one is using Fedora or wants to
> turn their Gentoo installation into a makeshift Fedora installation.
> This merge should not be forced on Gentoo whatsoever.
>
>
I would like to ask you to understand my intension. I believe that Gentoo
is a distro that is famous for providing choises (USE flags and so on).
/usr merge is also a choise, and I look for volunteers and supporters.
BTW, /usr merge is not just a Fedora's caprice: is is done in Arch this
year:
https://mailman.archlinux.org/pipermail/arch-dev-public/2012-March/022625.html


Sincerely,
Alessio Ababilov
Senior Software Engineer
Grid Dynamics

[-- Attachment #2: Type: text/html, Size: 3634 bytes --]

Re: [gentoo-user] Optional /usr merge in Gentoo

[Tanstaafl]

On 2013-08-16 8:29 AM, Alessio Ababilov <ilovegnulinux@gmail.com> wrote:
> 2013/8/13 Canek Peláez Valdés <caneko@gmail.com <mailto:caneko@gmail.com>>
>
>     I think it's a great experiment, but perhaps too much work for little
>     gain, at least currently.
>
> Thank you!
>
>     The next council meeting will vote if separated /usr without and
>     initramfs is officially supported by Gentoo; I hope this time around
>     finally is officially and unequivocally stated by the council that a
>     separated /usr without an initramfs is *NOT* supported.
>
> As I see from
> http://www.gentoo.org/proj/en/council/meeting-logs/20130813.txt, the
> council has stated that it is not supported anymore.

<sigh>

Great. So what does this mean for those of us with older systems with 
separate /usr and don't want initramfs?

Re: [gentoo-user] Optional /usr merge in Gentoo

[Canek Peláez Valdés]

On Fri, Aug 16, 2013 at 7:29 AM, Alessio Ababilov
<ilovegnulinux@gmail.com> wrote:
> 2013/8/13 Canek Peláez Valdés <caneko@gmail.com>
>>
>> I think it's a great experiment, but perhaps too much work for little
>> gain, at least currently.
>>
> Thank you!
>>
>> The next council meeting will vote if separated /usr without and
>> initramfs is officially supported by Gentoo; I hope this time around
>> finally is officially and unequivocally stated by the council that a
>> separated /usr without an initramfs is *NOT* supported.
>
> As I see from
> http://www.gentoo.org/proj/en/council/meeting-logs/20130813.txt, the council
> has stated that it is not supported anymore.

Well, better late than never. It was about time.

>> The usr-merge will be a slow, gradual change; it will probably take
>> years. The systemd package entered the tree in June 2011, after more
>> than a year in an overlay, and then it took more than two years to
>> make it an official alternative to OpenRC. The /usr merge will take a
>> similar amount of time, if not longer.
>>
> Yes, but systemd is a large important package and it requires changes to
> startup files in other packages, so, it took a lot of time.
>
> As the opposite, /usr merge is easier and, IMHO, it doesn't introduce any
> _obvious_ problems to Gentoo.

Perhaps; please understand that I'm 100% behind the /usr merge. But
even if it's easier than the introduction of virtual/service-manager,
it's still true that in Gentoo flag days kinda don't work. The /usr
merge will happen as more and more programs move naturally from / to
/usr.

Regards.
-- 
Canek Peláez Valdés
Posgrado en Ciencia e Ingeniería de la Computación
Universidad Nacional Autónoma de México

Re: [gentoo-user] Optional /usr merge in Gentoo

[Canek Peláez Valdés]

On Fri, Aug 16, 2013 at 7:35 AM, Tanstaafl <tanstaafl@libertytrek.org> wrote:
> On 2013-08-16 8:29 AM, Alessio Ababilov <ilovegnulinux@gmail.com> wrote:
>>
>> 2013/8/13 Canek Peláez Valdés <caneko@gmail.com <mailto:caneko@gmail.com>>
>>
>>
>>     I think it's a great experiment, but perhaps too much work for little
>>     gain, at least currently.
>>
>> Thank you!
>>
>>     The next council meeting will vote if separated /usr without and
>>     initramfs is officially supported by Gentoo; I hope this time around
>>     finally is officially and unequivocally stated by the council that a
>>     separated /usr without an initramfs is *NOT* supported.
>>
>> As I see from
>> http://www.gentoo.org/proj/en/council/meeting-logs/20130813.txt, the
>> council has stated that it is not supported anymore.
>
>
> <sigh>
>
> Great. So what does this mean for those of us with older systems with
> separate /usr and don't want initramfs?

It means exactly what the Council voted:

"Since that particular setup may already be subtly broken today
depending on the installed software, Council recommends using an early
boot mount mechanism, e.g. initramfs, to mount /usr if /usr is on a
separate partition."

If you don't want an initramfs, you are on your own. Things will start
to break subtly (probably they *are* broken *now*, you just didn't
noticed), and if you file bugs about it they will be closed as WONTFIX
or INVALID.

If you want your system to be supported, you need an initarmfs, or
anything similar that allows the system to mount /usr really early in
the boot process.

https://wiki.gentoo.org/wiki/Early_Userspace_Mounting

http://www.gentoo.org/doc/en/initramfs-guide.xml

By a quick lecture of the Council session, I believe they are even
open to a closer /usr merge than I thought. When that happens (if it
happens), your system (if you keep upgrading) will not be able to boot
for sure if you don't follow the Council suggestion.

Regards.
-- 
Canek Peláez Valdés
Posgrado en Ciencia e Ingeniería de la Computación
Universidad Nacional Autónoma de México

Re: [gentoo-user] Optional /usr merge in Gentoo

[Daniel Campbell]

On 08/16/2013 07:29 AM, Alessio Ababilov wrote:
> 2013/8/13 Canek Peláez Valdés <caneko@gmail.com <mailto:caneko@gmail.com>>
> 
>     I think it's a great experiment, but perhaps too much work for little
>     gain, at least currently.
> 
> Thank you! 
> 
>     The next council meeting will vote if separated /usr without and
>     initramfs is officially supported by Gentoo; I hope this time around
>     finally is officially and unequivocally stated by the council that a
>     separated /usr without an initramfs is *NOT* supported.
> 
> As I see
> from http://www.gentoo.org/proj/en/council/meeting-logs/20130813.txt,
> the council has stated that it is not supported anymore.
> 
>     The usr-merge will be a slow, gradual change; it will probably take
>     years. The systemd package entered the tree in June 2011, after more
>     than a year in an overlay, and then it took more than two years to
>     make it an official alternative to OpenRC. The /usr merge will take a
>     similar amount of time, if not longer.
> 
> Yes, but systemd is a large important package and it requires changes to
> startup files in other packages, so, it took a lot of time.
> 
> As the opposite, /usr merge is easier and, IMHO, it doesn't introduce
> any _obvious_ problems to Gentoo.
> 
> 2013/8/16 Daniel Campbell <lists@sporkbox.us <mailto:lists@sporkbox.us>>
> 
> 
>     Red Hat is only upstream for GNOME and systemd. What they choose to do
>     with their distro should not affect the choices of any other distro. I
>     see no reason for a /usr merge unless one is using Fedora or wants to
>     turn their Gentoo installation into a makeshift Fedora installation.
>     This merge should not be forced on Gentoo whatsoever.
> 
> 
> I would like to ask you to understand my intension. I believe that
> Gentoo is a distro that is famous for providing choises (USE flags and
> so on). /usr merge is also a choise, and I look for volunteers
> and supporters.
> BTW, /usr merge is not just a Fedora's caprice: is is done in Arch this
> year:
> https://mailman.archlinux.org/pipermail/arch-dev-public/2012-March/022625.html
> 
> 
> Sincerely,
> Alessio Ababilov
> Senior Software Engineer
> Grid Dynamics

I'm completely in favor of choice, but only if it doesn't impede on any
other choice(s). If /usr merges are completely optional and only tied to
software that require it (read: systemd), then I'm fine. But requiring
people to have an initramfs to boot a system that doesn't legitimately
require it is silly. I don't even have /usr mounted separately, but
there are many, many different system configurations out there and
Gentoo is famous for supporting a wide variety. That variety is stomped
on if something like a /usr merge is forced. It also makes building your
default environment more complicated due to generating an initramfs.

Arch is following Fedora as they consider them an upstream. They were
one of, if not *the* first non-Fedora distros to ship systemd by
default. They're a poor example. Really, Arch is just Fedora with a
better package manager.

~Daniel

Re: [gentoo-user] Optional /usr merge in Gentoo

[the.guard]

> But requiring
> people to have an initramfs to boot a system that doesn't legitimately
> require it is silly. I don't even have /usr mounted separately, but
> there are many, many different system configurations out there and
> Gentoo is famous for supporting a wide variety. That variety is stomped
> on if something like a /usr merge is forced. It also makes building your
> default environment more complicated due to generating an initramfs.

Absolutely agreed.

Re: [gentoo-user] Optional /usr merge in Gentoo

[Dan Johansson]

[-- Attachment #1.1: Type: text/plain, Size: 1423 bytes --]

On 16.08.2013 15:57, Canek Peláez Valdés wrote:
> On Fri, Aug 16, 2013 at 7:29 AM, Alessio Ababilov
> <ilovegnulinux@gmail.com> wrote:
>> 2013/8/13 Canek Peláez Valdés <caneko@gmail.com>
>>>
>>> I think it's a great experiment, but perhaps too much work for little
>>> gain, at least currently.
>>>
>> Thank you!
>>>
>>> The next council meeting will vote if separated /usr without and
>>> initramfs is officially supported by Gentoo; I hope this time around
>>> finally is officially and unequivocally stated by the council that a
>>> separated /usr without an initramfs is *NOT* supported.
>>
>> As I see from
>> http://www.gentoo.org/proj/en/council/meeting-logs/20130813.txt, the council
>> has stated that it is not supported anymore.
> 
> Well, better late than never. It was about time.
> 
>>> The usr-merge will be a slow, gradual change; it will probably take
>>> years. The systemd package entered the tree in June 2011, after more
>>> than a year in an overlay, and then it took more than two years to
>>> make it an official alternative to OpenRC. The /usr merge will take a
>>> similar amount of time, if not longer.
>>>

And when we are at it, why not rename '/' to 'C:\' ?

-- 
Dan Johansson, <http://www.dmj.nu>
***************************************************
This message is printed on 100% recycled electrons!
***************************************************

[-- Attachment #1.2: 0x2FB894AD.asc --]
[-- Type: application/pgp-keys, Size: 3477 bytes --]

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 255 bytes --]

Re: [gentoo-user] Optional /usr merge in Gentoo

[Alon Bar-Lev]

On Sat, Aug 17, 2013 at 2:49 PM, Dan Johansson <Dan.Johansson@dmj.nu> wrote:
> On 16.08.2013 15:57, Canek Peláez Valdés wrote:
>> On Fri, Aug 16, 2013 at 7:29 AM, Alessio Ababilov
>> <ilovegnulinux@gmail.com> wrote:
>>> 2013/8/13 Canek Peláez Valdés <caneko@gmail.com>
>>>>
>>>> I think it's a great experiment, but perhaps too much work for little
>>>> gain, at least currently.
>>>>
>>> Thank you!
>>>>
>>>> The next council meeting will vote if separated /usr without and
>>>> initramfs is officially supported by Gentoo; I hope this time around
>>>> finally is officially and unequivocally stated by the council that a
>>>> separated /usr without an initramfs is *NOT* supported.
>>>
>>> As I see from
>>> http://www.gentoo.org/proj/en/council/meeting-logs/20130813.txt, the council
>>> has stated that it is not supported anymore.
>>
>> Well, better late than never. It was about time.
>>
>>>> The usr-merge will be a slow, gradual change; it will probably take
>>>> years. The systemd package entered the tree in June 2011, after more
>>>> than a year in an overlay, and then it took more than two years to
>>>> make it an official alternative to OpenRC. The /usr merge will take a
>>>> similar amount of time, if not longer.
>>>>
>
> And when we are at it, why not rename '/' to 'C:\' ?

Good one!!!!! :)

I guess this merge happening only because systemd...

Now the council expects people to:

1. maintain initramfs, it can be complex or simple task, depend on the
configuration.

2. place all disk and filesystem recovery utilities within initramfs.

3. or... prepare to use rescue cd every time something is broken.

Unclear why exactly we do have support in separate /usr.

Regards,
Alon

Re: [gentoo-user] Optional /usr merge in Gentoo

[Stroller]

On 17 August 2013, at 12:49, Dan Johansson wrote:
> ...
>>>> The usr-merge will be a slow, gradual change; it will probably take
>>>> years. The systemd package entered the tree in June 2011, after more
>>>> than a year in an overlay, and then it took more than two years to
>>>> make it an official alternative to OpenRC. The /usr merge will take a
>>>> similar amount of time, if not longer.
>>>> 
> 
> And when we are at it, why not rename '/' to 'C:\' ?

Well, seriously, why not?

You haven't made any arguments against putting everything on a single partition, just made a cheap "lolz, micro$oft windoze" analogy.

I can understand wanting to put /home on a separate partition or /var/spool/mail or /var/www/sites but I don't understand this obsession with several different partitions for system files which are always going to be managed by portage and which I'm never going to move or mess with manually.

Having /usr on a separate partition dates back to an era in which 10MB and 40MB harddisks were prohibitively expensive - they cost $1000s.

Now we can host a complete Gentoo system on a $5 or $10 SDcard, I'm struggling to see the value.

Stroller.

Re: [gentoo-user] Optional /usr merge in Gentoo

[Alan McKinnon]

On 18/08/2013 08:40, Stroller wrote:
> 
> On 17 August 2013, at 12:49, Dan Johansson wrote:
>> ...
>>>>> The usr-merge will be a slow, gradual change; it will probably take
>>>>> years. The systemd package entered the tree in June 2011, after more
>>>>> than a year in an overlay, and then it took more than two years to
>>>>> make it an official alternative to OpenRC. The /usr merge will take a
>>>>> similar amount of time, if not longer.
>>>>>
>>
>> And when we are at it, why not rename '/' to 'C:\' ?
> 
> Well, seriously, why not?
> 
> You haven't made any arguments against putting everything on a single partition, just made a cheap "lolz, micro$oft windoze" analogy.
> 
> I can understand wanting to put /home on a separate partition or /var/spool/mail or /var/www/sites but I don't understand this obsession with several different partitions for system files which are always going to be managed by portage and which I'm never going to move or mess with manually.
> 
> Having /usr on a separate partition dates back to an era in which 10MB and 40MB harddisks were prohibitively expensive - they cost $1000s.
> 
> Now we can host a complete Gentoo system on a $5 or $10 SDcard, I'm struggling to see the value.

I agree.

You've read that post to an embedded list that lays out clearly why this
/usr thing happened, right? I see computer files falling in two large
categories - the system and data. Portage manages the system, I only
need to ensure there's enough space. The data is mine and I may well
have very different needs for different parts - the fs settings for the
portage tree definitely don't work well for my media store with 4G
BluRay rips!

While we're on the topic, what's the obsession with having different
bits of the file hierarchy as different *mount points*? That harks back
to the days when the only way to have a chunk of fs space be different
was to have it as a separate physical thing and mount it. Nowadays we
have something better - ZFS. To me this makes so much more sense. I have
a large amount of storage called a pool, and set size limits and
characteristics for various directories without having to deal with
fixed size volumes.

There's LVM of course which makes things far easier than not having LVM,
but by $DEITY, it forces me to think of my storage in terms of 4
distinctly different layers = far too complex (even though the clever
design appeals to my inner nerd).

I can think of only one modern use case where a separate /usr is
desirable - as a read-only NFS mount for terminal servers. But that is
already a large complex setup, very stable and not changing much,
usually with an admin, so a boot environment with an initramfs shouldn't
be any real burden at all.


-- 
Alan McKinnon
alan.mckinnon@gmail.com

Re: [gentoo-user] Optional /usr merge in Gentoo

[Tanstaafl]

On 2013-08-18 5:16 AM, Alan McKinnon <alan.mckinnon@gmail.com> wrote:
> While we're on the topic, what's the obsession with having different
> bits of the file hierarchy as different*mount points*? That harks back
> to the days when the only way to have a chunk of fs space be different
> was to have it as a separate physical thing and mount it. Nowadays we
> have something better - ZFS. To me this makes so much more sense. I have
> a large amount of storage called a pool, and set size limits and
> characteristics for various directories without having to deal with
> fixed size volumes.

Eh? *Who* has ZFS? Certainly not the linux kernel.

Re: [gentoo-user] Optional /usr merge in Gentoo

[Stroller]

On 18 August 2013, at 15:16, pk wrote:
> ...
> 1. Most of the time spent when cold booting is spent in the BIOS/UEFI
> cycle (around 30 seconds), the time from grub display to login (I'm
> using "slim") is 5 seconds (max). 

Blimey! You must have a slow BIOS cycle.

I mean, maybe my servers take that long (I'm not sure, I boot them annually and don't watch them rebooting) but I have a little eMachines nettop here - the first time I tried to enter BIOS, it look me several attempts, it boots past that so quick!

I've now enabled the option to wait 5 seconds before loading the bootloader, but quickboot on this system is less than 2 seconds in BIOS cycle.

(OTOH, going from grub to login in 5 seconds - that suggests to me that you're using an SSD and not a hard-drive). 

Stroller.

Re: [gentoo-user] Optional /usr merge in Gentoo

[Alan McKinnon]

On 19/08/2013 11:21, Stroller wrote:
> 
> On 18 August 2013, at 15:16, pk wrote:
>> ...
>> 1. Most of the time spent when cold booting is spent in the BIOS/UEFI
>> cycle (around 30 seconds), the time from grub display to login (I'm
>> using "slim") is 5 seconds (max). 
> 
> Blimey! You must have a slow BIOS cycle.
> 
> I mean, maybe my servers take that long (I'm not sure, I boot them annually and don't watch them rebooting) but I have a little eMachines nettop here - the first time I tried to enter BIOS, it look me several attempts, it boots past that so quick!
> 
> I've now enabled the option to wait 5 seconds before loading the bootloader, but quickboot on this system is less than 2 seconds in BIOS cycle.
> 
> (OTOH, going from grub to login in 5 seconds - that suggests to me that you're using an SSD and not a hard-drive). 


What pk says is quite normal in my experience.

This laptop is a Dell Precision, from pressing enter on the grub screen
to kdm showing on the screen is 3 seconds, another 4 seconds for KDE to
appear and start responding to mouse clicks.

From power-on to the grub menu showing, that's about 30 seconds. The
first 8 or so is a ... blank screen ... then I get the Dell logo,
followed by another 20 seconds or so where is does $SOMETHING.

Server hardware is even worse - the R[357]* series can easily take 4
MINUTES to get through all the various BIOS thingies. Bi-monthly
maintenance reboots get scary, 4 minutes is a loooooooong time when
you're flying blind on a critical machine that's physically on the other
side of town :-)


-- 
Alan McKinnon
alan.mckinnon@gmail.com

Re: [gentoo-user] Optional /usr merge in Gentoo

[pk]

On 2013-08-19 11:21, Stroller wrote:

> Blimey! You must have a slow BIOS cycle.

Yes, I bought the motherboard specifically for a slow BIOS cycle... ;-)
Joke aside, I have a SAS raid card in the machine which probes the
harddrives (four mechanical ones) which takes maybe half that time. I've
been toying with the idea of replacing BIOS/UEFI with coreboot/seabios
but time is lacking... :-(
For the record, I've always felt BIOS have been slow...

> (OTOH, going from grub to login in 5 seconds - that suggests to me that you're using an SSD and not a hard-drive). 

I recently bought 4 SSDs (Intel 520 60GB) and have them installed as
/usr, /var and /tmp with one spare. However / is still on the SAS raid
card and boot time has not improved by much with the SSD. It's matter of
what crap you load at boot that will affect your boot time.

Best regards

Peter K

Re: [gentoo-user] Optional /usr merge in Gentoo

[Alan McKinnon]

On 18/08/2013 21:38, Tanstaafl wrote:
> On 2013-08-18 5:16 AM, Alan McKinnon <alan.mckinnon@gmail.com> wrote:
>> While we're on the topic, what's the obsession with having different
>> bits of the file hierarchy as different*mount points*? That harks back
>> to the days when the only way to have a chunk of fs space be different
>> was to have it as a separate physical thing and mount it. Nowadays we
>> have something better - ZFS. To me this makes so much more sense. I have
>> a large amount of storage called a pool, and set size limits and
>> characteristics for various directories without having to deal with
>> fixed size volumes.
> 
> Eh? *Who* has ZFS? Certainly not the linux kernel.
> 

FreeBSD

You can get ZFS on Linux with relative ease, you just have to build it
yourself. Distros feel they can't redistribute that code.



The bit you quoted shouldn't be read to mean that we have ZFS, it works
on Linux and everyone should activate it and use it and chuck ext* out
the window.

I meant that we've been chugging along since 1982 or so with ancient
disk concepts that come mostly from MS_DOS and limited by that hardware
of that day.

And here we are in 2013 *still* fiddling with partition tables, fixed
file systems, fixed mountpoints and we still bang our heads weekly
because sda3 has proven to be too small, and it's a *huge* mission to
change it. Yes, LVM has made this sooooo much easier (kudos to Sistina
for that) but I believe the entire approach is wrong.

The ZFS approach is better - here's the storage, now do with it what I
want but don't employ arbitrary fixed limits and structures to do it.


-- 
Alan McKinnon
alan.mckinnon@gmail.com

Re: [gentoo-user] Optional /usr merge in Gentoo

[Pandu Poluan]

[-- Attachment #1: Type: text/plain, Size: 2492 bytes --]

On Aug 26, 2013 5:06 AM, "Alan McKinnon" <alan.mckinnon@gmail.com> wrote:
>
> On 18/08/2013 21:38, Tanstaafl wrote:
> > On 2013-08-18 5:16 AM, Alan McKinnon <alan.mckinnon@gmail.com> wrote:
> >> While we're on the topic, what's the obsession with having different
> >> bits of the file hierarchy as different*mount points*? That harks back
> >> to the days when the only way to have a chunk of fs space be different
> >> was to have it as a separate physical thing and mount it. Nowadays we
> >> have something better - ZFS. To me this makes so much more sense. I
have
> >> a large amount of storage called a pool, and set size limits and
> >> characteristics for various directories without having to deal with
> >> fixed size volumes.
> >
> > Eh? *Who* has ZFS? Certainly not the linux kernel.
> >
>
> FreeBSD
>
> You can get ZFS on Linux with relative ease, you just have to build it
> yourself. Distros feel they can't redistribute that code.
>
>
>
> The bit you quoted shouldn't be read to mean that we have ZFS, it works
> on Linux and everyone should activate it and use it and chuck ext* out
> the window.
>
> I meant that we've been chugging along since 1982 or so with ancient
> disk concepts that come mostly from MS_DOS and limited by that hardware
> of that day.
>
> And here we are in 2013 *still* fiddling with partition tables, fixed
> file systems, fixed mountpoints and we still bang our heads weekly
> because sda3 has proven to be too small, and it's a *huge* mission to
> change it. Yes, LVM has made this sooooo much easier (kudos to Sistina
> for that) but I believe the entire approach is wrong.
>
> The ZFS approach is better - here's the storage, now do with it what I
> want but don't employ arbitrary fixed limits and structures to do it.
>

+1 on ZFS. It's honestly a truly *modern* filesystem.

Been using it as the storage back-end of my company's email server.

The zpool and zfs command may need some time to be familiar with, but the
self-mounting self-sharing ability of zfs (i.e., no need to muck with fstab
and exports files) is really sweet.

I really leveraged its ability to do what I call "delta snapshot shipping"
(i.e., send only the differences between two snapshots to another place).
It's almost like an asynchronous DRBD, but with the added peace of mind
that if the files become corrupted (due to buggy app, almost no way for ZFS
to let corrupt data exist), I can easily 'roll back' to the time where the
files are still uncorrupted.

Rgds,
--

[-- Attachment #2: Type: text/html, Size: 3164 bytes --]

Re: [gentoo-user] Optional /usr merge in Gentoo

[Alan McKinnon]

On 26/08/2013 08:10, Pandu Poluan wrote:
>> The ZFS approach is better - here's the storage, now do with it what I
>> want but don't employ arbitrary fixed limits and structures to do it.
>>
> 
> +1 on ZFS. It's honestly a truly *modern* filesystem.
> 
> Been using it as the storage back-end of my company's email server.
> 
> The zpool and zfs command may need some time to be familiar with, but
> the self-mounting self-sharing ability of zfs (i.e., no need to muck
> with fstab and exports files) is really sweet.
> 
> I really leveraged its ability to do what I call "delta snapshot
> shipping" (i.e., send only the differences between two snapshots to
> another place). It's almost like an asynchronous DRBD, but with the
> added peace of mind that if the files become corrupted (due to buggy
> app, almost no way for ZFS to let corrupt data exist), I can easily
> 'roll back' to the time where the files are still uncorrupted.
> 


I run it on my NASes, and the thing that really sold me was what it lets
me as the admin do:

I get all the benefits of directories with none of the downsides.
I get all the benefits of mount points with none of the downsides.
I get all the benefits of discrete filesystems with none of the downsides.

Like you say, a truly modern fs built for modern needs.



-- 
Alan McKinnon
alan.mckinnon@gmail.com

Re: [gentoo-user] Optional /usr merge in Gentoo

[Neil Bothwick]

[-- Attachment #1: Type: text/plain, Size: 440 bytes --]

On Mon, 26 Aug 2013 00:02:17 +0200, Alan McKinnon wrote:

> > Eh? *Who* has ZFS? Certainly not the linux kernel.
> >   
> 
> FreeBSD
> 
> You can get ZFS on Linux with relative ease, you just have to build it
> yourself. Distros feel they can't redistribute that code.

emerge zfs works too :)

I really liek the way ZFS just lets you get on with things.


-- 
Neil Bothwick

Help put the "fun" back in "dysfunctional" !

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: [gentoo-user] Optional /usr merge in Gentoo

[Mick]

[-- Attachment #1: Type: Text/Plain, Size: 618 bytes --]

On Monday 26 Aug 2013 08:06:13 Neil Bothwick wrote:
> On Mon, 26 Aug 2013 00:02:17 +0200, Alan McKinnon wrote:
> > > Eh? *Who* has ZFS? Certainly not the linux kernel.
> > 
> > FreeBSD
> > 
> > You can get ZFS on Linux with relative ease, you just have to build it
> > yourself. Distros feel they can't redistribute that code.
> 
> emerge zfs works too :)
> 
> I really liek the way ZFS just lets you get on with things.

Does anyone run it on a desktop/laptop as their day to day fs?  Any drawbacks 
or gotchas?  Other than reliability, how does it perform compared say to ext4?

-- 
Regards,
Mick

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 490 bytes --]

Re: [gentoo-user] Optional /usr merge in Gentoo

[Neil Bothwick]

[-- Attachment #1: Type: text/plain, Size: 592 bytes --]

On Mon, 26 Aug 2013 09:45:15 +0100, Mick wrote:

> > emerge zfs works too :)
> > 
> > I really like the way ZFS just lets you get on with things.  
> 
> Does anyone run it on a desktop/laptop as their day to day fs?

Yes.

> Any
> drawbacks or gotchas?  Other than reliability, how does it perform
> compared say to ext4?

I haven't benchmarked it. It feels as if it may be a little slower on my
desktop with spinning disks, but that may be down to other factors, like
impatience. It flies on my laptop's SSD.


-- 
Neil Bothwick

Why is bra singular and pants plural?

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: [gentoo-user] Optional /usr merge in Gentoo

[Pandu Poluan]

[-- Attachment #1: Type: text/plain, Size: 1108 bytes --]

On Mon, Aug 26, 2013 at 4:56 PM, Neil Bothwick <neil@digimed.co.uk> wrote:

> On Mon, 26 Aug 2013 09:45:15 +0100, Mick wrote:
>
> > > emerge zfs works too :)
> > >
> > > I really like the way ZFS just lets you get on with things.
> >
> > Does anyone run it on a desktop/laptop as their day to day fs?
>
> Yes.
>
> > Any
> > drawbacks or gotchas?  Other than reliability, how does it perform
> > compared say to ext4?
>
> I haven't benchmarked it. It feels as if it may be a little slower on my
> desktop with spinning disks, but that may be down to other factors, like
> impatience. It flies on my laptop's SSD.
>

Additional note:

*Of course* it will be slower than ext*, because during every read it
ensures that the block being read has a proper checksum.

Likewise on writes.

But that IMO is very worth it just for the additional peace-of-mind,
knowing you will never ever have a silent corruption.


-- 
FdS Pandu E Poluan
* ~ IT Optimizer ~**
*
 • LOPSA Member #15248
 • Blog : http://pepoluan.tumblr.com
 • Linked-In : http://id.linkedin.com/in/pepoluan

[-- Attachment #2: Type: text/html, Size: 2647 bytes --]

Re: [gentoo-user] Optional /usr merge in Gentoo

[Stefan G. Weichinger]

Am 26.08.2013 10:45, schrieb Mick:

> Does anyone run it on a desktop/laptop as their day to day fs?  Any
> drawbacks or gotchas?  Other than reliability, how does it perform
> compared say to ext4?

Sorry for being shameless:

I once described a ZFS-based gentoo setup with encryption for the
german linux magazine. They translated it and it was published in
other parts of the world as well:

http://www.oops.co.at/en/publications/english-translation-of-zfs-article

I delivered a demo-VM as well but I don't run that setup on my
productive systems currently.

Stefan (not earning anything from those pdf-downloads, btw)

Re: [gentoo-user] Optional /usr merge in Gentoo

[Tanstaafl]

On 2013-08-25 6:02 PM, Alan McKinnon <alan.mckinnon@gmail.com> wrote:
> You can get ZFS on Linux with relative ease, you just have to build it
> yourself. Distros feel they can't redistribute that code.

I know you can do this as a module - but is there an overlay or patch to 
get it built directly into the kernel? I'd love to use ZFS on my gentoo 
server, but I disable modules on servers for security reasons.

Thanks...

Re: [gentoo-user] Optional /usr merge in Gentoo

[Neil Bothwick]

[-- Attachment #1: Type: text/plain, Size: 1535 bytes --]

On Mon, 26 Aug 2013 09:16:44 -0400, Tanstaafl wrote:

> > You can get ZFS on Linux with relative ease, you just have to build it
> > yourself. Distros feel they can't redistribute that code.  
> 
> I know you can do this as a module - but is there an overlay or patch
> to get it built directly into the kernel? I'd love to use ZFS on my
> gentoo server, but I disable modules on servers for security reasons.

You can do it. You have to unmask the kernel_builtin USE flag to stop zfs
bringing in zfs_kmod, then unpack the sources and run the script to
install them into the kernel tree.

I run this script after emerging a new kernel

==================================================

#!/bin/sh

[[ -f /usr/src/linux/.config ]] || zcat /proc/config.gz
>/usr/src/linux/.config

SPL_EBUILD=$(ls -1 /var/portage/sys-kernel/spl/spl-0* | tail -n 1)
ZFS_EBUILD=$(ls -1 /var/portage/sys-fs/zfs/zfs-0* | tail -n 1)

SPL_DIR=$(ebuild $SPL_EBUILD clean prepare | awk '/Preparing source in/
{print $5}') ZFS_DIR=$(ebuild $ZFS_EBUILD clean prepare | awk '/Preparing
source in/ {print $5}')

cd $SPL_DIR
./configure --enable-linux-builtin --with-linux=/usr/src/linux
./copy-builtin /usr/src/linux
      
cd $ZFS_DIR
./configure --enable-linux-builtin --with-linux=/usr/src/linux
--with-spl=$SPL_DIR ./copy-builtin /usr/src/linux

==================================================

Then run make oldconfig and compile as usual.


-- 
Neil Bothwick

Cross-country skiing is great in small countries.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: [gentoo-user] Optional /usr merge in Gentoo

[Alan McKinnon]

On 26/08/2013 16:38, Neil Bothwick wrote:
> On Mon, 26 Aug 2013 14:06:11 +0200, Stefan G. Weichinger wrote:
> 
>> Sorry for being shameless:
>>
>> I once described a ZFS-based gentoo setup with encryption for the
>> german linux magazine. They translated it and it was published in
>> other parts of the world as well:
> 
> That is pretty shameless. I would never be so blatant as to mention the
> ZFS tutorial in the current issue (175) of Linux Format.
> 
> 


If you give me a free subscription for life, I promise I won't breath a
word of you never mentioning ZFS




-- 
Alan McKinnon
alan.mckinnon@gmail.com

Re: [gentoo-user] Optional /usr merge in Gentoo

[Neil Bothwick]

[-- Attachment #1: Type: text/plain, Size: 580 bytes --]

On Mon, 26 Aug 2013 14:06:11 +0200, Stefan G. Weichinger wrote:

> Sorry for being shameless:
> 
> I once described a ZFS-based gentoo setup with encryption for the
> german linux magazine. They translated it and it was published in
> other parts of the world as well:

That is pretty shameless. I would never be so blatant as to mention the
ZFS tutorial in the current issue (175) of Linux Format.


-- 
Neil Bothwick

Head: (n.) the part of a disk drive which detects sectors and decides
which of the two possible values to return: 'lose a turn' or 'bankrupt.'

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: [gentoo-user] Optional /usr merge in Gentoo

[Stefan G. Weichinger]

Am 26.08.2013 16:38, schrieb Neil Bothwick:
> On Mon, 26 Aug 2013 14:06:11 +0200, Stefan G. Weichinger wrote:
> 
>> Sorry for being shameless:
>> 
>> I once described a ZFS-based gentoo setup with encryption for
>> the german linux magazine. They translated it and it was
>> published in other parts of the world as well:
> 
> That is pretty shameless. I would never be so blatant as to mention
> the ZFS tutorial in the current issue (175) of Linux Format.

;-)

Re: [gentoo-user] Optional /usr merge in Gentoo

[Tanstaafl]

On 2013-08-26 10:11 AM, Neil Bothwick <neil@digimed.co.uk> wrote:
> On Mon, 26 Aug 2013 09:16:44 -0400, Tanstaafl wrote:
>
>>> You can get ZFS on Linux with relative ease, you just have to build it
>>> yourself. Distros feel they can't redistribute that code.
>>
>> I know you can do this as a module - but is there an overlay or patch
>> to get it built directly into the kernel? I'd love to use ZFS on my
>> gentoo server, but I disable modules on servers for security reasons.
>
> You can do it. You have to unmask the kernel_builtin USE flag to stop zfs
> bringing in zfs_kmod, then unpack the sources and run the script to
> install them into the kernel tree.

<snip>

Very interesting, thanks... nice to know it can be done, but I wouldn't 
be uncomfortable doing that myself...

Would be nice if there was a kernel overlay for this...

Re: [gentoo-user] Optional /usr merge in Gentoo

[Neil Bothwick]

[-- Attachment #1: Type: text/plain, Size: 707 bytes --]

On Mon, 26 Aug 2013 12:36:30 -0400, Tanstaafl wrote:

> > You can do it. You have to unmask the kernel_builtin USE flag to stop
> > zfs bringing in zfs_kmod, then unpack the sources and run the script
> > to install them into the kernel tree.  
> 
> <snip>
> 
> Very interesting, thanks... nice to know it can be done, but I wouldn't 
> be uncomfortable doing that myself...
> 
> Would be nice if there was a kernel overlay for this...

The licensing conflict means that would not be possible. You have the
install the kernel source and then merge in the ZFS source yourself, it
can't be done for you and distributed.


-- 
Neil Bothwick

OPERATOR ERROR: Nyah, Nyah, Nyah, Nyah, Nyah!

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: [gentoo-user] Optional /usr merge in Gentoo

[Joerg Schilling]

Neil Bothwick <neil@digimed.co.uk> wrote:

> > Would be nice if there was a kernel overlay for this...
>
> The licensing conflict means that would not be possible. You have the
> install the kernel source and then merge in the ZFS source yourself, it
> can't be done for you and distributed.

Why do you believe this?

ZFS id doubtlessly an own "work" independent from the rest of the Linux kernel
and for this reason, adding ZFS just creates a collective work that is not 
affected by the GPL.

BTW: this was already explained in the GPL book from Till Jaeger et al. 
published in March 2005.

Jörg

-- 
 EMail:joerg@schily.isdn.cs.tu-berlin.de (home) Jörg Schilling D-13353 Berlin
       js@cs.tu-berlin.de                (uni)  
       joerg.schilling@fokus.fraunhofer.de (work) Blog: http://schily.blogspot.com/
 URL:  http://cdrecord.berlios.de/private/ ftp://ftp.berlios.de/pub/schily

Re: [gentoo-user] Optional /usr merge in Gentoo

[Neil Bothwick]

[-- Attachment #1: Type: text/plain, Size: 769 bytes --]

On Mon, 26 Aug 2013 19:30:05 +0200, Joerg Schilling wrote:

> > The licensing conflict means that would not be possible. You have the
> > install the kernel source and then merge in the ZFS source yourself,
> > it can't be done for you and distributed.  
> 
> Why do you believe this?
> 
> ZFS id doubtlessly an own "work" independent from the rest of the Linux
> kernel and for this reason, adding ZFS just creates a collective work
> that is not affected by the GPL.

But the CCDL licence of ZFS precludes its being distributed with the
kernel. At least, that's how I understand it and the fact that no distro
distributes a ZFS-enabled kernel makes me believe it is true.


-- 
Neil Bothwick

Friends come and friends go, but enemies accumulate.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: [gentoo-user] Optional /usr merge in Gentoo

[Joerg Schilling]

Neil Bothwick <neil@digimed.co.uk> wrote:

> On Mon, 26 Aug 2013 19:30:05 +0200, Joerg Schilling wrote:
>
> > > The licensing conflict means that would not be possible. You have the
> > > install the kernel source and then merge in the ZFS source yourself,
> > > it can't be done for you and distributed.  
> > 
> > Why do you believe this?
> > 
> > ZFS id doubtlessly an own "work" independent from the rest of the Linux
> > kernel and for this reason, adding ZFS just creates a collective work
> > that is not affected by the GPL.
>
> But the CCDL licence of ZFS precludes its being distributed with the
> kernel. At least, that's how I understand it and the fact that no distro
> distributes a ZFS-enabled kernel makes me believe it is true.

Did you ever read the CDDL?

People who believe that there is a problem use a wrong interpretation of the 
GPL. The CDDL definitely does not prevent combinations with other software.

Jörg

-- 
 EMail:joerg@schily.isdn.cs.tu-berlin.de (home) Jörg Schilling D-13353 Berlin
       js@cs.tu-berlin.de                (uni)  
       joerg.schilling@fokus.fraunhofer.de (work) Blog: http://schily.blogspot.com/
 URL:  http://cdrecord.berlios.de/private/ ftp://ftp.berlios.de/pub/schily

Re: [gentoo-user] Optional /usr merge in Gentoo

[Neil Bothwick]

[-- Attachment #1: Type: text/plain, Size: 998 bytes --]

On Mon, 26 Aug 2013 23:37:02 +0200, Joerg Schilling wrote:

> > But the CCDL licence of ZFS precludes its being distributed with the
> > kernel. At least, that's how I understand it and the fact that no
> > distro distributes a ZFS-enabled kernel makes me believe it is true.  
> 
> Did you ever read the CDDL?

Not completely.

> People who believe that there is a problem use a wrong interpretation
> of the GPL. The CDDL definitely does not prevent combinations with
> other software.

I didn't say the CDDL prevented this. I'm not blaming one of the other
licence, but they are considered to be incompatible. I realise you
believe otherwise, and you could well be correct, but those who distribute
the software either believe otherwise or feel there is enough doubt to be
cautious. If in doubt, don't.

I wish your interpretation was correct, but the prevailing option is
otherwise.


-- 
Neil Bothwick

Will we ever get out of this airport? asked Tom interminably.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: [gentoo-user] Optional /usr merge in Gentoo

[Joerg Schilling]

Neil Bothwick <neil@digimed.co.uk> wrote:

> > Did you ever read the CDDL?
>
> Not completely.

You should do it - it is even much shorter then GPLv3


> > People who believe that there is a problem use a wrong interpretation
> > of the GPL. The CDDL definitely does not prevent combinations with
> > other software.
>
> I didn't say the CDDL prevented this. I'm not blaming one of the other
> licence, but they are considered to be incompatible. I realise you
> believe otherwise, and you could well be correct, but those who distribute
> the software either believe otherwise or feel there is enough doubt to be
> cautious. If in doubt, don't.

There are several entities that frequently publish such unproven claims.
This sounds like marketing using the cause fear uncertaintly and doubt method.
You should not trust such entities that do not prove their claims.

> I wish your interpretation was correct, but the prevailing option is
> otherwise.

It is not my interpretation, this is the interpretation of all lawyers in the 
net that are willing to explain the background of their decisions.

This interpretation is based on two basic facts:

-	The CDDL was designed for best compatibilitiy with all licenses.

-	The parts of the GPL that are claimed to prevent this license
	combination are in conflict with the law and thus void.

Jörg

-- 
 EMail:joerg@schily.isdn.cs.tu-berlin.de (home) Jörg Schilling D-13353 Berlin
       js@cs.tu-berlin.de                (uni)  
       joerg.schilling@fokus.fraunhofer.de (work) Blog: http://schily.blogspot.com/
 URL:  http://cdrecord.berlios.de/private/ ftp://ftp.berlios.de/pub/schily

Re: [gentoo-user] Optional /usr merge in Gentoo

[Thomas Mueller]

On the issue of whether ZFS can be shipped with the Linux kernel, FreeBSD includes ZFS with the kernel, binary and source.

So does that mean it would be OK for Linux too?

FreeBSD has a different license (BSD) than Linux (GPL 2 or 3).

I am not a lawyer!

Tom

Re: [gentoo-user] Optional /usr merge in Gentoo

[Alan McKinnon]

On 27/08/2013 04:04, Thomas Mueller wrote:
> On the issue of whether ZFS can be shipped with the Linux kernel, FreeBSD includes ZFS 
> with the kernel, binary and source.
> 
> So does that mean it would be OK for Linux too?

No.

> FreeBSD has a different license (BSD) than Linux (GPL 2 or 3).

Please read file COPYING in the kernel sources, the Linux kernel ships
with license GPL-2

Not a later version at your choice (2.x) and certainly never GPL-3

The issue is that the Linux kernel devs consider the license terms for
ZFS to be incompatible with GPL-2.0 and therefore ZFS cannot be
redistributed as a Linux kernel module.

There's nothing in the GPL-2 to stop you as a user from building and
running ZFS on Linux, as GPL does not interfere with your right to run
whatever you wish. The GPL only kicks in when code is redistributed.

The BSD license has none of these conditions, in layman terms that
license essentially says "you can take this code and pretty much do with
it whatever you want, we don't care"

> I am not a lawyer!
> 
> Tom
> 
> 





-- 
Alan McKinnon
alan.mckinnon@gmail.com

Re: [gentoo-user] Optional /usr merge in Gentoo

[Alan McKinnon]

On 26/08/2013 23:37, Joerg Schilling wrote:
> Neil Bothwick <neil@digimed.co.uk> wrote:
> 
>> On Mon, 26 Aug 2013 19:30:05 +0200, Joerg Schilling wrote:
>>
>>>> The licensing conflict means that would not be possible. You have the
>>>> install the kernel source and then merge in the ZFS source yourself,
>>>> it can't be done for you and distributed.  
>>>
>>> Why do you believe this?
>>>
>>> ZFS id doubtlessly an own "work" independent from the rest of the Linux
>>> kernel and for this reason, adding ZFS just creates a collective work
>>> that is not affected by the GPL.
>>
>> But the CCDL licence of ZFS precludes its being distributed with the
>> kernel. At least, that's how I understand it and the fact that no distro
>> distributes a ZFS-enabled kernel makes me believe it is true.
> 
> Did you ever read the CDDL?
> 
> People who believe that there is a problem use a wrong interpretation of the 
> GPL. The CDDL definitely does not prevent combinations with other software.

The problem is not with CDDL, the problem is with the GPL.

ZFS in the kernel requires that ZFS as shipped be relicensed as GPL, it
forms a derivative work of the kernel. No external license can change
the terms of the GPL.

Admittedly this gets murky due to XFS.

But the clincher would appear to be that Oracle own ZFS and also
distribute a branded RedHat derivative distro. To the best of my
knowledge Oracle themselves do not ship a ZFS-enabled kernel. Surely, as
the owners of the code and with a large dev team, Oracle themselves
could solve this issue by doing just that? But they haven't done so.

Especially as ZFS is production-ready today whereas the competing btrfs
is not.

-- 
Alan McKinnon
alan.mckinnon@gmail.com

Re: [gentoo-user] Optional /usr merge in Gentoo

[Joerg Schilling]

Thomas Mueller <mueller6726@bellsouth.net> wrote:

> On the issue of whether ZFS can be shipped with the Linux kernel, FreeBSD includes ZFS with the kernel, binary and source.
>
> So does that mean it would be OK for Linux too?
>
> FreeBSD has a different license (BSD) than Linux (GPL 2 or 3).

For FreeBSD, things are less easy than for Linux.

FreeBSD comes with a license that gives real freedom and the CDDL
being copyleft, is a license that intentionally limits the freedom a bit
in order to achieve other benefits.

The GPL limits freedom in a way far beyond what the CDDl does.
Adding code (ZFS) that gives more freedom than the base project (Linux) 
is easy...

It however was a real challenge for me to convince the FreeBSD people in early 
2006 to add something to their code that reduces the freedom of the FreeBSD
project. I succeeded because I could explain them that ZFS is not code that is 
_needed_ in order to run FreeBSD - you just could use their UFS variant instead.
The same arguments worked for integrating DTrace into FreeBSD.


Jörg

-- 
 EMail:joerg@schily.isdn.cs.tu-berlin.de (home) Jörg Schilling D-13353 Berlin
       js@cs.tu-berlin.de                (uni)  
       joerg.schilling@fokus.fraunhofer.de (work) Blog: http://schily.blogspot.com/
 URL:  http://cdrecord.berlios.de/private/ ftp://ftp.berlios.de/pub/schily

Re: [gentoo-user] Optional /usr merge in Gentoo

[Joerg Schilling]

Alan McKinnon <alan.mckinnon@gmail.com> wrote:

> The issue is that the Linux kernel devs consider the license terms for
> ZFS to be incompatible with GPL-2.0 and therefore ZFS cannot be
> redistributed as a Linux kernel module.

Isn't it strange that those people seem to have less problems with closed 
source than with a license that gives more freedom than the GPL? But 
you are correct that the problem seem to be humans and not a license text.

> There's nothing in the GPL-2 to stop you as a user from building and
> running ZFS on Linux, as GPL does not interfere with your right to run
> whatever you wish. The GPL only kicks in when code is redistributed.

There is nothing non-void in the GPL that stops you from distributing binaries.

Jörg

-- 
 EMail:joerg@schily.isdn.cs.tu-berlin.de (home) Jörg Schilling D-13353 Berlin
       js@cs.tu-berlin.de                (uni)  
       joerg.schilling@fokus.fraunhofer.de (work) Blog: http://schily.blogspot.com/
 URL:  http://cdrecord.berlios.de/private/ ftp://ftp.berlios.de/pub/schily

Re: [gentoo-user] Optional /usr merge in Gentoo

[Joerg Schilling]

Alan McKinnon <alan.mckinnon@gmail.com> wrote:

> > People who believe that there is a problem use a wrong interpretation of the 
> > GPL. The CDDL definitely does not prevent combinations with other software.
>
> The problem is not with CDDL, the problem is with the GPL.
>
> ZFS in the kernel requires that ZFS as shipped be relicensed as GPL, it
> forms a derivative work of the kernel. No external license can change
> the terms of the GPL.

The law can!

The GPL is in conflict with the law and therefore the parts you have in mind 
are just void.

BTW: I am still waiting for a legally acceptable explanation on why the GPL
should be compatible to the BSD license. Note that the BSD license is very 
liberal, but it definitely does not permit to relicense code that was published
under the BSD license withour written permission of the Copyright holder.

So is the problem just a social problem given the fact that Linux comes with 
BSD licensed parts?

Jörg

-- 
 EMail:joerg@schily.isdn.cs.tu-berlin.de (home) Jörg Schilling D-13353 Berlin
       js@cs.tu-berlin.de                (uni)  
       joerg.schilling@fokus.fraunhofer.de (work) Blog: http://schily.blogspot.com/
 URL:  http://cdrecord.berlios.de/private/ ftp://ftp.berlios.de/pub/schily

Re: [gentoo-user] Optional /usr merge in Gentoo

[Alan McKinnon]

On 27/08/2013 09:59, Joerg Schilling wrote:
> Alan McKinnon <alan.mckinnon@gmail.com> wrote:
> 
>>> People who believe that there is a problem use a wrong interpretation of the 
>>> GPL. The CDDL definitely does not prevent combinations with other software.
>>
>> The problem is not with CDDL, the problem is with the GPL.
>>
>> ZFS in the kernel requires that ZFS as shipped be relicensed as GPL, it
>> forms a derivative work of the kernel. No external license can change
>> the terms of the GPL.
> 
> The law can!
> 
> The GPL is in conflict with the law and therefore the parts you have in mind 
> are just void.

Which law is the GPL in conflict with, and in which jurisdiction, and
what is the extent of the conflict?

To the best of my knowledge, what you claim has not been tested in a
court of law with jurisdiction, and is not a matter of law. Until that
happens, it is an untested legal opinion and as we know, opinions can vary.

The kernel devs have their position, you have yours. In this case, the
opinion of the kernel devs is the one that carries as they control what
does and does not ship.


> 
> BTW: I am still waiting for a legally acceptable explanation on why the GPL
> should be compatible to the BSD license. Note that the BSD license is very 
> liberal, but it definitely does not permit to relicense code that was published
> under the BSD license withour written permission of the Copyright holder.

There is no requirement that the GPL should be compatible with the BSD
license. The GPL only requires that derivative works comply with the
terms of the GPL.

If BSD code is shipped with GPL code and the BSD code is the derivative
work, the BSD license does not demand that the code be published.
However, the GPL does so the entire codebase is published under the
terms of the GPL. Thus the conditions of both licenses are satisfied,
and no relicensing is involved.

> 
> So is the problem just a social problem given the fact that Linux comes with 
> BSD licensed parts?

I don't follow your reasoning here. How does the BSD license affect CDDL
code in this case?


> 
> Jörg
> 


-- 
Alan McKinnon
alan.mckinnon@gmail.com

Re: [gentoo-user] Optional /usr merge in Gentoo

[Alan McKinnon]

On 27/08/2013 09:53, Joerg Schilling wrote:
> Alan McKinnon <alan.mckinnon@gmail.com> wrote:
> 
>> The issue is that the Linux kernel devs consider the license terms for
>> ZFS to be incompatible with GPL-2.0 and therefore ZFS cannot be
>> redistributed as a Linux kernel module.
> 
> Isn't it strange that those people seem to have less problems with closed 
> source than with a license that gives more freedom than the GPL? But 
> you are correct that the problem seem to be humans and not a license text.

You are aware that the GPL was not really intended to be used together
with other licenses? It was really intended to create an entire
operating system, all of which was 100% licensed as GPL, all of which
comprise an original work written from scratch

Stallman never makes this claim as bluntly as I've said it here, but
it's the only intelligent reading of his intent as far as I can make
out. This is why so many arguments arise over the GPL, the wording of
that license was not really intended to have it co-exist with other
licenses.

That's how I see it anyway.

> 
>> There's nothing in the GPL-2 to stop you as a user from building and
>> running ZFS on Linux, as GPL does not interfere with your right to run
>> whatever you wish. The GPL only kicks in when code is redistributed.
> 
> There is nothing non-void in the GPL that stops you from distributing binaries.

That's a question of packaging and bundling, which is not covered by the
GPL. But kernel code and kernel modules are not mere bundles, they are
derivative works by virtue of how tightly they integrate with the
kernel, and how the code can only ever run unchanged on Linux.

That is how ZFS as a fuse module works, no license issues with the
kernel there at all.

-- 
Alan McKinnon
alan.mckinnon@gmail.com

Re: [gentoo-user] Optional /usr merge in Gentoo

[Joerg Schilling]

Alan McKinnon <alan.mckinnon@gmail.com> wrote:

> > The law can!
> > 
> > The GPL is in conflict with the law and therefore the parts you have in mind 
> > are just void.
>
> Which law is the GPL in conflict with, and in which jurisdiction, and
> what is the extent of the conflict?

The GPL is in conflict with US Copyright law Section 17 Paragraph 106.
In Europe, the law on business conditions apply and allow the licensee to 
chose his best interpretation in case of 

> To the best of my knowledge, what you claim has not been tested in a
> court of law with jurisdiction, and is not a matter of law. Until that
> happens, it is an untested legal opinion and as we know, opinions can vary.

There is no need to test something so obvious in court.
A license is not allowed to redefine the definition of what a derivative work 
is and the problem with the GPL only exists in case the GPL succeeds to redefine
the lawful definition of a drivative work.

> The kernel devs have their position, you have yours. In this case, the
> opinion of the kernel devs is the one that carries as they control what
> does and does not ship.

While I am quoting the papers from lawyers (Determann, Rosen, Gordon)
you are quoting laymen.

Note that Lothar Determan is professor of law at Freie Univerität Berlin _and_
the university of San Francisco.

>
> > 
> > BTW: I am still waiting for a legally acceptable explanation on why the GPL
> > should be compatible to the BSD license. Note that the BSD license is very 
> > liberal, but it definitely does not permit to relicense code that was published
> > under the BSD license withour written permission of the Copyright holder.
>
> There is no requirement that the GPL should be compatible with the BSD
> license. The GPL only requires that derivative works comply with the
> terms of the GPL.

The GPL requires to relicense the whole work under the GPL and this is not 
permitted for code under the BSD license.


> If BSD code is shipped with GPL code and the BSD code is the derivative
> work, the BSD license does not demand that the code be published.
> However, the GPL does so the entire codebase is published under the
> terms of the GPL. Thus the conditions of both licenses are satisfied,
> and no relicensing is involved.

If the Linux kernel uses the BSD code, it is the Linux kernel that has become 
the derivative work.

Note that you cannot publishe the entire codebase under GPL as parts are under 
BSD license already.

> > So is the problem just a social problem given the fact that Linux comes with 
> > BSD licensed parts?
>
> I don't follow your reasoning here. How does the BSD license affect CDDL
> code in this case?

It demonstrates that the Linux kernel people do not really honor the GPL and I 
see no difference between adding code under BSD compared to code under CDDL.
Both licenses do not allow relicensing without written permission of the 
Copyright owner.

Jörg

-- 
 EMail:joerg@schily.isdn.cs.tu-berlin.de (home) Jörg Schilling D-13353 Berlin
       js@cs.tu-berlin.de                (uni)  
       joerg.schilling@fokus.fraunhofer.de (work) Blog: http://schily.blogspot.com/
 URL:  http://cdrecord.berlios.de/private/ ftp://ftp.berlios.de/pub/schily

Re: [gentoo-user] Optional /usr merge in Gentoo

[Joerg Schilling]

Alan McKinnon <alan.mckinnon@gmail.com> wrote:

> > Isn't it strange that those people seem to have less problems with closed 
> > source than with a license that gives more freedom than the GPL? But 
> > you are correct that the problem seem to be humans and not a license text.
>
> You are aware that the GPL was not really intended to be used together
> with other licenses? It was really intended to create an entire
> operating system, all of which was 100% licensed as GPL, all of which
> comprise an original work written from scratch

But it has been proven that you cannot create a 100% GPL OS.
More than 50% of all Linux distros are under different licenses...

> Stallman never makes this claim as bluntly as I've said it here, but
> it's the only intelligent reading of his intent as far as I can make
> out. This is why so many arguments arise over the GPL, the wording of
> that license was not really intended to have it co-exist with other
> licenses.

Stallman does not look at reality. The first GCC version in 1986 has been 
published under something I call GPLv0 and this license did not permit a legal 
use of the GCC in public.

The license was later converted to GPLv1 by using proposals I made but 
Stallman still only talks about what has been in GPLv0.

> > There is nothing non-void in the GPL that stops you from distributing binaries.
>
> That's a question of packaging and bundling, which is not covered by the
> GPL. But kernel code and kernel modules are not mere bundles, they are
> derivative works by virtue of how tightly they integrate with the
> kernel, and how the code can only ever run unchanged on Linux.

If a kernel uses ZFS, you have to decide on whether the kernel is a derivative 
work of ZFS or whether just a collective work exists.

_Using_ ZFS definitely does not make ZFS a derivative work.

Jörg

-- 
 EMail:joerg@schily.isdn.cs.tu-berlin.de (home) Jörg Schilling D-13353 Berlin
       js@cs.tu-berlin.de                (uni)  
       joerg.schilling@fokus.fraunhofer.de (work) Blog: http://schily.blogspot.com/
 URL:  http://cdrecord.berlios.de/private/ ftp://ftp.berlios.de/pub/schily

Re: [gentoo-user] Optional /usr merge in Gentoo

[Joerg Schilling]

Joerg Schilling <Joerg.Schilling@fokus.fraunhofer.de> wrote:

> Alan McKinnon <alan.mckinnon@gmail.com> wrote:
>
> > > Isn't it strange that those people seem to have less problems with closed 
> > > source than with a license that gives more freedom than the GPL? But 
> > > you are correct that the problem seem to be humans and not a license text.
> >
> > You are aware that the GPL was not really intended to be used together
> > with other licenses? It was really intended to create an entire
> > operating system, all of which was 100% licensed as GPL, all of which
> > comprise an original work written from scratch
>
> But it has been proven that you cannot create a 100% GPL OS.
> More than 50% of all Linux distros are under different licenses...
>

Sorry, this should be: More than 50% of a typical Linux distro is 
under different licenses...

Jörg

-- 
 EMail:joerg@schily.isdn.cs.tu-berlin.de (home) Jörg Schilling D-13353 Berlin
       js@cs.tu-berlin.de                (uni)  
       joerg.schilling@fokus.fraunhofer.de (work) Blog: http://schily.blogspot.com/
 URL:  http://cdrecord.berlios.de/private/ ftp://ftp.berlios.de/pub/schily

Re: [gentoo-user] Optional /usr merge in Gentoo

[Tanstaafl]

Ummm... I didn't suggest that ZFS be shipped with or distributed with 
the kernel...

I was talking about some kind of overlay or patch system, where I could 
add zfs to my kernel use flag, and it would pull the gentoo-sources from 
wherver it pulls them, and pul;l the patch from a *separate*/*different* 
source/location, and then put the patch where it needs to go to be 
properly compiled into the kernel.

Again, the overlay would *not* contain or provide the kernel sources, 
only the zfs 'patch'.

I don't see a problem with that.

On 2013-08-26 10:04 PM, Thomas Mueller <mueller6726@bellsouth.net> wrote:
> On the issue of whether ZFS can be shipped with the Linux kernel, FreeBSD includes ZFS with the kernel, binary and source.
>
> So does that mean it would be OK for Linux too?
>
> FreeBSD has a different license (BSD) than Linux (GPL 2 or 3).
>
> I am not a lawyer!
>
> Tom
>
>

Re: [gentoo-user] Optional /usr merge in Gentoo

[Tanstaafl]

On 2013-08-26 2:23 AM, Alan McKinnon <alan.mckinnon@gmail.com> wrote:
> I run it on my NASes, and the thing that really sold me was what it lets
> me as the admin do:
>
> I get all the benefits of directories with none of the downsides.
> I get all the benefits of mount points with none of the downsides.
> I get all the benefits of discrete filesystems with none of the downsides.
>
> Like you say, a truly modern fs built for modern needs.

Are these home-built NAS's running FreeBSD (or maybe FreeNAS)? Or 
TrueNAS or Nexenta boxes?

I'm wondering what the best way would be to get something set up for ZFS 
file storage. I have some older servers that I can use, so was leaning 
toward FreeNAS...

Re: [gentoo-user] Optional /usr merge in Gentoo

[Alan McKinnon]

On 27/08/2013 13:36, Tanstaafl wrote:
> On 2013-08-26 2:23 AM, Alan McKinnon <alan.mckinnon@gmail.com> wrote:
>> I run it on my NASes, and the thing that really sold me was what it lets
>> me as the admin do:
>>
>> I get all the benefits of directories with none of the downsides.
>> I get all the benefits of mount points with none of the downsides.
>> I get all the benefits of discrete filesystems with none of the
>> downsides.
>>
>> Like you say, a truly modern fs built for modern needs.
> 
> Are these home-built NAS's running FreeBSD (or maybe FreeNAS)? Or
> TrueNAS or Nexenta boxes?
> 
> I'm wondering what the best way would be to get something set up for ZFS
> file storage. I have some older servers that I can use, so was leaning
> toward FreeNAS...
> 

Mine are HP mini-servers (the cube shaped ones) with 4 SATA bays running
FreeNAS 8.0.something.

Dunno if you've worked with FreeNAS before, but it's literally a case of
write the image to USB or flash storage and boot off it. Then play.

You will need to be able to boot off a USB stick, CF card or similar,
FreeNAS uses an entire drive for it's system partition and it's a shame
to waste a whole high-capacity disk just for a 2G system image



-- 
Alan McKinnon
alan.mckinnon@gmail.com

Re: [gentoo-user] Optional /usr merge in Gentoo

[Tanstaafl]

On 2013-08-27 7:42 AM, Alan McKinnon <alan.mckinnon@gmail.com> wrote:
> On 27/08/2013 13:36, Tanstaafl wrote:
>> I'm wondering what the best way would be to get something set up for ZFS
>> file storage. I have some older servers that I can use, so was leaning
>> toward FreeNAS...

> Mine are HP mini-servers (the cube shaped ones) with 4 SATA bays running
> FreeNAS 8.0.something.
>
> Dunno if you've worked with FreeNAS before, but it's literally a case of
> write the image to USB or flash storage and boot off it. Then play.
>
> You will need to be able to boot off a USB stick, CF card or similar,
> FreeNAS uses an entire drive for it's system partition and it's a shame
> to waste a whole high-capacity disk just for a 2G system image

I haven't worked with it before, but this comment of yours means I soon 
will be - thanks... :)

So, once I have something up and running and fully configured, it is 
relatively easy to backup the new/running system image, in case the 
flash drive ever crashes and burns?

Thanks Alan, starting to get excited about playing with ZFS.

How would you rate their docs and support community (for the free version)?

Thanks again Alan

Charles

Re: [gentoo-user] Optional /usr merge in Gentoo

[Neil Bothwick]

[-- Attachment #1: Type: text/plain, Size: 785 bytes --]

On Tue, 27 Aug 2013 06:33:52 -0400, Tanstaafl wrote:

> Ummm... I didn't suggest that ZFS be shipped with or distributed with 
> the kernel...
> 
> I was talking about some kind of overlay or patch system, where I could 
> add zfs to my kernel use flag, and it would pull the gentoo-sources
> from wherver it pulls them, and pul;l the patch from a
> *separate*/*different* source/location, and then put the patch where it
> needs to go to be properly compiled into the kernel.

I already posted the script I use to do exactly that.

emerge gentoo-sources
run the script

I wonder it it would be possible to have the spl and zfs-kmod ebuilds do
this with an appropriate USE flag.


-- 
Neil Bothwick

Bury a lawyer 12 feet under, because deep down they're nice.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: [gentoo-user] Optional /usr merge in Gentoo

[Tanstaafl]

On 2013-08-27 8:25 AM, Neil Bothwick <neil@digimed.co.uk> wrote:
> On Tue, 27 Aug 2013 06:33:52 -0400, Tanstaafl wrote:
>
>> Ummm... I didn't suggest that ZFS be shipped with or distributed with
>> the kernel...
>>
>> I was talking about some kind of overlay or patch system, where I could
>> add zfs to my kernel use flag, and it would pull the gentoo-sources
>> from wherver it pulls them, and pul;l the patch from a
>> *separate*/*different* source/location, and then put the patch where it
>> needs to go to be properly compiled into the kernel.
>
> I already posted the script I use to do exactly that.
>
> emerge gentoo-sources
> run the script
>
> I wonder it it would be possible to have the spl and zfs-kmod ebuilds do
> this with an appropriate USE flag.

Thats what I'm looking for... something that is automatic and basically 
'just works'.

Manually running a script as part of each kernel update just... well, 
computers do automation best.

But thanks very much for your script. I'm just not comfortable (at this 
point at least) doing it that way on a production system...

Re: [gentoo-user] Optional /usr merge in Gentoo

[Alan McKinnon]

On 27/08/2013 14:05, Tanstaafl wrote:
> On 2013-08-27 7:42 AM, Alan McKinnon <alan.mckinnon@gmail.com> wrote:
>> On 27/08/2013 13:36, Tanstaafl wrote:
>>> I'm wondering what the best way would be to get something set up for ZFS
>>> file storage. I have some older servers that I can use, so was leaning
>>> toward FreeNAS...
> 
>> Mine are HP mini-servers (the cube shaped ones) with 4 SATA bays running
>> FreeNAS 8.0.something.
>>
>> Dunno if you've worked with FreeNAS before, but it's literally a case of
>> write the image to USB or flash storage and boot off it. Then play.
>>
>> You will need to be able to boot off a USB stick, CF card or similar,
>> FreeNAS uses an entire drive for it's system partition and it's a shame
>> to waste a whole high-capacity disk just for a 2G system image
> 
> I haven't worked with it before, but this comment of yours means I soon
> will be - thanks... :)
> 
> So, once I have something up and running and fully configured, it is
> relatively easy to backup the new/running system image, in case the
> flash drive ever crashes and burns?

It's a small image (<100M compressed), so just keep a copy handy
somewhere and reflash. The GUI has a function where you can backup the
running config, a restore is a simple matter of click restore in the GUI

The USBstick/CF card you boot off will keep a copy of the current image
and one version back (i.e. the one the current one replaced), so you can
boot the old system by pressing F2 if the new one fails for some weird
reason.

Most of the config is GUI-driven in a browser, a lot but not all options
can be set on the CLI. But honestly, it's a file server and you will
find that once you set your shares up the way you like you will seldom
change stuff. Your main interaction will probably be watching the pretty
connectd graphs in a browser

For shares you get everything you could possibly need - cifs, nfs (2,3
and 4), iSCSI, FTP, scp, some Apple thing, and tftp and a few more. And
rsync!

> Thanks Alan, starting to get excited about playing with ZFS.
> 
> How would you rate their docs and support community (for the free version)?

Support is top-notch, on par with what you find around here if that
helps ;-)

Each major.minor version has a .pdf manual published, while the next
version is in development, the docs get updated on a wiki and the final
version is an export of that. There's a forum with knowledgeable users
and the devs hang around just in case regular users can't help with a
question.

No mailing list though :-(
And the forum does have a lot of noise from n00bs, but that's common
with web forums. Like on Gentoo, you quickly learn to spot those posts
and scan over them.

> 
> Thanks again Alan
> 
> Charles
> 


-- 
Alan McKinnon
alan.mckinnon@gmail.com

Re: [gentoo-user] Optional /usr merge in Gentoo

[Tanstaafl]

On 2013-08-27 9:03 AM, Alan McKinnon <alan.mckinnon@gmail.com> wrote:
> Each major.minor version has a .pdf manual published, while the next
> version is in development, the docs get updated on a wiki and the final
> version is an export of that. There's a forum with knowledgeable users
> and the devs hang around just in case regular users can't help with a
> question.

Ok, that brings up another issue...

One thing I've always loved about gentoo is it is a rolling release, 
which means no 'major update' pains to speak of (at least not like 
binary based distros like redhat etc)...

So, have you ever gone through any major system updates, and if so, any 
issues to speak of?

Thanks again for sharing this...

Re: [gentoo-user] Optional /usr merge in Gentoo

[Alan McKinnon]

On 27/08/2013 15:11, Tanstaafl wrote:
> On 2013-08-27 9:03 AM, Alan McKinnon <alan.mckinnon@gmail.com> wrote:
>> Each major.minor version has a .pdf manual published, while the next
>> version is in development, the docs get updated on a wiki and the final
>> version is an export of that. There's a forum with knowledgeable users
>> and the devs hang around just in case regular users can't help with a
>> question.
> 
> Ok, that brings up another issue...
> 
> One thing I've always loved about gentoo is it is a rolling release,
> which means no 'major update' pains to speak of (at least not like
> binary based distros like redhat etc)...
> 
> So, have you ever gone through any major system updates, and if so, any
> issues to speak of?
> 
> Thanks again for sharing this...
> 


No issues ever whatsoever. An upgrade is almost exactly the same as
upgrading firmware on your DSL router or reflashing OpenElec[1]. The
longest part is waiting for the NAS to reboot twice and get through
whatever your disk controller does at power up :-)

Once in the early days I had an incompatible database format for configs
and got a message at the start, so I had to do something manually to get
past that. But that was long ago. These days the migration script always
just dealt with it properly.


[1] another awesome project that JustWorks. I'm getting to like these
Unix-based appliances that JustWork. if I need to get under the overs
and tweak stuff, I can. Most mostly I don't need to :-)



-- 
Alan McKinnon
alan.mckinnon@gmail.com

Re: [gentoo-user] Optional /usr merge in Gentoo

[Neil Bothwick]

[-- Attachment #1: Type: text/plain, Size: 1148 bytes --]

On Tue, 27 Aug 2013 08:37:54 -0400, Tanstaafl wrote:

> > I already posted the script I use to do exactly that.
> >
> > emerge gentoo-sources
> > run the script
> >
> > I wonder it it would be possible to have the spl and zfs-kmod ebuilds
> > do this with an appropriate USE flag.  
> 
> Thats what I'm looking for... something that is automatic and basically 
> 'just works'.
> 
> Manually running a script as part of each kernel update just... well, 
> computers do automation best.

I use a script to configure, build and install new kernels. It's called
from there, so it is automatic for me :)

> But thanks very much for your script. I'm just not comfortable (at this 
> point at least) doing it that way on a production system...

That's the recommended way, since the script follows the instructions for
merging the modules in the kernel tree and uses the make scripts that
come with the sources. It will not mess up your kernel since it only adds
code, code that isn't even used until you enable it in the .config.


-- 
Neil Bothwick

MACINTOSH: Most Applications Crash; If Not, The Operating System Hangs

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: [gentoo-user] Optional /usr merge in Gentoo

[Tanstaafl]

On 2013-08-27 9:03 AM, Alan McKinnon <alan.mckinnon@gmail.com> wrote:
> It's a small image (<100M compressed), so just keep a copy handy
> somewhere and reflash. The GUI has a function where you can backup the
> running config, a restore is a simple matter of click restore in the GUI
>
> The USBstick/CF card you boot off will keep a copy of the current image
> and one version back (i.e. the one the current one replaced), so you can
> boot the old system by pressing F2 if the new one fails for some weird
> reason.

Crazy question...

Wondering of I could run this in a VM on my ESXi server?

Purpose would be threefold...

hosting windows user homes and roaming profiles

hosting alternate email storage for dovecot (for mail archival)

hosting email backups (rsync)

hmm.... maybe I could even make it primary mail storage?

Have to give this some thought...

Re: [gentoo-user] Optional /usr merge in Gentoo

[Alan McKinnon]

On 27/08/2013 17:55, Tanstaafl wrote:
> On 2013-08-27 9:03 AM, Alan McKinnon <alan.mckinnon@gmail.com> wrote:
>> It's a small image (<100M compressed), so just keep a copy handy
>> somewhere and reflash. The GUI has a function where you can backup the
>> running config, a restore is a simple matter of click restore in the GUI
>>
>> The USBstick/CF card you boot off will keep a copy of the current image
>> and one version back (i.e. the one the current one replaced), so you can
>> boot the old system by pressing F2 if the new one fails for some weird
>> reason.
> 
> Crazy question...
> 
> Wondering of I could run this in a VM on my ESXi server?
> 
> Purpose would be threefold...
> 
> hosting windows user homes and roaming profiles
> 
> hosting alternate email storage for dovecot (for mail archival)
> 
> hosting email backups (rsync)
> 
> hmm.... maybe I could even make it primary mail storage?
> 
> Have to give this some thought...
> 


Many people do just that (for testing and evaluation). ESXi lets you
present an image file as a boot device so that's sorted.

As always with VMs, IO performance is pretty sucky if you present
file-based storage to the guest. It's OK to evaluate and learn the
commands with, but for production you really want direct access to
proper storage devices. Just make sure your backend storage is NOT
itself doing RAID - ZFS doesn't play nicely with that. It really wants a
JBOD with no firmware interference.




-- 
Alan McKinnon
alan.mckinnon@gmail.com

Re: [gentoo-user] Optional /usr merge in Gentoo

[joost]

[-- Attachment #1: Type: text/plain, Size: 1942 bytes --]

Alan McKinnon <alan.mckinnon@gmail.com> wrote:
>On 27/08/2013 13:36, Tanstaafl wrote:
>> On 2013-08-26 2:23 AM, Alan McKinnon <alan.mckinnon@gmail.com> wrote:
>>> I run it on my NASes, and the thing that really sold me was what it
>lets
>>> me as the admin do:
>>>
>>> I get all the benefits of directories with none of the downsides.
>>> I get all the benefits of mount points with none of the downsides.
>>> I get all the benefits of discrete filesystems with none of the
>>> downsides.
>>>
>>> Like you say, a truly modern fs built for modern needs.
>> 
>> Are these home-built NAS's running FreeBSD (or maybe FreeNAS)? Or
>> TrueNAS or Nexenta boxes?
>> 
>> I'm wondering what the best way would be to get something set up for
>ZFS
>> file storage. I have some older servers that I can use, so was
>leaning
>> toward FreeNAS...
>> 
>
>Mine are HP mini-servers (the cube shaped ones) with 4 SATA bays
>running
>FreeNAS 8.0.something.
>
>Dunno if you've worked with FreeNAS before, but it's literally a case
>of
>write the image to USB or flash storage and boot off it. Then play.
>
>You will need to be able to boot off a USB stick, CF card or similar,
>FreeNAS uses an entire drive for it's system partition and it's a shame
>to waste a whole high-capacity disk just for a 2G system image
>
>
>
>-- 
>Alan McKinnon
>alan.mckinnon@gmail.com

Alan.

How is the security settings on the shares now?

I had issues when accessing through NFS and CIFS simultaneously where files written over NFS had to have the permissions altered before they were accessible over CIFS.

Other issue I had was inability to have users only being able to access files they were allowed to. With CIFS it sort of worked. But with NFS I had full access to all files.

That is the reason why I setup my NAS manually using Gentoo.

--
Joost
-- 
Sent from my Android phone with K-9 Mail. Please excuse my brevity.

[-- Attachment #2: Type: text/html, Size: 2498 bytes --]

Re: [gentoo-user] Optional /usr merge in Gentoo

[Alan McKinnon]

On 27/08/2013 21:24, joost@antarean.org wrote:
> Alan McKinnon <alan.mckinnon@gmail.com> wrote:
> 
>     On 27/08/2013 13:36, Tanstaafl wrote:
> 
>         On 2013-08-26 2:23 AM, Alan McKinnon <alan.mckinnon@gmail.com>
>         wrote:
> 
>             I run it on my NASes, and the thing that really sold me was
>             what it lets
>             me as the admin do:
> 
>             I get all the benefits of directories with none of the
>             downsides.
>             I get all the benefits of mount points with none of the
>             downsides.
>             I get all the benefits of discrete filesystems with none of the
>             downsides.
> 
>             Like you say, a truly modern fs built for modern needs.
> 
> 
>         Are these home-built NAS's running FreeBSD (or maybe FreeNAS)? Or
>         TrueNAS or Nexenta boxes?
> 
>         I'm wondering what the best way would be to get something set up
>         for ZFS
>         file storage. I have some older servers that I can use, so was
>         leaning
>         toward FreeNAS...
> 
> 
> 
>     Mine are HP mini-servers (the cube shaped ones) with 4 SATA bays running
>     FreeNAS 8.0.something.
> 
>     Dunno if you've worked with FreeNAS before, but it's literally a case of
>     write the image to USB or flash storage and boot off it. Then play.
> 
>     You will need to be able to boot off a USB stick, CF card or similar,
>     FreeNAS uses an entire drive for it's system partition and it's a shame
>     to waste a whole high-capacity disk just for a 2G system image
> 
> 
> 
> Alan.
> 
> How is the security settings on the shares now?
> 
> I had issues when accessing through NFS and CIFS simultaneously where
> files written over NFS had to have the permissions altered before they
> were accessible over CIFS.

I've never run into this situation myself, my shares are either accessed
via cfs or via nfs, but never both at the same time.

The permissions issue is an artifact of how NFS works. Sun designed it
to deliver entire filesystems over the network (most often /usr and-or
/home) to trusted clients. "trusted" being the operative word. To get
Unix permissions to work, the uid on the share and client have to match
- that's why we also have NIS - but I've never seen NIS actually used
anywhere, so UIDs tend to be a mix 'n match and almost always devolves
into "full access" to get it to work.

CIFS work different, it auths users by username and supports per-field
access control. That's how that protocol works.

There is no known way to fix NFS v2 & v3 in a mixed network and still
stay sane. NFS v4 does a good job but it's not NFS v3 :-)

it's common for NAS vendors to recommend you not try share the same
files over CIFS and NFS, especially if write access is involced.



> 
> Other issue I had was inability to have users only being able to access
> files they were allowed to. With CIFS it sort of worked. But with NFS I
> had full access to all files.
> 
> That is the reason why I setup my NAS manually using Gentoo.
> 
> --
> Joost
> -- 
> Sent from my Android phone with K-9 Mail. Please excuse my brevity.


-- 
Alan McKinnon
alan.mckinnon@gmail.com

Re: [gentoo-user] Optional /usr merge in Gentoo

[Alan McKinnon]

On 27/08/2013 11:08, Joerg Schilling wrote:
> Alan McKinnon <alan.mckinnon@gmail.com> wrote:
> 
>>> Isn't it strange that those people seem to have less problems with closed 
>>> source than with a license that gives more freedom than the GPL? But 
>>> you are correct that the problem seem to be humans and not a license text.
>>
>> You are aware that the GPL was not really intended to be used together
>> with other licenses? It was really intended to create an entire
>> operating system, all of which was 100% licensed as GPL, all of which
>> comprise an original work written from scratch
> 
> But it has been proven that you cannot create a 100% GPL OS.
> More than 50% of all Linux distros are under different licenses...
> 
>> Stallman never makes this claim as bluntly as I've said it here, but
>> it's the only intelligent reading of his intent as far as I can make
>> out. This is why so many arguments arise over the GPL, the wording of
>> that license was not really intended to have it co-exist with other
>> licenses.
> 
> Stallman does not look at reality. The first GCC version in 1986 has been 
> published under something I call GPLv0 and this license did not permit a legal 
> use of the GCC in public.
> 
> The license was later converted to GPLv1 by using proposals I made but 
> Stallman still only talks about what has been in GPLv0.


I didn't bring this up to discuss fine points of licenses. I brought it
up for those who might want to understand what the GPL is intended to
do; that can only be truly understood by determining what Stallman
intended. The GPL is a reflection of Stallman's intent, and can only be
truly understood in that light.

Whether the legal wording accurately matches his intent is another
matter altogether. I personally feel it doesn't, won't and cannot, for
reasons of psychology and philosophy, not for reasons of technology or
law. What the GPL tries to do and how it does it is quite foreign to
most who practice law. Humans don't like foreign concepts. Heck, GPL-2
doesn't even remotely read like something that came off a lawyer's desk.


> 
>>> There is nothing non-void in the GPL that stops you from distributing binaries.
>>
>> That's a question of packaging and bundling, which is not covered by the
>> GPL. But kernel code and kernel modules are not mere bundles, they are
>> derivative works by virtue of how tightly they integrate with the
>> kernel, and how the code can only ever run unchanged on Linux.
> 
> If a kernel uses ZFS, you have to decide on whether the kernel is a derivative 
> work of ZFS or whether just a collective work exists.
> 
> _Using_ ZFS definitely does not make ZFS a derivative work.

I never said it did. I was concentrating on those parts of ZFS that
interact with kernel internals - that might not be been entirely clear

You are making a spurious claim by saying "you have to decide on whether
the kernel is a derivative work of ZFS or ..."

In what possible way could the entire Linux kernel be considered a
derivative work of ZFS? That doesn't make any sense.



-- 
Alan McKinnon
alan.mckinnon@gmail.com

Re: [gentoo-user] Optional /usr merge in Gentoo

[Alan McKinnon]

On 27/08/2013 11:26, Joerg Schilling wrote:
> Joerg Schilling <Joerg.Schilling@fokus.fraunhofer.de> wrote:
> 
>> Alan McKinnon <alan.mckinnon@gmail.com> wrote:
>>
>>>> Isn't it strange that those people seem to have less problems with closed 
>>>> source than with a license that gives more freedom than the GPL? But 
>>>> you are correct that the problem seem to be humans and not a license text.
>>>
>>> You are aware that the GPL was not really intended to be used together
>>> with other licenses? It was really intended to create an entire
>>> operating system, all of which was 100% licensed as GPL, all of which
>>> comprise an original work written from scratch
>>
>> But it has been proven that you cannot create a 100% GPL OS.
>> More than 50% of all Linux distros are under different licenses...
>>
> 
> Sorry, this should be: More than 50% of a typical Linux distro is 
> under different licenses...


All we can state for sure is that no-one has yet created a fully 100%
GPL operating system. If you persuade FSF to relicense glibc to you as
GPL it *is* possible to do it for kernel and (a somewhat crippled)
userland. But not for firmware.

But this is beside the point, I was illustrating Stallman's intent, not
whether that intent could be realized or not.


-- 
Alan McKinnon
alan.mckinnon@gmail.com

Re: [gentoo-user] Optional /usr merge in Gentoo

[Joerg Schilling]

Alan McKinnon <alan.mckinnon@gmail.com> wrote:

> The permissions issue is an artifact of how NFS works. Sun designed it
> to deliver entire filesystems over the network (most often /usr and-or
> /home) to trusted clients. "trusted" being the operative word. To get
> Unix permissions to work, the uid on the share and client have to match
> - that's why we also have NIS - but I've never seen NIS actually used
> anywhere, so UIDs tend to be a mix 'n match and almost always devolves
> into "full access" to get it to work.

This is how NFS was designed before 1987, when Kerberos came up....
>
> CIFS work different, it auths users by username and supports per-field
> access control. That's how that protocol works.

This is how NFSv4 works.

BTW: as long as Linux does not support modern ACLs (originally defined by NTFS, 
now standardized by NFSv4) Linux will not be able to take advantage from CIFS 
ACLs.

Jörg

-- 
 EMail:joerg@schily.isdn.cs.tu-berlin.de (home) Jörg Schilling D-13353 Berlin
       js@cs.tu-berlin.de                (uni)  
       joerg.schilling@fokus.fraunhofer.de (work) Blog: http://schily.blogspot.com/
 URL:  http://cdrecord.berlios.de/private/ ftp://ftp.berlios.de/pub/schily

Re: [gentoo-user] Optional /usr merge in Gentoo

[Joerg Schilling]

Alan McKinnon <alan.mckinnon@gmail.com> wrote:

> >> That's a question of packaging and bundling, which is not covered by the
> >> GPL. But kernel code and kernel modules are not mere bundles, they are
> >> derivative works by virtue of how tightly they integrate with the
> >> kernel, and how the code can only ever run unchanged on Linux.
> > 
> > If a kernel uses ZFS, you have to decide on whether the kernel is a derivative 
> > work of ZFS or whether just a collective work exists.
> > 
> > _Using_ ZFS definitely does not make ZFS a derivative work.
>
> I never said it did. I was concentrating on those parts of ZFS that
> interact with kernel internals - that might not be been entirely clear

You wrote that modules become derivatives of the Linux kernel and this is the 
same as writing ZFS would become a kernel derivative.

The linux kernel does not come with a modern VFS implementation, so if you like 
to use ZFS on Linux you first need to provide a suitable VFS interface.
ZFS will not interact with the Linux kernel directly but with the expected VFS 
layer. Shouldn't it be possible to put this intermediate layer under a license 
that makes even the zealots happy?

> You are making a spurious claim by saying "you have to decide on whether
> the kernel is a derivative work of ZFS or ..."

If you go the non-lawful Stallman way and insist in a derivative work to be 
build, then the linux kernel is the derivative work. I prefer to assume that 
this just builds a collective work ;-)

> In what possible way could the entire Linux kernel be considered a
> derivative work of ZFS? That doesn't make any sense.

I am just quoting claims from Stallman ;-)

Jörg

-- 
 EMail:joerg@schily.isdn.cs.tu-berlin.de (home) Jörg Schilling D-13353 Berlin
       js@cs.tu-berlin.de                (uni)  
       joerg.schilling@fokus.fraunhofer.de (work) Blog: http://schily.blogspot.com/
 URL:  http://cdrecord.berlios.de/private/ ftp://ftp.berlios.de/pub/schily

Re: [gentoo-user] Optional /usr merge in Gentoo

[Pandu Poluan]

On Tue, Aug 27, 2013 at 8:03 PM, Alan McKinnon <alan.mckinnon@gmail.com> wrote:
>
> On 27/08/2013 14:05, Tanstaafl wrote:

[-- snippy --]

> > Thanks Alan, starting to get excited about playing with ZFS.
> >
> > How would you rate their docs and support community (for the free version)?
>
> Support is top-notch, on par with what you find around here if that
> helps ;-)
>
> Each major.minor version has a .pdf manual published, while the next
> version is in development, the docs get updated on a wiki and the final
> version is an export of that. There's a forum with knowledgeable users
> and the devs hang around just in case regular users can't help with a
> question.
>
> No mailing list though :-(
> And the forum does have a lot of noise from n00bs, but that's common
> with web forums. Like on Gentoo, you quickly learn to spot those posts
> and scan over them.
>

Actually, there *is* a mailing list. I happened upon it accidentally
several minutes ago.

Two of them in fact.

https://groups.google.com/a/zfsonlinux.org/forum/#!forum/zfs-discuss

... and if you want to partake in development of ZFS-on-Linux:

https://groups.google.com/a/zfsonlinux.org/forum/#!forum/zfs-devel


(I've just subscribed to the first list)


Rgds,
-- 
FdS Pandu E Poluan
~ IT Optimizer ~

 • LOPSA Member #15248
 • Blog : http://pepoluan.tumblr.com
 • Linked-In : http://id.linkedin.com/in/pepoluan

Re: [gentoo-user] Optional /usr merge in Gentoo

[Tanstaafl]

On 2013-08-27 5:06 PM, Joerg Schilling 
<Joerg.Schilling@fokus.fraunhofer.de> wrote:
> You wrote that modules become derivatives of the Linux kernel and this is the
> same as writing ZFS would become a kernel derivative.

Just for clarification, I was talking about compiling ZFS support INTO 
the kernel, not running it as a module.

Do you claim that support for compiling ZFS directly into the kernel 
also does not violate the license?

Re: [gentoo-user] Optional /usr merge in Gentoo

[Alan McKinnon]

On 28/08/2013 12:58, Tanstaafl wrote:
> On 2013-08-27 5:06 PM, Joerg Schilling
> <Joerg.Schilling@fokus.fraunhofer.de> wrote:
>> You wrote that modules become derivatives of the Linux kernel and this
>> is the
>> same as writing ZFS would become a kernel derivative.
> 
> Just for clarification, I was talking about compiling ZFS support INTO
> the kernel, not running it as a module.
> 
> Do you claim that support for compiling ZFS directly into the kernel
> also does not violate the license?
> 


Whether the code is compile in or a module makes no difference wrt
licenses as far as I know.

There's no limitation on *running* the code, you can fetch and patch and
edit and compile and run all you want and have it on as many of your (or
the company's) machines as you want - neither license interferes with
your right to do that.

You may not redistribute the code though.

A common misconception with these license is that they have something to
do with whether you may run the code or not. That is incorrect. Free
licenses are all about redistribution and your obligations about sharing
when you hand the code over to others.



-- 
Alan McKinnon
alan.mckinnon@gmail.com

Re: [gentoo-user] Optional /usr merge in Gentoo

[Joerg Schilling]

Tanstaafl <tanstaafl@libertytrek.org> wrote:

> On 2013-08-27 5:06 PM, Joerg Schilling 
> <Joerg.Schilling@fokus.fraunhofer.de> wrote:
> > You wrote that modules become derivatives of the Linux kernel and this is the
> > same as writing ZFS would become a kernel derivative.
>
> Just for clarification, I was talking about compiling ZFS support INTO 
> the kernel, not running it as a module.
>
> Do you claim that support for compiling ZFS directly into the kernel 
> also does not violate the license?

There is no difference, both is permitted.

Jörg

-- 
 EMail:joerg@schily.isdn.cs.tu-berlin.de (home) Jörg Schilling D-13353 Berlin
       js@cs.tu-berlin.de                (uni)  
       joerg.schilling@fokus.fraunhofer.de (work) Blog: http://schily.blogspot.com/
 URL:  http://cdrecord.berlios.de/private/ ftp://ftp.berlios.de/pub/schily

Re: [gentoo-user] Optional /usr merge in Gentoo

[Tanstaafl]

On 2013-08-28 7:12 AM, Alan McKinnon <alan.mckinnon@gmail.com> wrote:
> Whether the code is compile in or a module makes no difference wrt
> licenses as far as I know.
>
> There's no limitation on*running*  the code, you can fetch and patch and
> edit and compile and run all you want and have it on as many of your (or
> the company's) machines as you want - neither license interferes with
> your right to do that.
>
> You may not redistribute the code though.

So, can you answer me this...

Why would there be a problem if someone decided to create a 3rd party 
overlay *not* part of the official gentoo portage tree that contained 
*only* the zfs stuff, and when this overlay was installed combined with 
a zfs keyword for the kernel, portage would then pull in the required 
files, and automagically build a kernel with an up to date version of 
zfs properly and fully integrated?

Would this not work, *and* have no problems with licensing?

Re: [gentoo-user] Optional /usr merge in Gentoo

[Alan McKinnon]

On 30/08/2013 16:29, Tanstaafl wrote:
> On 2013-08-28 7:12 AM, Alan McKinnon <alan.mckinnon@gmail.com> wrote:
>> Whether the code is compile in or a module makes no difference wrt
>> licenses as far as I know.
>>
>> There's no limitation on*running*  the code, you can fetch and patch and
>> edit and compile and run all you want and have it on as many of your (or
>> the company's) machines as you want - neither license interferes with
>> your right to do that.
>>
>> You may not redistribute the code though.
> 
> So, can you answer me this...
> 
> Why would there be a problem if someone decided to create a 3rd party
> overlay *not* part of the official gentoo portage tree that contained
> *only* the zfs stuff, and when this overlay was installed combined with
> a zfs keyword for the kernel, portage would then pull in the required
> files, and automagically build a kernel with an up to date version of
> zfs properly and fully integrated?
> 
> Would this not work, *and* have no problems with licensing?
> 

there is no problem with licensing in that case.
The ebuild could even go in the portage tree, as Gentoo is not
redistributing sources when it publishes an ebuild.

-- 
Alan McKinnon
alan.mckinnon@gmail.com

Re: Integrated ZFS for Gentoo - WAS Re: [gentoo-user] Optional /usr merge in Gentoo

[Gregory Shearman]

In linux.gentoo.user, Mr Schilling wrote:
>
> On Solaris, you can disable loading unsigned modules,  is this not supported by 
> Linux?

CONFIG_MODULE_SIG

-- 
Regards,
Gregory.

Re: Integrated ZFS for Gentoo - WAS Re: [gentoo-user] Optional /usr merge in Gentoo

[Joerg Schilling]

Gregory Shearman <zekeyg@gmail.com> wrote:

> In linux.gentoo.user, Mr Schilling wrote:
> >
> > On Solaris, you can disable loading unsigned modules,  is this not supported by 
> > Linux?
>
> CONFIG_MODULE_SIG

So there seems to be no real need to create a static linux kernel with ZFS 
inside.

Jörg

-- 
 EMail:joerg@schily.isdn.cs.tu-berlin.de (home) Jörg Schilling D-13353 Berlin
       js@cs.tu-berlin.de                (uni)  
       joerg.schilling@fokus.fraunhofer.de (work) Blog: http://schily.blogspot.com/
 URL:  http://cdrecord.berlios.de/private/ ftp://ftp.berlios.de/pub/schily

Re: Integrated ZFS for Gentoo - WAS Re: [gentoo-user] Optional /usr merge in Gentoo

[Walter Dnes]

On Sat, Aug 31, 2013 at 02:19:56PM +0200, Joerg Schilling wrote

> So there seems to be no real need to create a static linux kernel
> with ZFS inside.

See http://www.gentoo.org/doc/en/handbook/handbook-amd64.xml?full=1#book_part1_chap7

> Now go to File Systems and select support for the filesystems you use.
> Don't compile the file system you use for the root filesystem as
> module, otherwise your Gentoo system will not be able to mount
> your partition.

  You can get away with most stuff as modules; ***BUT NOT THE ROOT
FILESYSTEM***.  Think about it for a minute.  Gentoo reads modules off
the disk.  If the code for the root filesystem is a module, Gentoo would
have to read the module off the disk to enable it to read the module off
the disk... OOPS.  This is a classic "chicken and egg" situation.

-- 
Walter Dnes <waltdnes@waltdnes.org>
I don't run "desktop environments"; I run useful applications

Re: Integrated ZFS for Gentoo - WAS Re: [gentoo-user] Optional /usr merge in Gentoo

[Canek Peláez Valdés]

On Sat, Aug 31, 2013 at 7:13 PM, Walter Dnes <waltdnes@waltdnes.org> wrote:
> On Sat, Aug 31, 2013 at 02:19:56PM +0200, Joerg Schilling wrote
>
>> So there seems to be no real need to create a static linux kernel
>> with ZFS inside.
>
> See http://www.gentoo.org/doc/en/handbook/handbook-amd64.xml?full=1#book_part1_chap7
>
>> Now go to File Systems and select support for the filesystems you use.
>> Don't compile the file system you use for the root filesystem as
>> module, otherwise your Gentoo system will not be able to mount
>> your partition.
>
>   You can get away with most stuff as modules; ***BUT NOT THE ROOT
> FILESYSTEM***.  Think about it for a minute.  Gentoo reads modules off
> the disk.  If the code for the root filesystem is a module, Gentoo would
> have to read the module off the disk to enable it to read the module off
> the disk... OOPS.  This is a classic "chicken and egg" situation.

I usally use ext4 as filesystem.

# lsmod|grep ext
ext3                  100768  0
jbd                    39586  1 ext3
ext2                   49572  0
ext4                  263621  1
crc16                   1255  2 ext4,bluetooth
mbcache                 4450  3 ext2,ext3,ext4
jbd2                   48679  1 ext4

Isn't great what an initramfs can do?

Regards.
-- 
Canek Peláez Valdés
Posgrado en Ciencia e Ingeniería de la Computación
Universidad Nacional Autónoma de México

Re: Integrated ZFS for Gentoo - WAS Re: [gentoo-user] Optional /usr merge in Gentoo

[Mark David Dumlao]

On Sun, Sep 1, 2013 at 8:13 AM, Walter Dnes <waltdnes@waltdnes.org> wrote:
> On Sat, Aug 31, 2013 at 02:19:56PM +0200, Joerg Schilling wrote
>
>> So there seems to be no real need to create a static linux kernel
>> with ZFS inside.
>
> See http://www.gentoo.org/doc/en/handbook/handbook-amd64.xml?full=1#book_part1_chap7
>
>> Now go to File Systems and select support for the filesystems you use.
>> Don't compile the file system you use for the root filesystem as
>> module, otherwise your Gentoo system will not be able to mount
>> your partition.
>
>   You can get away with most stuff as modules; ***BUT NOT THE ROOT
> FILESYSTEM***.  Think about it for a minute.  Gentoo reads modules off
> the disk.  If the code for the root filesystem is a module, Gentoo would
> have to read the module off the disk to enable it to read the module off
> the disk... OOPS.  This is a classic "chicken and egg" situation.

And this is why the initrd was actually invented.
http://en.wikipedia.org/wiki/Initrd

It's a means of loading kernel modules so that the root filesystem can be
mounted as a module.
-- 
This email is:    [ ] actionable   [x] fyi        [ ] social
Response needed:  [ ] yes          [ ] up to you  [x] no
Time-sensitive:   [ ] immediate    [ ] soon       [x] none

Re: Integrated ZFS for Gentoo - WAS Re: [gentoo-user] Optional /usr merge in Gentoo

[Pandu Poluan]

[-- Attachment #1: Type: text/plain, Size: 1483 bytes --]

On Sep 1, 2013 7:51 AM, "Mark David Dumlao" <madumlao@gmail.com> wrote:
>
> On Sun, Sep 1, 2013 at 8:13 AM, Walter Dnes <waltdnes@waltdnes.org> wrote:
> > On Sat, Aug 31, 2013 at 02:19:56PM +0200, Joerg Schilling wrote
> >
> >> So there seems to be no real need to create a static linux kernel
> >> with ZFS inside.
> >
> > See
http://www.gentoo.org/doc/en/handbook/handbook-amd64.xml?full=1#book_part1_chap7
> >
> >> Now go to File Systems and select support for the filesystems you use.
> >> Don't compile the file system you use for the root filesystem as
> >> module, otherwise your Gentoo system will not be able to mount
> >> your partition.
> >
> >   You can get away with most stuff as modules; ***BUT NOT THE ROOT
> > FILESYSTEM***.  Think about it for a minute.  Gentoo reads modules off
> > the disk.  If the code for the root filesystem is a module, Gentoo would
> > have to read the module off the disk to enable it to read the module off
> > the disk... OOPS.  This is a classic "chicken and egg" situation.
>
> And this is why the initrd was actually invented.
> http://en.wikipedia.org/wiki/Initrd
>
> It's a means of loading kernel modules so that the root filesystem can be
> mounted as a module.

Not everyone is willing to use an initr* thingy. It's another potential
breaking point.

I have no problem with /usr being 'merged' with /, in fact I have been
doing that for a couple of years now.

But I will keep myself a mile away from an initr* thingy.

Rgds,
--

[-- Attachment #2: Type: text/html, Size: 2140 bytes --]

Re: Integrated ZFS for Gentoo - WAS Re: [gentoo-user] Optional /usr merge in Gentoo

[Walter Dnes]

> I usally use ext4 as filesystem.
> 
> # lsmod|grep ext
> ext3                  100768  0
> jbd                    39586  1 ext3
> ext2                   49572  0
> ext4                  263621  1
> crc16                   1255  2 ext4,bluetooth
> mbcache                 4450  3 ext2,ext3,ext4
> jbd2                   48679  1 ext4
> 
> Isn't great what an initramfs can do?

  In this case, initramfs is your root filesystem, from which you load
another fs and then transfer (pivot root?) to it.  You have to build
initramfs support into the kernel, to boot an initramfs.  So my argument
still stands, regardless of whether your *INITIAL* filesystem is ext4fs,
or ZFS, or initramfs, that *INITIAL* filesystem has to be built into the
kernel.  Also, I really wonder what the point is in having to use
initramfs on a system where /usr is part of /.

-- 
Walter Dnes <waltdnes@waltdnes.org>
I don't run "desktop environments"; I run useful applications

Re: Integrated ZFS for Gentoo - WAS Re: [gentoo-user] Optional /usr merge in Gentoo

[Canek Peláez Valdés]

On Sat, Aug 31, 2013 at 10:55 PM, Walter Dnes <waltdnes@waltdnes.org> wrote:
>> I usally use ext4 as filesystem.
>>
>> # lsmod|grep ext
>> ext3                  100768  0
>> jbd                    39586  1 ext3
>> ext2                   49572  0
>> ext4                  263621  1
>> crc16                   1255  2 ext4,bluetooth
>> mbcache                 4450  3 ext2,ext3,ext4
>> jbd2                   48679  1 ext4
>>
>> Isn't great what an initramfs can do?
>
>   In this case, initramfs is your root filesystem, from which you load
> another fs and then transfer (pivot root?) to it.  You have to build
> initramfs support into the kernel, to boot an initramfs.  So my argument
> still stands, regardless of whether your *INITIAL* filesystem is ext4fs,
> or ZFS, or initramfs, that *INITIAL* filesystem has to be built into the
> kernel.

Interesting perspective. Of course, support for an initramfs is not
actually a file system (it's not even in the File systems section of
the kernel configuration, is in General setup); it's not possible to
have initramfs as a module (that would make no sense at all); and it's
code that is several orders of magnitude more simpler than the one
used by ext4 (or any other journal file system).

But you are right that for booting with an initramfs, you need
initramfs support.

> Also, I really wonder what the point is in having to use
> initramfs on a system where /usr is part of /.

Well, since some months ago I've been running as a module almost
everything that can be compiled as a module. This allows me to run a
*truly* minimal kernel, and only the necessary modules autoload
automatically (one big exception: binfmt_script, I compiled that into
the kernel because it was not loading automatically). I can also
unload some modules when not in use anymore (and this is great to
debug sometimes).

This also lets me to add a lot of stuff in the kernel, as long as I
add them as modules, without me worrying about bloating my kernel.
Only when they are needed they are loaded. I have USB speakers, but I
almost never use them; no problem, they (like almost everything else)
live as modules, and only are loaded (automagically, thanks to udev)
when needed. And again, I can unload them when not in use.

And also, it turns out that by using dracut+systemd you could boot
faster than without initramfs (although I can't find the link
anymore).

Finally, using only modules and dracut liberates me from thinking what
should it be compiled in and what not; I just put *everything* as a
module, and the kernel, udev and dracut take care of loading what's
necessary. Thus, my kernel (the one running in memory) is as minimal
as it can be, all the time.

Oh, and one more thing; by having everything as a module, if suddenly
I need support for new hardware, usually I can do a quick "make
menuconfig; make modules_install", and the new module can be
modprobe'd into the kernel without needing a reboot. That's
convenient.

Regards.
-- 
Canek Peláez Valdés
Posgrado en Ciencia e Ingeniería de la Computación
Universidad Nacional Autónoma de México

Re: Integrated ZFS for Gentoo - WAS Re: [gentoo-user] Optional /usr merge in Gentoo

[Mark David Dumlao]

On Sun, Sep 1, 2013 at 11:55 AM, Walter Dnes <waltdnes@waltdnes.org> wrote:
>> I usally use ext4 as filesystem.
>>
>> # lsmod|grep ext
>> ext3                  100768  0
>> jbd                    39586  1 ext3
>> ext2                   49572  0
>> ext4                  263621  1
>> crc16                   1255  2 ext4,bluetooth
>> mbcache                 4450  3 ext2,ext3,ext4
>> jbd2                   48679  1 ext4
>>
>> Isn't great what an initramfs can do?
>
>   In this case, initramfs is your root filesystem, from which you load
> another fs and then transfer (pivot root?) to it.  You have to build
> initramfs support into the kernel, to boot an initramfs.  So my argument
> still stands, regardless of whether your *INITIAL* filesystem is ext4fs,
> or ZFS, or initramfs, that *INITIAL* filesystem has to be built into the
> kernel.  Also, I really wonder what the point is in having to use
> initramfs on a system where /usr is part of /.

It allows you to keep some kernel bits in modules. If ever you change your mind
on whether to include / exclude / reconfigure those kernel bits in the
future, your
kernel recompile will take a lot, lot, shorter.

Case in point - do you enable all the ext4 options, like acls and
whatnot? Let's say no.

What if you suddenly have to mount an external hard disk to recover some system
on your server and the hard disk uses those ext4 options? If ext4 is
hard built into your
kernel, your recompile will have to basically redo the whole thing,
whereas if ext4
was a module you would only recompile ext4 itself.
-- 
This email is:    [ ] actionable   [x] fyi        [ ] social
Response needed:  [ ] yes          [ ] up to you  [x] no
Time-sensitive:   [ ] immediate    [ ] soon       [x] none

Re: Integrated ZFS for Gentoo - WAS Re: [gentoo-user] Optional /usr merge in Gentoo

[Joerg Schilling]

Walter Dnes <waltdnes@waltdnes.org> wrote:

>   You can get away with most stuff as modules; ***BUT NOT THE ROOT
> FILESYSTEM***.  Think about it for a minute.  Gentoo reads modules off
> the disk.  If the code for the root filesystem is a module, Gentoo would
> have to read the module off the disk to enable it to read the module off
> the disk... OOPS.  This is a classic "chicken and egg" situation.

On Solaris no problem with loadable modules - everything is dynamically loaded.
You need a grub that understands ZFS and that gives a ZFS interface to the 
kernel to use before ZFS was loaded.

Jörg

-- 
 EMail:joerg@schily.isdn.cs.tu-berlin.de (home) Jörg Schilling D-13353 Berlin
       js@cs.tu-berlin.de                (uni)  
       joerg.schilling@fokus.fraunhofer.de (work) Blog: http://schily.blogspot.com/
 URL:  http://cdrecord.berlios.de/private/ ftp://ftp.berlios.de/pub/schily

Re: Integrated ZFS for Gentoo - WAS Re: [gentoo-user] Optional /usr merge in Gentoo

[Tanstaafl]

On 2013-08-31 11:55 PM, Walter Dnes <waltdnes@waltdnes.org> wrote:
> Also, I really wonder what the point is in having to use
> initramfs on a system where /usr is part of /.

You don't, it is only *required* if you have a separate /usr... in fact 
that is what the whole argument was about.

At least that is my understanding of the situation now... please don't 
tell me I'm wrong and there was another vote and it is now required just 
to be able to use gentoo?

Re: [gentoo-user] Optional /usr merge in Gentoo

[Tanstaafl]

On 2013-09-01 12:31 AM, Canek Peláez Valdés <caneko@gmail.com> wrote:
> Of course, support for an initramfs is not actually a file system
> (it's not even in the File systems section of the kernel
> configuration, is in General setup); it's not possible to have
> initramfs as a module (that would make no sense at all); and it's
> code that is several orders of magnitude more simpler than the one
> used by ext4 (or any other journal file system).

Is there any reason that the creation, use and maintenance of the 
initramfs couldn't be as simple as a checkbox in the kernel config, so 
that running 'make' after the kernel was configured would automatically 
build it? Then, all I'd have to do is move it into /boot along with the 
new kernel (just like I do now), with *nothing* else required, and the 
kernel would call it, and things would just work (as long as it was 
there and I didn't forget to copy it to /boot).

Re: [gentoo-user] Optional /usr merge in Gentoo

[Alan McKinnon]

On 01/09/2013 16:30, Tanstaafl wrote:
> On 2013-09-01 12:31 AM, Canek Peláez Valdés <caneko@gmail.com> wrote:
>> Of course, support for an initramfs is not actually a file system
>> (it's not even in the File systems section of the kernel
>> configuration, is in General setup); it's not possible to have
>> initramfs as a module (that would make no sense at all); and it's
>> code that is several orders of magnitude more simpler than the one
>> used by ext4 (or any other journal file system).
> 
> Is there any reason that the creation, use and maintenance of the
> initramfs couldn't be as simple as a checkbox in the kernel config, so
> that running 'make' after the kernel was configured would automatically
> build it? Then, all I'd have to do is move it into /boot along with the
> new kernel (just like I do now), with *nothing* else required, and the
> kernel would call it, and things would just work (as long as it was
> there and I didn't forget to copy it to /boot).


That would require a config file of some sort to define what files you
want in the initramfs, and it must be available to the kernel build
process. It also has to read your self-defined arbitrary stuff from your
userland.

The kernel build machinery is a self-contained environment, the kernel
devs work very hard to keep userland out of it. So expect Linux to shoot
you down in flames for the very suggestion.

You keep asking for tools to automate the production of an initramfs;
you should realize that the thing has got absolutely nothing to do with
building and running a kernel, it's a helper function, and not really
tied to the kernel per se.

Just rig your kernel update process to add a section where you run the
command that builds an initramfs. You already have so many steps where
you do exactly that in other areas so it's not a realistic issue, and
you take that in your stride. Or at it to the end of your kernel build
wrapper script if you wrote such a thing for yourself.


-- 
Alan McKinnon
alan.mckinnon@gmail.com

Re: Integrated ZFS for Gentoo - WAS Re: [gentoo-user] Optional /usr merge in Gentoo

[Walter Dnes]

On Sun, Sep 01, 2013 at 09:49:23AM +0200, Joerg Schilling wrote
> Walter Dnes <waltdnes@waltdnes.org> wrote:
> 
> >   You can get away with most stuff as modules; ***BUT NOT THE ROOT
> > FILESYSTEM***.  Think about it for a minute.  Gentoo reads modules off
> > the disk.  If the code for the root filesystem is a module, Gentoo would
> > have to read the module off the disk to enable it to read the module off
> > the disk... OOPS.  This is a classic "chicken and egg" situation.
> 
> On Solaris no problem with loadable modules - everything is
> dynamically loaded.  ***YOU NEED A GRUB THAT UNDERSTANDS ZFS AND THAT
> GIVES A ZFS INTERFACE TO THE KERNEL TO USE BEFORE ZFS WAS LOADED***.

  So instead of needing ZFS built into the kernel, you need ZFS built
into GRUB... ***AND*** you need a ZFS module for the main system...
***AND*** you need to keep both versions in sync.  I'm not impressed.

-- 
Walter Dnes <waltdnes@waltdnes.org>
I don't run "desktop environments"; I run useful applications

Re: Integrated ZFS for Gentoo - WAS Re: [gentoo-user] Optional /usr merge in Gentoo

[Walter Dnes]

On Sun, Sep 01, 2013 at 10:11:01AM -0400, Tanstaafl wrote

> You don't, it is only *required* if you have a separate /usr... in fact 
> that is what the whole argument was about.
> 
> At least that is my understanding of the situation now... please don't 
> tell me I'm wrong and there was another vote and it is now required just 
> to be able to use gentoo?

  This is for the people who want *EVERYTHING INCLUDING THE ROOT FILE
SYSTEM CODE* built as a module.  Note that the Gentoo (AMD64) docs at
http://www.gentoo.org/doc/en/handbook/handbook-amd64.xml?full=1#book_part1_chap7say...
> Don't compile the file system you use for the root filesystem as
> module, otherwise your Gentoo system will not be able to mount
> your partition.

  Using an initramfs allows you to ignore that warning.

-- 
Walter Dnes <waltdnes@waltdnes.org>
I don't run "desktop environments"; I run useful applications

Re: Integrated ZFS for Gentoo - WAS Re: [gentoo-user] Optional /usr merge in Gentoo

[Mark David Dumlao]

[-- Attachment #1: Type: text/plain, Size: 1155 bytes --]

On Sep 2, 2013 5:21 AM, "Walter Dnes" <waltdnes@waltdnes.org> wrote:
>
> On Sun, Sep 01, 2013 at 09:49:23AM +0200, Joerg Schilling wrote
> > Walter Dnes <waltdnes@waltdnes.org> wrote:
> >
> > >   You can get away with most stuff as modules; ***BUT NOT THE ROOT
> > > FILESYSTEM***.  Think about it for a minute.  Gentoo reads modules off
> > > the disk.  If the code for the root filesystem is a module, Gentoo
would
> > > have to read the module off the disk to enable it to read the module
off
> > > the disk... OOPS.  This is a classic "chicken and egg" situation.
> >
> > On Solaris no problem with loadable modules - everything is
> > dynamically loaded.  ***YOU NEED A GRUB THAT UNDERSTANDS ZFS AND THAT
> > GIVES A ZFS INTERFACE TO THE KERNEL TO USE BEFORE ZFS WAS LOADED***.

I'm confused as to what this means. Grub reads a filesystem, loads a kernel
with options, and may give it an initrd. What happens from then on is none
of grub's business. The filesystem it reads from and the one the kernel
uses may be completely unrelated - this is why we have /boot filesystems.

At what point does grub "present a zfs interface for the kernel to use"?

[-- Attachment #2: Type: text/html, Size: 1479 bytes --]

Re: Integrated ZFS for Gentoo - WAS Re: [gentoo-user] Optional /usr merge in Gentoo

[Walter Dnes]

On Sun, Sep 01, 2013 at 01:41:30PM +0800, Mark David Dumlao wrote

> Case in point - do you enable all the ext4 options, like acls and
> whatnot? Let's say no.
> 
> What if you suddenly have to mount an external hard disk to
> recover some system on your server and the hard disk uses those ext4
> options? If ext4 is hard built into your kernel, your recompile will
> have to basically redo the whole thing, whereas if ext4 was a module
> you would only recompile ext4 itself.

  Have you ever actually done this?  I'd be very leery of pulling such a
stunt.  The clean way of switching module versions is to...
* unload the old module, and
* load the new module

  You obviously can't do this in your setup, because unloading the old
module would mean you could no longer access the file system to read in
the new module... OOPS.

  You could run a script that creates /dev/shm/lib/3.1.4.1.5.9-gentoo/
(easy as pie<G>) and copies the new module to that dir.  Then unload the
old module and load the new one, using modprobe with "-d /dev/shm/".

  That still looks impossible.  The problem is that you generally have a
whole bunch of files open at any time.  E.g. try...

lsof -d txt | grep -v "/proc/" | less

...and look at the output.  Shutting down all those open files would
be disastrous.  But that's not what you're saying.  You seem to imply
that file system code can be overwritten *IN PLACE, WHILE IN USE*,
without any problems.  Colour me skeptical about that one.

-- 
Walter Dnes <waltdnes@waltdnes.org>
I don't run "desktop environments"; I run useful applications

Re: Integrated ZFS for Gentoo - WAS Re: [gentoo-user] Optional /usr merge in Gentoo

[Joerg Schilling]

Mark David Dumlao <madumlao@gmail.com> wrote:

> > > > the disk... OOPS.  This is a classic "chicken and egg" situation.
> > >
> > > On Solaris no problem with loadable modules - everything is
> > > dynamically loaded.  ***YOU NEED A GRUB THAT UNDERSTANDS ZFS AND THAT
> > > GIVES A ZFS INTERFACE TO THE KERNEL TO USE BEFORE ZFS WAS LOADED***.
>
> I'm confused as to what this means. Grub reads a filesystem, loads a kernel
> with options, and may give it an initrd. What happens from then on is none
> of grub's business. The filesystem it reads from and the one the kernel
> uses may be completely unrelated - this is why we have /boot filesystems.
>
> At what point does grub "present a zfs interface for the kernel to use"?

After it booted the kernel

You may not know dynamic kernels as Linux is a static kernel that just may load 
additional modules _after_ it mounted the root fs.

Solaris is dynamic from the beginning:

-	no static loading at all

-	no predefined data sizes - everything is allocated

-	no predefined major device numbers - numbers are assigned at first load

Grub works this way:

1)	It loads /platform/i86pc/kernel/$ISADIR/unix

2)	It checks the file "unix" and sees ELF dependencies.

	It loads the ELF dependencies (genunix and dtracestubs) listed
	in the ELF headers from "unix".

3)	It loads /platform/i86pc/$ISADIR/boot_archive

The Kernel then uses the filesystem callbacks in grub to load modules from the 
filesystem in the boot archive.

After the kernel did mount the root filesystem, it switches to the normal 
kernel drivers just loaded and frees the memory space used by grub before.

Jörg

-- 
 EMail:joerg@schily.isdn.cs.tu-berlin.de (home) Jörg Schilling D-13353 Berlin
       js@cs.tu-berlin.de                (uni)  
       joerg.schilling@fokus.fraunhofer.de (work) Blog: http://schily.blogspot.com/
 URL:  http://cdrecord.berlios.de/private/ ftp://ftp.berlios.de/pub/schily

Re: [gentoo-user] Optional /usr merge in Gentoo

[Douglas J Hunley]

[-- Attachment #1: Type: text/plain, Size: 1095 bytes --]

On Sun, Sep 1, 2013 at 10:30 AM, Tanstaafl <tanstaafl@libertytrek.org>wrote:

> Is there any reason that the creation, use and maintenance of the
> initramfs couldn't be as simple as a checkbox in the kernel config, so that
> running 'make' after the kernel was configured would automatically build
> it? Then, all I'd have to do is move it into /boot along with the new
> kernel (just like I do now), with *nothing* else required, and the kernel
> would call it, and things would just work (as long as it was there and I
> didn't forget to copy it to /boot).


This exists. You can built initramfs right into the kernel. I've been doing
it here for quite some time. You just tell the kernel either:
* where to find a filespec so it knows what to include in the initramfs
* what directory contains everything you want in the initramfs

and then the kernel builds is and attaches it to itself during 'make'

It's actually pretty trivial


-- 
Douglas J Hunley (doug.hunley@gmail.com)
Twitter: @hunleyd                                               Web:
douglasjhunley.com
G+: http://goo.gl/sajR3

[-- Attachment #2: Type: text/html, Size: 1845 bytes --]

Re: [gentoo-user] Optional /usr merge in Gentoo

[Dale]

[-- Attachment #1: Type: text/plain, Size: 1548 bytes --]

Douglas J Hunley wrote:
>
> On Sun, Sep 1, 2013 at 10:30 AM, Tanstaafl <tanstaafl@libertytrek.org
> <mailto:tanstaafl@libertytrek.org>> wrote:
>
>     Is there any reason that the creation, use and maintenance of the
>     initramfs couldn't be as simple as a checkbox in the kernel
>     config, so that running 'make' after the kernel was configured
>     would automatically build it? Then, all I'd have to do is move it
>     into /boot along with the new kernel (just like I do now), with
>     *nothing* else required, and the kernel would call it, and things
>     would just work (as long as it was there and I didn't forget to
>     copy it to /boot).
>
>
> This exists. You can built initramfs right into the kernel. I've been
> doing it here for quite some time. You just tell the kernel either:
> * where to find a filespec so it knows what to include in the initramfs
> * what directory contains everything you want in the initramfs
>
> and then the kernel builds is and attaches it to itself during 'make'
>
> It's actually pretty trivial
>
>
> -- 
> Douglas J Hunley (doug.hunley@gmail.com <mailto:doug.hunley@gmail.com>)
> Twitter: @hunleyd                                               Web:
> douglasjhunley.com <http://douglasjhunley.com>
> G+: http://goo.gl/sajR3

I tried that a while back.  Followed a howto step by step, Gentoo one I
think, and it never worked, not even once.  Trivial, not hardly. 

Dale

:-)  :-) 

-- 
I am only responsible for what I said ... Not for what you understood or how you interpreted my words!


[-- Attachment #2: Type: text/html, Size: 3146 bytes --]

Re: Integrated ZFS for Gentoo - WAS Re: [gentoo-user] Optional /usr merge in Gentoo

[Walter Dnes]

On Mon, Sep 02, 2013 at 10:47:35AM +0200, Joerg Schilling wrote
> Mark David Dumlao <madumlao@gmail.com> wrote:
> > At what point does grub "present a zfs interface for the kernel to use"?
> 
> After it booted the kernel
> 
> You may not know dynamic kernels as Linux is a static kernel that
> just may load additional modules _after_ it mounted the root fs.
> 
> Solaris is dynamic from the beginning:
> 
> -	no static loading at all
> 
> -	no predefined data sizes - everything is allocated
> 
> -	no predefined major device numbers - numbers are assigned at first load
> 
> Grub works this way:
> 
> 1)	It loads /platform/i86pc/kernel/$ISADIR/unix

  Question... how does it read that file off a ZFS partition?  OK, so
ZFS code has to be installed statically into GRUB instead of statically
into the kernel.  Please stop the shell game.

  Note also that this is a Gentoo *LINUX* mailing list.  We're more
concerned about how Linux works.

-- 
Walter Dnes <waltdnes@waltdnes.org>
I don't run "desktop environments"; I run useful applications

Re: Integrated ZFS for Gentoo - WAS Re: [gentoo-user] Optional /usr merge in Gentoo

[Marc Stürmer]

Am 02.09.2013 10:47, schrieb Joerg Schilling:

> Solaris is dynamic from the beginning:

Well in my point of view it boils down to that: someone wants to use ZFS 
on Linux. Fine. This means you've got to be a good citizen and obey its 
license, of course.

It is for those legal reasons that ZFS is not included into the Linux 
kernel mainline source tree. It is also for those reasons you got to 
compile it as a module.

So somebody wants it being static into his kernel, modules being 
disabled on his machine because of security concerns. Unless he is going 
to do that stuff himself this is unlikely to ever happen.

So it boils down to those possible solutions:

a) writing that stuff himself (unlikely to happen),
b) just using the module and going to be happy (also unlikely to happen 
as it seems),
c) choosing another, native file system like Btrfs (which is still yet 
not production ready as a fast moving target) or going with something 
like XFS or Ext4 (and LVM),

or the most natural choice then, which is

d) choosing an operating system, which supports ZFS out of the box like 
FreeBSD and forget about all the rest of the problems.

I would go for d and forget about all of the rest of the problems. 
FreeBSD has been around long enough, and is stable and mature enough for 
most anything you can throw at and it is a nice, clean, well structured 
system anyway.

There's also Gentoo/FreeBSD around, but personally I would use the 
native ports system instead.

Re: Integrated ZFS for Gentoo - WAS Re: [gentoo-user] Optional /usr merge in Gentoo

[Joerg Schilling]

Walter Dnes <waltdnes@waltdnes.org> wrote:

> > Grub works this way:
> > 
> > 1)	It loads /platform/i86pc/kernel/$ISADIR/unix
>
>   Question... how does it read that file off a ZFS partition?  OK, so
> ZFS code has to be installed statically into GRUB instead of statically
> into the kernel.  Please stop the shell game.

Grub was enhanced by Sun to understand ZFS. You need such an enhanced grub if 
you like to boot off ZFS.

>   Note also that this is a Gentoo *LINUX* mailing list.  We're more
> concerned about how Linux works.

Linux does not contain code to boot AFAIK....

Jörg

-- 
 EMail:joerg@schily.isdn.cs.tu-berlin.de (home) Jörg Schilling D-13353 Berlin
       js@cs.tu-berlin.de                (uni)  
       joerg.schilling@fokus.fraunhofer.de (work) Blog: http://schily.blogspot.com/
 URL:  http://cdrecord.berlios.de/private/ ftp://ftp.berlios.de/pub/schily

Re: Integrated ZFS for Gentoo - WAS Re: [gentoo-user] Optional /usr merge in Gentoo

[Mark David Dumlao]

On Mon, Sep 2, 2013 at 4:47 PM, Joerg Schilling
<Joerg.Schilling@fokus.fraunhofer.de> wrote:
> Mark David Dumlao <madumlao@gmail.com> wrote:
>
>> > > > the disk... OOPS.  This is a classic "chicken and egg" situation.
>> > >
>> > > On Solaris no problem with loadable modules - everything is
>> > > dynamically loaded.  ***YOU NEED A GRUB THAT UNDERSTANDS ZFS AND THAT
>> > > GIVES A ZFS INTERFACE TO THE KERNEL TO USE BEFORE ZFS WAS LOADED***.
>>
>> I'm confused as to what this means. Grub reads a filesystem, loads a kernel
>> with options, and may give it an initrd. What happens from then on is none
>> of grub's business. The filesystem it reads from and the one the kernel
>> uses may be completely unrelated - this is why we have /boot filesystems.
>>
>> At what point does grub "present a zfs interface for the kernel to use"?
>
> After it booted the kernel
>
> You may not know dynamic kernels as Linux is a static kernel that just may load
> additional modules _after_ it mounted the root fs.
>
> Solaris is dynamic from the beginning:

Ah I see. But I think by default when we talk about "the kernel" on
this mailing list,
it's assumed that we're talking about Linux. And in the Linux case,
Grub does not do
anything like provide a filesystem interface to Linux. It just loads
the kernel into memory,
and passes it any arguments, like the initrd. So your grub needs to be
able to read the
filesystem containing the kernel and that's it. If the filesystem
containing the kernel is
also a zfs filesystem, then your grub needs a driver that can read
that filesystem.

Well sys-boot/grub-2.00 provides one. See /boot/grub/zfs.mod
-- 
This email is:    [ ] actionable   [x] fyi        [ ] social
Response needed:  [ ] yes          [ ] up to you  [x] no
Time-sensitive:   [ ] immediate    [ ] soon       [x] none

Re: Integrated ZFS for Gentoo - WAS Re: [gentoo-user] Optional /usr merge in Gentoo

[Joerg Schilling]

Mark David Dumlao <madumlao@gmail.com> wrote:

> containing the kernel is
> also a zfs filesystem, then your grub needs a driver that can read
> that filesystem.
>
> Well sys-boot/grub-2.00 provides one. See /boot/grub/zfs.mod

You don't need grub2, a capable older grub does it also, see:

http://hg.berlios.de/repos/schillix-on

for a related source.

Jörg

-- 
 EMail:joerg@schily.isdn.cs.tu-berlin.de (home) Jörg Schilling D-13353 Berlin
       js@cs.tu-berlin.de                (uni)  
       joerg.schilling@fokus.fraunhofer.de (work) Blog: http://schily.blogspot.com/
 URL:  http://cdrecord.berlios.de/private/ ftp://ftp.berlios.de/pub/schily

Re: Integrated ZFS for Gentoo - WAS Re:[gentoo-user] Optional /usr merge in Gentoo

[Nicolas Sebrecht]

The 04/09/13, Joerg Schilling wrote:

> Linux does not contain code to boot AFAIK....

Sure, it does. You can boot on the kernel directly without a boot
manager.

-- 
Nicolas Sebrecht

Re: Integrated ZFS for Gentoo - WAS Re: [gentoo-user] Optional /usr merge in Gentoo

[Tanstaafl]

On 2013-09-04 2:49 AM, Marc Stürmer <mail@marc-stuermer.de> wrote:
> Well in my point of view it boils down to that: someone wants to use ZFS
> on Linux. Fine. This means you've got to be a good citizen and obey its
> license, of course.
>
> It is for those legal reasons that ZFS is not included into the Linux
> kernel mainline source tree. It is also for those reasons you got to
> compile it as a module.

One of the points made was that this is FUD, and that there is NO logel 
reason that it cannot be included.

There is also the fact that it *could* be included, as long as it wasn't 
provided directly in the kernel sources, but as an overlay/patch type 
process, which could still be provided by the gentoo source repositories.