Re: [gentoo-user] Optional /usr merge in Gentoo

Re: [gentoo-user] Optional /usr merge in Gentoo

[Thomas Mueller]

On the issue of whether ZFS can be shipped with the Linux kernel, FreeBSD includes ZFS with the kernel, binary and source.

So does that mean it would be OK for Linux too?

FreeBSD has a different license (BSD) than Linux (GPL 2 or 3).

I am not a lawyer!

Tom

Re: [gentoo-user] Optional /usr merge in Gentoo

[Alan McKinnon]

On 27/08/2013 04:04, Thomas Mueller wrote:
> On the issue of whether ZFS can be shipped with the Linux kernel, FreeBSD includes ZFS 
> with the kernel, binary and source.
> 
> So does that mean it would be OK for Linux too?

No.

> FreeBSD has a different license (BSD) than Linux (GPL 2 or 3).

Please read file COPYING in the kernel sources, the Linux kernel ships
with license GPL-2

Not a later version at your choice (2.x) and certainly never GPL-3

The issue is that the Linux kernel devs consider the license terms for
ZFS to be incompatible with GPL-2.0 and therefore ZFS cannot be
redistributed as a Linux kernel module.

There's nothing in the GPL-2 to stop you as a user from building and
running ZFS on Linux, as GPL does not interfere with your right to run
whatever you wish. The GPL only kicks in when code is redistributed.

The BSD license has none of these conditions, in layman terms that
license essentially says "you can take this code and pretty much do with
it whatever you want, we don't care"

> I am not a lawyer!
> 
> Tom
> 
> 





-- 
Alan McKinnon
alan.mckinnon@gmail.com

Re: [gentoo-user] Optional /usr merge in Gentoo

[Joerg Schilling]

Thomas Mueller <mueller6726@bellsouth.net> wrote:

> On the issue of whether ZFS can be shipped with the Linux kernel, FreeBSD includes ZFS with the kernel, binary and source.
>
> So does that mean it would be OK for Linux too?
>
> FreeBSD has a different license (BSD) than Linux (GPL 2 or 3).

For FreeBSD, things are less easy than for Linux.

FreeBSD comes with a license that gives real freedom and the CDDL
being copyleft, is a license that intentionally limits the freedom a bit
in order to achieve other benefits.

The GPL limits freedom in a way far beyond what the CDDl does.
Adding code (ZFS) that gives more freedom than the base project (Linux) 
is easy...

It however was a real challenge for me to convince the FreeBSD people in early 
2006 to add something to their code that reduces the freedom of the FreeBSD
project. I succeeded because I could explain them that ZFS is not code that is 
_needed_ in order to run FreeBSD - you just could use their UFS variant instead.
The same arguments worked for integrating DTrace into FreeBSD.


Jörg

-- 
 EMail:joerg@schily.isdn.cs.tu-berlin.de (home) Jörg Schilling D-13353 Berlin
       js@cs.tu-berlin.de                (uni)  
       joerg.schilling@fokus.fraunhofer.de (work) Blog: http://schily.blogspot.com/
 URL:  http://cdrecord.berlios.de/private/ ftp://ftp.berlios.de/pub/schily

Re: [gentoo-user] Optional /usr merge in Gentoo

[Joerg Schilling]

Alan McKinnon <alan.mckinnon@gmail.com> wrote:

> The issue is that the Linux kernel devs consider the license terms for
> ZFS to be incompatible with GPL-2.0 and therefore ZFS cannot be
> redistributed as a Linux kernel module.

Isn't it strange that those people seem to have less problems with closed 
source than with a license that gives more freedom than the GPL? But 
you are correct that the problem seem to be humans and not a license text.

> There's nothing in the GPL-2 to stop you as a user from building and
> running ZFS on Linux, as GPL does not interfere with your right to run
> whatever you wish. The GPL only kicks in when code is redistributed.

There is nothing non-void in the GPL that stops you from distributing binaries.

Jörg

-- 
 EMail:joerg@schily.isdn.cs.tu-berlin.de (home) Jörg Schilling D-13353 Berlin
       js@cs.tu-berlin.de                (uni)  
       joerg.schilling@fokus.fraunhofer.de (work) Blog: http://schily.blogspot.com/
 URL:  http://cdrecord.berlios.de/private/ ftp://ftp.berlios.de/pub/schily

Re: [gentoo-user] Optional /usr merge in Gentoo

[Alan McKinnon]

On 27/08/2013 09:53, Joerg Schilling wrote:
> Alan McKinnon <alan.mckinnon@gmail.com> wrote:
> 
>> The issue is that the Linux kernel devs consider the license terms for
>> ZFS to be incompatible with GPL-2.0 and therefore ZFS cannot be
>> redistributed as a Linux kernel module.
> 
> Isn't it strange that those people seem to have less problems with closed 
> source than with a license that gives more freedom than the GPL? But 
> you are correct that the problem seem to be humans and not a license text.

You are aware that the GPL was not really intended to be used together
with other licenses? It was really intended to create an entire
operating system, all of which was 100% licensed as GPL, all of which
comprise an original work written from scratch

Stallman never makes this claim as bluntly as I've said it here, but
it's the only intelligent reading of his intent as far as I can make
out. This is why so many arguments arise over the GPL, the wording of
that license was not really intended to have it co-exist with other
licenses.

That's how I see it anyway.

> 
>> There's nothing in the GPL-2 to stop you as a user from building and
>> running ZFS on Linux, as GPL does not interfere with your right to run
>> whatever you wish. The GPL only kicks in when code is redistributed.
> 
> There is nothing non-void in the GPL that stops you from distributing binaries.

That's a question of packaging and bundling, which is not covered by the
GPL. But kernel code and kernel modules are not mere bundles, they are
derivative works by virtue of how tightly they integrate with the
kernel, and how the code can only ever run unchanged on Linux.

That is how ZFS as a fuse module works, no license issues with the
kernel there at all.

-- 
Alan McKinnon
alan.mckinnon@gmail.com

Re: [gentoo-user] Optional /usr merge in Gentoo

[Joerg Schilling]

Alan McKinnon <alan.mckinnon@gmail.com> wrote:

> > Isn't it strange that those people seem to have less problems with closed 
> > source than with a license that gives more freedom than the GPL? But 
> > you are correct that the problem seem to be humans and not a license text.
>
> You are aware that the GPL was not really intended to be used together
> with other licenses? It was really intended to create an entire
> operating system, all of which was 100% licensed as GPL, all of which
> comprise an original work written from scratch

But it has been proven that you cannot create a 100% GPL OS.
More than 50% of all Linux distros are under different licenses...

> Stallman never makes this claim as bluntly as I've said it here, but
> it's the only intelligent reading of his intent as far as I can make
> out. This is why so many arguments arise over the GPL, the wording of
> that license was not really intended to have it co-exist with other
> licenses.

Stallman does not look at reality. The first GCC version in 1986 has been 
published under something I call GPLv0 and this license did not permit a legal 
use of the GCC in public.

The license was later converted to GPLv1 by using proposals I made but 
Stallman still only talks about what has been in GPLv0.

> > There is nothing non-void in the GPL that stops you from distributing binaries.
>
> That's a question of packaging and bundling, which is not covered by the
> GPL. But kernel code and kernel modules are not mere bundles, they are
> derivative works by virtue of how tightly they integrate with the
> kernel, and how the code can only ever run unchanged on Linux.

If a kernel uses ZFS, you have to decide on whether the kernel is a derivative 
work of ZFS or whether just a collective work exists.

_Using_ ZFS definitely does not make ZFS a derivative work.

Jörg

-- 
 EMail:joerg@schily.isdn.cs.tu-berlin.de (home) Jörg Schilling D-13353 Berlin
       js@cs.tu-berlin.de                (uni)  
       joerg.schilling@fokus.fraunhofer.de (work) Blog: http://schily.blogspot.com/
 URL:  http://cdrecord.berlios.de/private/ ftp://ftp.berlios.de/pub/schily

Re: [gentoo-user] Optional /usr merge in Gentoo

[Joerg Schilling]

Joerg Schilling <Joerg.Schilling@fokus.fraunhofer.de> wrote:

> Alan McKinnon <alan.mckinnon@gmail.com> wrote:
>
> > > Isn't it strange that those people seem to have less problems with closed 
> > > source than with a license that gives more freedom than the GPL? But 
> > > you are correct that the problem seem to be humans and not a license text.
> >
> > You are aware that the GPL was not really intended to be used together
> > with other licenses? It was really intended to create an entire
> > operating system, all of which was 100% licensed as GPL, all of which
> > comprise an original work written from scratch
>
> But it has been proven that you cannot create a 100% GPL OS.
> More than 50% of all Linux distros are under different licenses...
>

Sorry, this should be: More than 50% of a typical Linux distro is 
under different licenses...

Jörg

-- 
 EMail:joerg@schily.isdn.cs.tu-berlin.de (home) Jörg Schilling D-13353 Berlin
       js@cs.tu-berlin.de                (uni)  
       joerg.schilling@fokus.fraunhofer.de (work) Blog: http://schily.blogspot.com/
 URL:  http://cdrecord.berlios.de/private/ ftp://ftp.berlios.de/pub/schily

Re: [gentoo-user] Optional /usr merge in Gentoo

[Tanstaafl]

Ummm... I didn't suggest that ZFS be shipped with or distributed with 
the kernel...

I was talking about some kind of overlay or patch system, where I could 
add zfs to my kernel use flag, and it would pull the gentoo-sources from 
wherver it pulls them, and pul;l the patch from a *separate*/*different* 
source/location, and then put the patch where it needs to go to be 
properly compiled into the kernel.

Again, the overlay would *not* contain or provide the kernel sources, 
only the zfs 'patch'.

I don't see a problem with that.

On 2013-08-26 10:04 PM, Thomas Mueller <mueller6726@bellsouth.net> wrote:
> On the issue of whether ZFS can be shipped with the Linux kernel, FreeBSD includes ZFS with the kernel, binary and source.
>
> So does that mean it would be OK for Linux too?
>
> FreeBSD has a different license (BSD) than Linux (GPL 2 or 3).
>
> I am not a lawyer!
>
> Tom
>
>

Re: [gentoo-user] Optional /usr merge in Gentoo

[Neil Bothwick]

[-- Attachment #1: Type: text/plain, Size: 785 bytes --]

On Tue, 27 Aug 2013 06:33:52 -0400, Tanstaafl wrote:

> Ummm... I didn't suggest that ZFS be shipped with or distributed with 
> the kernel...
> 
> I was talking about some kind of overlay or patch system, where I could 
> add zfs to my kernel use flag, and it would pull the gentoo-sources
> from wherver it pulls them, and pul;l the patch from a
> *separate*/*different* source/location, and then put the patch where it
> needs to go to be properly compiled into the kernel.

I already posted the script I use to do exactly that.

emerge gentoo-sources
run the script

I wonder it it would be possible to have the spl and zfs-kmod ebuilds do
this with an appropriate USE flag.


-- 
Neil Bothwick

Bury a lawyer 12 feet under, because deep down they're nice.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: [gentoo-user] Optional /usr merge in Gentoo

[Tanstaafl]

On 2013-08-27 8:25 AM, Neil Bothwick <neil@digimed.co.uk> wrote:
> On Tue, 27 Aug 2013 06:33:52 -0400, Tanstaafl wrote:
>
>> Ummm... I didn't suggest that ZFS be shipped with or distributed with
>> the kernel...
>>
>> I was talking about some kind of overlay or patch system, where I could
>> add zfs to my kernel use flag, and it would pull the gentoo-sources
>> from wherver it pulls them, and pul;l the patch from a
>> *separate*/*different* source/location, and then put the patch where it
>> needs to go to be properly compiled into the kernel.
>
> I already posted the script I use to do exactly that.
>
> emerge gentoo-sources
> run the script
>
> I wonder it it would be possible to have the spl and zfs-kmod ebuilds do
> this with an appropriate USE flag.

Thats what I'm looking for... something that is automatic and basically 
'just works'.

Manually running a script as part of each kernel update just... well, 
computers do automation best.

But thanks very much for your script. I'm just not comfortable (at this 
point at least) doing it that way on a production system...

Re: [gentoo-user] Optional /usr merge in Gentoo

[Neil Bothwick]

[-- Attachment #1: Type: text/plain, Size: 1148 bytes --]

On Tue, 27 Aug 2013 08:37:54 -0400, Tanstaafl wrote:

> > I already posted the script I use to do exactly that.
> >
> > emerge gentoo-sources
> > run the script
> >
> > I wonder it it would be possible to have the spl and zfs-kmod ebuilds
> > do this with an appropriate USE flag.  
> 
> Thats what I'm looking for... something that is automatic and basically 
> 'just works'.
> 
> Manually running a script as part of each kernel update just... well, 
> computers do automation best.

I use a script to configure, build and install new kernels. It's called
from there, so it is automatic for me :)

> But thanks very much for your script. I'm just not comfortable (at this 
> point at least) doing it that way on a production system...

That's the recommended way, since the script follows the instructions for
merging the modules in the kernel tree and uses the make scripts that
come with the sources. It will not mess up your kernel since it only adds
code, code that isn't even used until you enable it in the .config.


-- 
Neil Bothwick

MACINTOSH: Most Applications Crash; If Not, The Operating System Hangs

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: [gentoo-user] Optional /usr merge in Gentoo

[Alan McKinnon]

On 27/08/2013 11:08, Joerg Schilling wrote:
> Alan McKinnon <alan.mckinnon@gmail.com> wrote:
> 
>>> Isn't it strange that those people seem to have less problems with closed 
>>> source than with a license that gives more freedom than the GPL? But 
>>> you are correct that the problem seem to be humans and not a license text.
>>
>> You are aware that the GPL was not really intended to be used together
>> with other licenses? It was really intended to create an entire
>> operating system, all of which was 100% licensed as GPL, all of which
>> comprise an original work written from scratch
> 
> But it has been proven that you cannot create a 100% GPL OS.
> More than 50% of all Linux distros are under different licenses...
> 
>> Stallman never makes this claim as bluntly as I've said it here, but
>> it's the only intelligent reading of his intent as far as I can make
>> out. This is why so many arguments arise over the GPL, the wording of
>> that license was not really intended to have it co-exist with other
>> licenses.
> 
> Stallman does not look at reality. The first GCC version in 1986 has been 
> published under something I call GPLv0 and this license did not permit a legal 
> use of the GCC in public.
> 
> The license was later converted to GPLv1 by using proposals I made but 
> Stallman still only talks about what has been in GPLv0.


I didn't bring this up to discuss fine points of licenses. I brought it
up for those who might want to understand what the GPL is intended to
do; that can only be truly understood by determining what Stallman
intended. The GPL is a reflection of Stallman's intent, and can only be
truly understood in that light.

Whether the legal wording accurately matches his intent is another
matter altogether. I personally feel it doesn't, won't and cannot, for
reasons of psychology and philosophy, not for reasons of technology or
law. What the GPL tries to do and how it does it is quite foreign to
most who practice law. Humans don't like foreign concepts. Heck, GPL-2
doesn't even remotely read like something that came off a lawyer's desk.


> 
>>> There is nothing non-void in the GPL that stops you from distributing binaries.
>>
>> That's a question of packaging and bundling, which is not covered by the
>> GPL. But kernel code and kernel modules are not mere bundles, they are
>> derivative works by virtue of how tightly they integrate with the
>> kernel, and how the code can only ever run unchanged on Linux.
> 
> If a kernel uses ZFS, you have to decide on whether the kernel is a derivative 
> work of ZFS or whether just a collective work exists.
> 
> _Using_ ZFS definitely does not make ZFS a derivative work.

I never said it did. I was concentrating on those parts of ZFS that
interact with kernel internals - that might not be been entirely clear

You are making a spurious claim by saying "you have to decide on whether
the kernel is a derivative work of ZFS or ..."

In what possible way could the entire Linux kernel be considered a
derivative work of ZFS? That doesn't make any sense.



-- 
Alan McKinnon
alan.mckinnon@gmail.com

Re: [gentoo-user] Optional /usr merge in Gentoo

[Alan McKinnon]

On 27/08/2013 11:26, Joerg Schilling wrote:
> Joerg Schilling <Joerg.Schilling@fokus.fraunhofer.de> wrote:
> 
>> Alan McKinnon <alan.mckinnon@gmail.com> wrote:
>>
>>>> Isn't it strange that those people seem to have less problems with closed 
>>>> source than with a license that gives more freedom than the GPL? But 
>>>> you are correct that the problem seem to be humans and not a license text.
>>>
>>> You are aware that the GPL was not really intended to be used together
>>> with other licenses? It was really intended to create an entire
>>> operating system, all of which was 100% licensed as GPL, all of which
>>> comprise an original work written from scratch
>>
>> But it has been proven that you cannot create a 100% GPL OS.
>> More than 50% of all Linux distros are under different licenses...
>>
> 
> Sorry, this should be: More than 50% of a typical Linux distro is 
> under different licenses...


All we can state for sure is that no-one has yet created a fully 100%
GPL operating system. If you persuade FSF to relicense glibc to you as
GPL it *is* possible to do it for kernel and (a somewhat crippled)
userland. But not for firmware.

But this is beside the point, I was illustrating Stallman's intent, not
whether that intent could be realized or not.


-- 
Alan McKinnon
alan.mckinnon@gmail.com

Re: [gentoo-user] Optional /usr merge in Gentoo

[Joerg Schilling]

Alan McKinnon <alan.mckinnon@gmail.com> wrote:

> >> That's a question of packaging and bundling, which is not covered by the
> >> GPL. But kernel code and kernel modules are not mere bundles, they are
> >> derivative works by virtue of how tightly they integrate with the
> >> kernel, and how the code can only ever run unchanged on Linux.
> > 
> > If a kernel uses ZFS, you have to decide on whether the kernel is a derivative 
> > work of ZFS or whether just a collective work exists.
> > 
> > _Using_ ZFS definitely does not make ZFS a derivative work.
>
> I never said it did. I was concentrating on those parts of ZFS that
> interact with kernel internals - that might not be been entirely clear

You wrote that modules become derivatives of the Linux kernel and this is the 
same as writing ZFS would become a kernel derivative.

The linux kernel does not come with a modern VFS implementation, so if you like 
to use ZFS on Linux you first need to provide a suitable VFS interface.
ZFS will not interact with the Linux kernel directly but with the expected VFS 
layer. Shouldn't it be possible to put this intermediate layer under a license 
that makes even the zealots happy?

> You are making a spurious claim by saying "you have to decide on whether
> the kernel is a derivative work of ZFS or ..."

If you go the non-lawful Stallman way and insist in a derivative work to be 
build, then the linux kernel is the derivative work. I prefer to assume that 
this just builds a collective work ;-)

> In what possible way could the entire Linux kernel be considered a
> derivative work of ZFS? That doesn't make any sense.

I am just quoting claims from Stallman ;-)

Jörg

-- 
 EMail:joerg@schily.isdn.cs.tu-berlin.de (home) Jörg Schilling D-13353 Berlin
       js@cs.tu-berlin.de                (uni)  
       joerg.schilling@fokus.fraunhofer.de (work) Blog: http://schily.blogspot.com/
 URL:  http://cdrecord.berlios.de/private/ ftp://ftp.berlios.de/pub/schily

Re: [gentoo-user] Optional /usr merge in Gentoo

[Tanstaafl]

On 2013-08-27 5:06 PM, Joerg Schilling 
<Joerg.Schilling@fokus.fraunhofer.de> wrote:
> You wrote that modules become derivatives of the Linux kernel and this is the
> same as writing ZFS would become a kernel derivative.

Just for clarification, I was talking about compiling ZFS support INTO 
the kernel, not running it as a module.

Do you claim that support for compiling ZFS directly into the kernel 
also does not violate the license?

Re: [gentoo-user] Optional /usr merge in Gentoo

[Alan McKinnon]

On 28/08/2013 12:58, Tanstaafl wrote:
> On 2013-08-27 5:06 PM, Joerg Schilling
> <Joerg.Schilling@fokus.fraunhofer.de> wrote:
>> You wrote that modules become derivatives of the Linux kernel and this
>> is the
>> same as writing ZFS would become a kernel derivative.
> 
> Just for clarification, I was talking about compiling ZFS support INTO
> the kernel, not running it as a module.
> 
> Do you claim that support for compiling ZFS directly into the kernel
> also does not violate the license?
> 


Whether the code is compile in or a module makes no difference wrt
licenses as far as I know.

There's no limitation on *running* the code, you can fetch and patch and
edit and compile and run all you want and have it on as many of your (or
the company's) machines as you want - neither license interferes with
your right to do that.

You may not redistribute the code though.

A common misconception with these license is that they have something to
do with whether you may run the code or not. That is incorrect. Free
licenses are all about redistribution and your obligations about sharing
when you hand the code over to others.



-- 
Alan McKinnon
alan.mckinnon@gmail.com

Re: [gentoo-user] Optional /usr merge in Gentoo

[Joerg Schilling]

Tanstaafl <tanstaafl@libertytrek.org> wrote:

> On 2013-08-27 5:06 PM, Joerg Schilling 
> <Joerg.Schilling@fokus.fraunhofer.de> wrote:
> > You wrote that modules become derivatives of the Linux kernel and this is the
> > same as writing ZFS would become a kernel derivative.
>
> Just for clarification, I was talking about compiling ZFS support INTO 
> the kernel, not running it as a module.
>
> Do you claim that support for compiling ZFS directly into the kernel 
> also does not violate the license?

There is no difference, both is permitted.

Jörg

-- 
 EMail:joerg@schily.isdn.cs.tu-berlin.de (home) Jörg Schilling D-13353 Berlin
       js@cs.tu-berlin.de                (uni)  
       joerg.schilling@fokus.fraunhofer.de (work) Blog: http://schily.blogspot.com/
 URL:  http://cdrecord.berlios.de/private/ ftp://ftp.berlios.de/pub/schily

Re: [gentoo-user] Optional /usr merge in Gentoo

[Tanstaafl]

On 2013-08-28 7:12 AM, Alan McKinnon <alan.mckinnon@gmail.com> wrote:
> Whether the code is compile in or a module makes no difference wrt
> licenses as far as I know.
>
> There's no limitation on*running*  the code, you can fetch and patch and
> edit and compile and run all you want and have it on as many of your (or
> the company's) machines as you want - neither license interferes with
> your right to do that.
>
> You may not redistribute the code though.

So, can you answer me this...

Why would there be a problem if someone decided to create a 3rd party 
overlay *not* part of the official gentoo portage tree that contained 
*only* the zfs stuff, and when this overlay was installed combined with 
a zfs keyword for the kernel, portage would then pull in the required 
files, and automagically build a kernel with an up to date version of 
zfs properly and fully integrated?

Would this not work, *and* have no problems with licensing?

Re: [gentoo-user] Optional /usr merge in Gentoo

[Alan McKinnon]

On 30/08/2013 16:29, Tanstaafl wrote:
> On 2013-08-28 7:12 AM, Alan McKinnon <alan.mckinnon@gmail.com> wrote:
>> Whether the code is compile in or a module makes no difference wrt
>> licenses as far as I know.
>>
>> There's no limitation on*running*  the code, you can fetch and patch and
>> edit and compile and run all you want and have it on as many of your (or
>> the company's) machines as you want - neither license interferes with
>> your right to do that.
>>
>> You may not redistribute the code though.
> 
> So, can you answer me this...
> 
> Why would there be a problem if someone decided to create a 3rd party
> overlay *not* part of the official gentoo portage tree that contained
> *only* the zfs stuff, and when this overlay was installed combined with
> a zfs keyword for the kernel, portage would then pull in the required
> files, and automagically build a kernel with an up to date version of
> zfs properly and fully integrated?
> 
> Would this not work, *and* have no problems with licensing?
> 

there is no problem with licensing in that case.
The ebuild could even go in the portage tree, as Gentoo is not
redistributing sources when it publishes an ebuild.

-- 
Alan McKinnon
alan.mckinnon@gmail.com

Integrated ZFS for Gentoo - WAS Re: [gentoo-user] Optional /usr merge in Gentoo

[Tanstaafl]

On 2013-08-30 10:34 AM, Alan McKinnon <alan.mckinnon@gmail.com> wrote:
> On 30/08/2013 16:29, Tanstaafl wrote:
>> Why would there be a problem if someone decided to create a 3rd party
>> overlay *not* part of the official gentoo portage tree that contained
>> *only* the zfs stuff, and when this overlay was installed combined with
>> a zfs keyword for the kernel, portage would then pull in the required
>> files, and automagically build a kernel with an up to date version of
>> zfs properly and fully integrated?
>>
>> Would this not work, *and* have no problems with licensing?

> there is no problem with licensing in that case.
> The ebuild could even go in the portage tree, as Gentoo is not
> redistributing sources when it publishes an ebuild.

Thanks Alan! Just the answer I wanted.

Ok, so... how hard would this be then? What would the chances be that 
this could actually happen? I'll happily go open a bug for it if you 
think the work would be minimal...

It seems to me that I can't be the only one who would like to see this 
happen?

Re: Integrated ZFS for Gentoo - WAS Re: [gentoo-user] Optional /usr merge in Gentoo

[Alan McKinnon]

On 30/08/2013 16:44, Tanstaafl wrote:
> On 2013-08-30 10:34 AM, Alan McKinnon <alan.mckinnon@gmail.com> wrote:
>> On 30/08/2013 16:29, Tanstaafl wrote:
>>> Why would there be a problem if someone decided to create a 3rd party
>>> overlay *not* part of the official gentoo portage tree that contained
>>> *only* the zfs stuff, and when this overlay was installed combined with
>>> a zfs keyword for the kernel, portage would then pull in the required
>>> files, and automagically build a kernel with an up to date version of
>>> zfs properly and fully integrated?
>>>
>>> Would this not work, *and* have no problems with licensing?
> 
>> there is no problem with licensing in that case.
>> The ebuild could even go in the portage tree, as Gentoo is not
>> redistributing sources when it publishes an ebuild.
> 
> Thanks Alan! Just the answer I wanted.
> 
> Ok, so... how hard would this be then? What would the chances be that
> this could actually happen? I'll happily go open a bug for it if you
> think the work would be minimal...
> 
> It seems to me that I can't be the only one who would like to see this
> happen?
> 


Ahem, Mr Bothwick!

Our friend with the thing about free lunches needs you to demonstrate
your penmanship, considering you have some proven results in this area.

:-)



-- 
Alan McKinnon
alan.mckinnon@gmail.com

Re: Integrated ZFS for Gentoo - WAS Re: [gentoo-user] Optional /usr merge in Gentoo

[Mick]

[-- Attachment #1: Type: Text/Plain, Size: 1227 bytes --]

On Friday 30 Aug 2013 15:44:35 Tanstaafl wrote:
> On 2013-08-30 10:34 AM, Alan McKinnon <alan.mckinnon@gmail.com> wrote:
> > On 30/08/2013 16:29, Tanstaafl wrote:
> >> Why would there be a problem if someone decided to create a 3rd party
> >> overlay *not* part of the official gentoo portage tree that contained
> >> *only* the zfs stuff, and when this overlay was installed combined with
> >> a zfs keyword for the kernel, portage would then pull in the required
> >> files, and automagically build a kernel with an up to date version of
> >> zfs properly and fully integrated?
> >> 
> >> Would this not work, *and* have no problems with licensing?
> > 
> > there is no problem with licensing in that case.
> > The ebuild could even go in the portage tree, as Gentoo is not
> > redistributing sources when it publishes an ebuild.
> 
> Thanks Alan! Just the answer I wanted.
> 
> Ok, so... how hard would this be then? What would the chances be that
> this could actually happen? I'll happily go open a bug for it if you
> think the work would be minimal...
> 
> It seems to me that I can't be the only one who would like to see this
> happen?

Nope! I will vote for you.  ;-)

-- 
Regards,
Mick

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 490 bytes --]

Re: Integrated ZFS for Gentoo - WAS Re: [gentoo-user] Optional /usr merge in Gentoo

[Mark David Dumlao]

On Sat, Aug 31, 2013 at 4:16 AM, Mick <michaelkintzios@gmail.com> wrote:
> On Friday 30 Aug 2013 15:44:35 Tanstaafl wrote:
>> On 2013-08-30 10:34 AM, Alan McKinnon <alan.mckinnon@gmail.com> wrote:
>> > On 30/08/2013 16:29, Tanstaafl wrote:
>> >> Why would there be a problem if someone decided to create a 3rd party
>> >> overlay *not* part of the official gentoo portage tree that contained
>> >> *only* the zfs stuff, and when this overlay was installed combined with
>> >> a zfs keyword for the kernel, portage would then pull in the required
>> >> files, and automagically build a kernel with an up to date version of
>> >> zfs properly and fully integrated?
>> >>
>> >> Would this not work, *and* have no problems with licensing?
>> >
>> > there is no problem with licensing in that case.
>> > The ebuild could even go in the portage tree, as Gentoo is not
>> > redistributing sources when it publishes an ebuild.
>>
>> Thanks Alan! Just the answer I wanted.
>>
>> Ok, so... how hard would this be then? What would the chances be that
>> this could actually happen? I'll happily go open a bug for it if you
>> think the work would be minimal...
>>
>> It seems to me that I can't be the only one who would like to see this
>> happen?
>
> Nope! I will vote for you.  ;-)
>
> --
> Regards,
> Mick

Sounds like an awful lot of trouble for a "problem" that's already solved by
installing sys-kernel/module-rebuild and running "module-rebuild rebuild"
after every kernel update, which is how nvidia, broadcom, and other
kernel modules are dealt painlessly with anyways...

-- 
This email is:    [ ] actionable   [x] fyi        [ ] social
Response needed:  [ ] yes          [x] up to you  [ ] no
Time-sensitive:   [ ] immediate    [ ] soon       [x] none

Re: Integrated ZFS for Gentoo - WAS Re: [gentoo-user] Optional /usr merge in Gentoo

[Peter Humphrey]

On Friday 30 Aug 2013 21:21:10 Alan McKinnon wrote:

> Ahem, Mr Bothwick!
> 
> Our friend with the thing about free lunches needs you to demonstrate
> your penmanship, considering you have some proven results in this area.

...and I'd happily act as editor...

:-)   ;-)

-- 
Regards,
Peter

Re: Integrated ZFS for Gentoo - WAS Re: [gentoo-user] Optional /usr merge in Gentoo

[Pandu Poluan]

On Sat, Aug 31, 2013 at 12:10 PM, Mark David Dumlao <madumlao@gmail.com> wrote:
> On Sat, Aug 31, 2013 at 4:16 AM, Mick <michaelkintzios@gmail.com> wrote:
>> On Friday 30 Aug 2013 15:44:35 Tanstaafl wrote:
>>> On 2013-08-30 10:34 AM, Alan McKinnon <alan.mckinnon@gmail.com> wrote:
>>> > On 30/08/2013 16:29, Tanstaafl wrote:
>>> >> Why would there be a problem if someone decided to create a 3rd party
>>> >> overlay *not* part of the official gentoo portage tree that contained
>>> >> *only* the zfs stuff, and when this overlay was installed combined with
>>> >> a zfs keyword for the kernel, portage would then pull in the required
>>> >> files, and automagically build a kernel with an up to date version of
>>> >> zfs properly and fully integrated?
>>> >>
>>> >> Would this not work, *and* have no problems with licensing?
>>> >
>>> > there is no problem with licensing in that case.
>>> > The ebuild could even go in the portage tree, as Gentoo is not
>>> > redistributing sources when it publishes an ebuild.
>>>
>>> Thanks Alan! Just the answer I wanted.
>>>
>>> Ok, so... how hard would this be then? What would the chances be that
>>> this could actually happen? I'll happily go open a bug for it if you
>>> think the work would be minimal...
>>>
>>> It seems to me that I can't be the only one who would like to see this
>>> happen?
>>
>> Nope! I will vote for you.  ;-)
>>
>> --
>> Regards,
>> Mick
>
> Sounds like an awful lot of trouble for a "problem" that's already solved by
> installing sys-kernel/module-rebuild and running "module-rebuild rebuild"
> after every kernel update, which is how nvidia, broadcom, and other
> kernel modules are dealt painlessly with anyways...
>

Well, if you follow Tanstaafl in the other thread, you'll see that he
wants ZFS to be integrated into the kernel, not existing as a kernel
module.


Rgds,
-- 
FdS Pandu E Poluan
~ IT Optimizer ~

 • LOPSA Member #15248
 • Blog : http://pepoluan.tumblr.com
 • Linked-In : http://id.linkedin.com/in/pepoluan

Re: Integrated ZFS for Gentoo - WAS Re: [gentoo-user] Optional /usr merge in Gentoo

[Joerg Schilling]

Pandu Poluan <pandu@poluan.info> wrote:

> Well, if you follow Tanstaafl in the other thread, you'll see that he
> wants ZFS to be integrated into the kernel, not existing as a kernel
> module.
>

But why does someone want things to be inside a static kernel?

Since 1991/1992, Solaris does not have anything in the static "kernel" than 
the startup code, the basic scheduler code and the pager daemon. You need a 
bootloader that knows about ELF dependencies, but grub has been enhanced for 
that feature.

Everything is dynamic, you would however put a lot of effort into the linux 
kernel to get to that state...e.g. automated major device numbering.



Jörg

-- 
 EMail:joerg@schily.isdn.cs.tu-berlin.de (home) Jörg Schilling D-13353 Berlin
       js@cs.tu-berlin.de                (uni)  
       joerg.schilling@fokus.fraunhofer.de (work) Blog: http://schily.blogspot.com/
 URL:  http://cdrecord.berlios.de/private/ ftp://ftp.berlios.de/pub/schily

Re: Integrated ZFS for Gentoo - WAS Re: [gentoo-user] Optional /usr merge in Gentoo

[Tanstaafl]

On 2013-08-31 1:10 AM, Mark David Dumlao <madumlao@gmail.com> wrote:
> Sounds like an awful lot of trouble for a "problem" that's already solved by
> installing sys-kernel/module-rebuild and running "module-rebuild rebuild"
> after every kernel update, which is how nvidia, broadcom, and other
> kernel modules are dealt painlessly with anyways...

You must have missed the point that this is for *servers*, that most 
people *disable modules* on. I *know* that it is available as a module.

Re: Integrated ZFS for Gentoo - WAS Re: [gentoo-user] Optional /usr merge in Gentoo

[Tanstaafl]

On 2013-08-31 7:04 AM, Joerg Schilling 
<Joerg.Schilling@fokus.fraunhofer.de> wrote:
> Everything is dynamic, you would however put a lot of effort into the linux
> kernel to get to that state...e.g. automated major device numbering.

??? I've been running my servers without modules since... I started 
running servers.

Servers are not like desktops - constantly changing devices. They - in 
most cases - *are* static, and most people *want* them that way.

Regardless, please do *not* distract this thread with arguments about 
it. If you don't want or see the benefit, fine, just ignore this thread.

Re: Integrated ZFS for Gentoo - WAS Re: [gentoo-user] Optional /usr merge in Gentoo

[Joerg Schilling]

Tanstaafl <tanstaafl@libertytrek.org> wrote:

> On 2013-08-31 1:10 AM, Mark David Dumlao <madumlao@gmail.com> wrote:
> > Sounds like an awful lot of trouble for a "problem" that's already solved by
> > installing sys-kernel/module-rebuild and running "module-rebuild rebuild"
> > after every kernel update, which is how nvidia, broadcom, and other
> > kernel modules are dealt painlessly with anyways...
>
> You must have missed the point that this is for *servers*, that most 
> people *disable modules* on. I *know* that it is available as a module.

Why, for security reasons?


On Solaris, you can disable loading unsigned modules,  is this not supported by 
Linux?

Jörg

-- 
 EMail:joerg@schily.isdn.cs.tu-berlin.de (home) Jörg Schilling D-13353 Berlin
       js@cs.tu-berlin.de                (uni)  
       joerg.schilling@fokus.fraunhofer.de (work) Blog: http://schily.blogspot.com/
 URL:  http://cdrecord.berlios.de/private/ ftp://ftp.berlios.de/pub/schily

Re: Integrated ZFS for Gentoo - WAS Re: [gentoo-user] Optional /usr merge in Gentoo

[Alon Bar-Lev]

On Sat, Aug 31, 2013 at 2:28 PM, Tanstaafl <tanstaafl@libertytrek.org> wrote:
>
> On 2013-08-31 7:04 AM, Joerg Schilling <Joerg.Schilling@fokus.fraunhofer.de> wrote:
>>
>> Everything is dynamic, you would however put a lot of effort into the linux
>> kernel to get to that state...e.g. automated major device numbering.
>
>
> ??? I've been running my servers without modules since... I started running servers.
>
> Servers are not like desktops - constantly changing devices. They - in most cases - *are* static, and most people *want* them that way.
>
> Regardless, please do *not* distract this thread with arguments about it. If you don't want or see the benefit, fine, just ignore this thread.
>

I do not understand this thread.

If this is not mainline, and it is not trivial gentoo kernels
maintainer patch, and you must have this as static, you can just put
the patch within /etc/portage/patches/sys-kernel/gentoo-sources/, so
it will patch your kernel every time you emerge new one.

Regards,
Alon

Re: Integrated ZFS for Gentoo - WAS Re: [gentoo-user] Optional /usr merge in Gentoo

[Gregory Shearman]

In linux.gentoo.user, Mr Schilling wrote:
>
> On Solaris, you can disable loading unsigned modules,  is this not supported by 
> Linux?

CONFIG_MODULE_SIG

-- 
Regards,
Gregory.

Re: Integrated ZFS for Gentoo - WAS Re: [gentoo-user] Optional /usr merge in Gentoo

[Joerg Schilling]

Gregory Shearman <zekeyg@gmail.com> wrote:

> In linux.gentoo.user, Mr Schilling wrote:
> >
> > On Solaris, you can disable loading unsigned modules,  is this not supported by 
> > Linux?
>
> CONFIG_MODULE_SIG

So there seems to be no real need to create a static linux kernel with ZFS 
inside.

Jörg

-- 
 EMail:joerg@schily.isdn.cs.tu-berlin.de (home) Jörg Schilling D-13353 Berlin
       js@cs.tu-berlin.de                (uni)  
       joerg.schilling@fokus.fraunhofer.de (work) Blog: http://schily.blogspot.com/
 URL:  http://cdrecord.berlios.de/private/ ftp://ftp.berlios.de/pub/schily

Re: Integrated ZFS for Gentoo - WAS Re: [gentoo-user] Optional /usr merge in Gentoo

[Mark David Dumlao]

On Sat, Aug 31, 2013 at 7:25 PM, Tanstaafl <tanstaafl@libertytrek.org> wrote:
> On 2013-08-31 1:10 AM, Mark David Dumlao <madumlao@gmail.com> wrote:
>>
>> Sounds like an awful lot of trouble for a "problem" that's already solved
>> by
>> installing sys-kernel/module-rebuild and running "module-rebuild rebuild"
>> after every kernel update, which is how nvidia, broadcom, and other
>> kernel modules are dealt painlessly with anyways...
>
>
> You must have missed the point that this is for *servers*, that most people
> *disable modules* on. I *know* that it is available as a module.
>


Ok, I was just asking. But as for what "most people" do on their servers,
speak for yourself.

-- 
This email is:    [ ] actionable   [ ] fyi        [x] social
Response needed:  [ ] yes          [ ] up to you  [x] no
Time-sensitive:   [ ] immediate    [ ] soon       [x] none

Re: Integrated ZFS for Gentoo - WAS Re: [gentoo-user] Optional /usr merge in Gentoo

[Walter Dnes]

On Sat, Aug 31, 2013 at 02:19:56PM +0200, Joerg Schilling wrote

> So there seems to be no real need to create a static linux kernel
> with ZFS inside.

See http://www.gentoo.org/doc/en/handbook/handbook-amd64.xml?full=1#book_part1_chap7

> Now go to File Systems and select support for the filesystems you use.
> Don't compile the file system you use for the root filesystem as
> module, otherwise your Gentoo system will not be able to mount
> your partition.

  You can get away with most stuff as modules; ***BUT NOT THE ROOT
FILESYSTEM***.  Think about it for a minute.  Gentoo reads modules off
the disk.  If the code for the root filesystem is a module, Gentoo would
have to read the module off the disk to enable it to read the module off
the disk... OOPS.  This is a classic "chicken and egg" situation.

-- 
Walter Dnes <waltdnes@waltdnes.org>
I don't run "desktop environments"; I run useful applications

Re: Integrated ZFS for Gentoo - WAS Re: [gentoo-user] Optional /usr merge in Gentoo

[Canek Peláez Valdés]

On Sat, Aug 31, 2013 at 7:13 PM, Walter Dnes <waltdnes@waltdnes.org> wrote:
> On Sat, Aug 31, 2013 at 02:19:56PM +0200, Joerg Schilling wrote
>
>> So there seems to be no real need to create a static linux kernel
>> with ZFS inside.
>
> See http://www.gentoo.org/doc/en/handbook/handbook-amd64.xml?full=1#book_part1_chap7
>
>> Now go to File Systems and select support for the filesystems you use.
>> Don't compile the file system you use for the root filesystem as
>> module, otherwise your Gentoo system will not be able to mount
>> your partition.
>
>   You can get away with most stuff as modules; ***BUT NOT THE ROOT
> FILESYSTEM***.  Think about it for a minute.  Gentoo reads modules off
> the disk.  If the code for the root filesystem is a module, Gentoo would
> have to read the module off the disk to enable it to read the module off
> the disk... OOPS.  This is a classic "chicken and egg" situation.

I usally use ext4 as filesystem.

# lsmod|grep ext
ext3                  100768  0
jbd                    39586  1 ext3
ext2                   49572  0
ext4                  263621  1
crc16                   1255  2 ext4,bluetooth
mbcache                 4450  3 ext2,ext3,ext4
jbd2                   48679  1 ext4

Isn't great what an initramfs can do?

Regards.
-- 
Canek Peláez Valdés
Posgrado en Ciencia e Ingeniería de la Computación
Universidad Nacional Autónoma de México

Re: Integrated ZFS for Gentoo - WAS Re: [gentoo-user] Optional /usr merge in Gentoo

[Mark David Dumlao]

On Sun, Sep 1, 2013 at 8:13 AM, Walter Dnes <waltdnes@waltdnes.org> wrote:
> On Sat, Aug 31, 2013 at 02:19:56PM +0200, Joerg Schilling wrote
>
>> So there seems to be no real need to create a static linux kernel
>> with ZFS inside.
>
> See http://www.gentoo.org/doc/en/handbook/handbook-amd64.xml?full=1#book_part1_chap7
>
>> Now go to File Systems and select support for the filesystems you use.
>> Don't compile the file system you use for the root filesystem as
>> module, otherwise your Gentoo system will not be able to mount
>> your partition.
>
>   You can get away with most stuff as modules; ***BUT NOT THE ROOT
> FILESYSTEM***.  Think about it for a minute.  Gentoo reads modules off
> the disk.  If the code for the root filesystem is a module, Gentoo would
> have to read the module off the disk to enable it to read the module off
> the disk... OOPS.  This is a classic "chicken and egg" situation.

And this is why the initrd was actually invented.
http://en.wikipedia.org/wiki/Initrd

It's a means of loading kernel modules so that the root filesystem can be
mounted as a module.
-- 
This email is:    [ ] actionable   [x] fyi        [ ] social
Response needed:  [ ] yes          [ ] up to you  [x] no
Time-sensitive:   [ ] immediate    [ ] soon       [x] none

Re: Integrated ZFS for Gentoo - WAS Re: [gentoo-user] Optional /usr merge in Gentoo

[Pandu Poluan]

[-- Attachment #1: Type: text/plain, Size: 1483 bytes --]

On Sep 1, 2013 7:51 AM, "Mark David Dumlao" <madumlao@gmail.com> wrote:
>
> On Sun, Sep 1, 2013 at 8:13 AM, Walter Dnes <waltdnes@waltdnes.org> wrote:
> > On Sat, Aug 31, 2013 at 02:19:56PM +0200, Joerg Schilling wrote
> >
> >> So there seems to be no real need to create a static linux kernel
> >> with ZFS inside.
> >
> > See
http://www.gentoo.org/doc/en/handbook/handbook-amd64.xml?full=1#book_part1_chap7
> >
> >> Now go to File Systems and select support for the filesystems you use.
> >> Don't compile the file system you use for the root filesystem as
> >> module, otherwise your Gentoo system will not be able to mount
> >> your partition.
> >
> >   You can get away with most stuff as modules; ***BUT NOT THE ROOT
> > FILESYSTEM***.  Think about it for a minute.  Gentoo reads modules off
> > the disk.  If the code for the root filesystem is a module, Gentoo would
> > have to read the module off the disk to enable it to read the module off
> > the disk... OOPS.  This is a classic "chicken and egg" situation.
>
> And this is why the initrd was actually invented.
> http://en.wikipedia.org/wiki/Initrd
>
> It's a means of loading kernel modules so that the root filesystem can be
> mounted as a module.

Not everyone is willing to use an initr* thingy. It's another potential
breaking point.

I have no problem with /usr being 'merged' with /, in fact I have been
doing that for a couple of years now.

But I will keep myself a mile away from an initr* thingy.

Rgds,
--

[-- Attachment #2: Type: text/html, Size: 2140 bytes --]

Re: Integrated ZFS for Gentoo - WAS Re: [gentoo-user] Optional /usr merge in Gentoo

[Walter Dnes]

> I usally use ext4 as filesystem.
> 
> # lsmod|grep ext
> ext3                  100768  0
> jbd                    39586  1 ext3
> ext2                   49572  0
> ext4                  263621  1
> crc16                   1255  2 ext4,bluetooth
> mbcache                 4450  3 ext2,ext3,ext4
> jbd2                   48679  1 ext4
> 
> Isn't great what an initramfs can do?

  In this case, initramfs is your root filesystem, from which you load
another fs and then transfer (pivot root?) to it.  You have to build
initramfs support into the kernel, to boot an initramfs.  So my argument
still stands, regardless of whether your *INITIAL* filesystem is ext4fs,
or ZFS, or initramfs, that *INITIAL* filesystem has to be built into the
kernel.  Also, I really wonder what the point is in having to use
initramfs on a system where /usr is part of /.

-- 
Walter Dnes <waltdnes@waltdnes.org>
I don't run "desktop environments"; I run useful applications

Re: Integrated ZFS for Gentoo - WAS Re: [gentoo-user] Optional /usr merge in Gentoo

[Canek Peláez Valdés]

On Sat, Aug 31, 2013 at 10:55 PM, Walter Dnes <waltdnes@waltdnes.org> wrote:
>> I usally use ext4 as filesystem.
>>
>> # lsmod|grep ext
>> ext3                  100768  0
>> jbd                    39586  1 ext3
>> ext2                   49572  0
>> ext4                  263621  1
>> crc16                   1255  2 ext4,bluetooth
>> mbcache                 4450  3 ext2,ext3,ext4
>> jbd2                   48679  1 ext4
>>
>> Isn't great what an initramfs can do?
>
>   In this case, initramfs is your root filesystem, from which you load
> another fs and then transfer (pivot root?) to it.  You have to build
> initramfs support into the kernel, to boot an initramfs.  So my argument
> still stands, regardless of whether your *INITIAL* filesystem is ext4fs,
> or ZFS, or initramfs, that *INITIAL* filesystem has to be built into the
> kernel.

Interesting perspective. Of course, support for an initramfs is not
actually a file system (it's not even in the File systems section of
the kernel configuration, is in General setup); it's not possible to
have initramfs as a module (that would make no sense at all); and it's
code that is several orders of magnitude more simpler than the one
used by ext4 (or any other journal file system).

But you are right that for booting with an initramfs, you need
initramfs support.

> Also, I really wonder what the point is in having to use
> initramfs on a system where /usr is part of /.

Well, since some months ago I've been running as a module almost
everything that can be compiled as a module. This allows me to run a
*truly* minimal kernel, and only the necessary modules autoload
automatically (one big exception: binfmt_script, I compiled that into
the kernel because it was not loading automatically). I can also
unload some modules when not in use anymore (and this is great to
debug sometimes).

This also lets me to add a lot of stuff in the kernel, as long as I
add them as modules, without me worrying about bloating my kernel.
Only when they are needed they are loaded. I have USB speakers, but I
almost never use them; no problem, they (like almost everything else)
live as modules, and only are loaded (automagically, thanks to udev)
when needed. And again, I can unload them when not in use.

And also, it turns out that by using dracut+systemd you could boot
faster than without initramfs (although I can't find the link
anymore).

Finally, using only modules and dracut liberates me from thinking what
should it be compiled in and what not; I just put *everything* as a
module, and the kernel, udev and dracut take care of loading what's
necessary. Thus, my kernel (the one running in memory) is as minimal
as it can be, all the time.

Oh, and one more thing; by having everything as a module, if suddenly
I need support for new hardware, usually I can do a quick "make
menuconfig; make modules_install", and the new module can be
modprobe'd into the kernel without needing a reboot. That's
convenient.

Regards.
-- 
Canek Peláez Valdés
Posgrado en Ciencia e Ingeniería de la Computación
Universidad Nacional Autónoma de México

Re: Integrated ZFS for Gentoo - WAS Re: [gentoo-user] Optional /usr merge in Gentoo

[Mark David Dumlao]

On Sun, Sep 1, 2013 at 11:55 AM, Walter Dnes <waltdnes@waltdnes.org> wrote:
>> I usally use ext4 as filesystem.
>>
>> # lsmod|grep ext
>> ext3                  100768  0
>> jbd                    39586  1 ext3
>> ext2                   49572  0
>> ext4                  263621  1
>> crc16                   1255  2 ext4,bluetooth
>> mbcache                 4450  3 ext2,ext3,ext4
>> jbd2                   48679  1 ext4
>>
>> Isn't great what an initramfs can do?
>
>   In this case, initramfs is your root filesystem, from which you load
> another fs and then transfer (pivot root?) to it.  You have to build
> initramfs support into the kernel, to boot an initramfs.  So my argument
> still stands, regardless of whether your *INITIAL* filesystem is ext4fs,
> or ZFS, or initramfs, that *INITIAL* filesystem has to be built into the
> kernel.  Also, I really wonder what the point is in having to use
> initramfs on a system where /usr is part of /.

It allows you to keep some kernel bits in modules. If ever you change your mind
on whether to include / exclude / reconfigure those kernel bits in the
future, your
kernel recompile will take a lot, lot, shorter.

Case in point - do you enable all the ext4 options, like acls and
whatnot? Let's say no.

What if you suddenly have to mount an external hard disk to recover some system
on your server and the hard disk uses those ext4 options? If ext4 is
hard built into your
kernel, your recompile will have to basically redo the whole thing,
whereas if ext4
was a module you would only recompile ext4 itself.
-- 
This email is:    [ ] actionable   [x] fyi        [ ] social
Response needed:  [ ] yes          [ ] up to you  [x] no
Time-sensitive:   [ ] immediate    [ ] soon       [x] none

Re: Integrated ZFS for Gentoo - WAS Re: [gentoo-user] Optional /usr merge in Gentoo

[Joerg Schilling]

Walter Dnes <waltdnes@waltdnes.org> wrote:

>   You can get away with most stuff as modules; ***BUT NOT THE ROOT
> FILESYSTEM***.  Think about it for a minute.  Gentoo reads modules off
> the disk.  If the code for the root filesystem is a module, Gentoo would
> have to read the module off the disk to enable it to read the module off
> the disk... OOPS.  This is a classic "chicken and egg" situation.

On Solaris no problem with loadable modules - everything is dynamically loaded.
You need a grub that understands ZFS and that gives a ZFS interface to the 
kernel to use before ZFS was loaded.

Jörg

-- 
 EMail:joerg@schily.isdn.cs.tu-berlin.de (home) Jörg Schilling D-13353 Berlin
       js@cs.tu-berlin.de                (uni)  
       joerg.schilling@fokus.fraunhofer.de (work) Blog: http://schily.blogspot.com/
 URL:  http://cdrecord.berlios.de/private/ ftp://ftp.berlios.de/pub/schily

Re: Integrated ZFS for Gentoo - WAS Re: [gentoo-user] Optional /usr merge in Gentoo

[Tanstaafl]

On 2013-08-31 7:29 AM, Joerg Schilling 
<Joerg.Schilling@fokus.fraunhofer.de> wrote:
> Tanstaafl<tanstaafl@libertytrek.org>  wrote:
>> You must have missed the point that this is for*servers*, that most
>> people*disable modules*  on. I*know* that it is available as a module.

> Why, for security reasons?

Because if you don't need something, why enable it?

If modules are totally disabled, then there is no worry about any 
security issue involving modules at all.

Re: Integrated ZFS for Gentoo - WAS Re: [gentoo-user] Optional /usr merge in Gentoo

[Tanstaafl]

On 2013-08-31 11:55 PM, Walter Dnes <waltdnes@waltdnes.org> wrote:
> Also, I really wonder what the point is in having to use
> initramfs on a system where /usr is part of /.

You don't, it is only *required* if you have a separate /usr... in fact 
that is what the whole argument was about.

At least that is my understanding of the situation now... please don't 
tell me I'm wrong and there was another vote and it is now required just 
to be able to use gentoo?

Re: Integrated ZFS for Gentoo - WAS Re: [gentoo-user] Optional /usr merge in Gentoo

[Tanstaafl]

On 2013-08-31 7:32 AM, Alon Bar-Lev <alonbl@gentoo.org> wrote:
> If this is not mainline, and it is not trivial gentoo kernels
> maintainer patch, and you must have this as static, you can just put
> the patch within/etc/portage/patches/sys-kernel/gentoo-sources/, so
> it will patch your kernel every time you emerge new one.

Interesting, but this would require manually updating the patch every 
time, right?

Or could the 'patch' be configured to automatically pull the right 
version (compatible with the kernel being installed) every time? That 
would not be such a bad thing... but if not... well...

Computers excel at automating things. People excel at breaking things, 
and I'd like this to be automated as much as possible.

That said, I've never applied patches in this manner, so, is there an up 
to date how-to on how to do this? It might be something I can get 
comfortable with unless/until an automated process is implemented.

On 2013-08-31 8:19 AM, Joerg Schilling wrote:
 > So there seems to be no real need to create a static linux kernel
 > with ZFS inside.

<sigh>

There is for those who *do not want modules enabled on their servers*.

Why is it so hard for some people to just not get that their way is not 
the only way.

Again, Joerg... please *stop arguing* about this point, it has *nothing* 
to do with the thread.

On 2013-08-31 2:44 PM, Mark David Dumlao <madumlao@gmail.com> wrote:
>> You must have missed the point that this is for *servers*, that
>> most people *disable modules* on. I*know* that it is available as a
>> module.

> Ok, I was just asking. But as for what "most people" do on their
> servers, speak for yourself.

Ok, I left out two words: '... I know ... ' - and the fact is, most 
everyone I know (over a dozen) who runs linux servers (not just gentoo) 
runs them with modules disabled, and I've seen countless others say the 
same thing over the years...

The fact is, *many* people do this, and if it trivial to implement it in 
gentoo (which appears it is), then why not do so?

Re: Integrated ZFS for Gentoo - WAS Re: [gentoo-user] Optional /usr merge in Gentoo

[Walter Dnes]

On Sun, Sep 01, 2013 at 09:49:23AM +0200, Joerg Schilling wrote
> Walter Dnes <waltdnes@waltdnes.org> wrote:
> 
> >   You can get away with most stuff as modules; ***BUT NOT THE ROOT
> > FILESYSTEM***.  Think about it for a minute.  Gentoo reads modules off
> > the disk.  If the code for the root filesystem is a module, Gentoo would
> > have to read the module off the disk to enable it to read the module off
> > the disk... OOPS.  This is a classic "chicken and egg" situation.
> 
> On Solaris no problem with loadable modules - everything is
> dynamically loaded.  ***YOU NEED A GRUB THAT UNDERSTANDS ZFS AND THAT
> GIVES A ZFS INTERFACE TO THE KERNEL TO USE BEFORE ZFS WAS LOADED***.

  So instead of needing ZFS built into the kernel, you need ZFS built
into GRUB... ***AND*** you need a ZFS module for the main system...
***AND*** you need to keep both versions in sync.  I'm not impressed.

-- 
Walter Dnes <waltdnes@waltdnes.org>
I don't run "desktop environments"; I run useful applications

Re: Integrated ZFS for Gentoo - WAS Re: [gentoo-user] Optional /usr merge in Gentoo

[Walter Dnes]

On Sun, Sep 01, 2013 at 10:11:01AM -0400, Tanstaafl wrote

> You don't, it is only *required* if you have a separate /usr... in fact 
> that is what the whole argument was about.
> 
> At least that is my understanding of the situation now... please don't 
> tell me I'm wrong and there was another vote and it is now required just 
> to be able to use gentoo?

  This is for the people who want *EVERYTHING INCLUDING THE ROOT FILE
SYSTEM CODE* built as a module.  Note that the Gentoo (AMD64) docs at
http://www.gentoo.org/doc/en/handbook/handbook-amd64.xml?full=1#book_part1_chap7say...
> Don't compile the file system you use for the root filesystem as
> module, otherwise your Gentoo system will not be able to mount
> your partition.

  Using an initramfs allows you to ignore that warning.

-- 
Walter Dnes <waltdnes@waltdnes.org>
I don't run "desktop environments"; I run useful applications

Re: Integrated ZFS for Gentoo - WAS Re: [gentoo-user] Optional /usr merge in Gentoo

[Mark David Dumlao]

[-- Attachment #1: Type: text/plain, Size: 1155 bytes --]

On Sep 2, 2013 5:21 AM, "Walter Dnes" <waltdnes@waltdnes.org> wrote:
>
> On Sun, Sep 01, 2013 at 09:49:23AM +0200, Joerg Schilling wrote
> > Walter Dnes <waltdnes@waltdnes.org> wrote:
> >
> > >   You can get away with most stuff as modules; ***BUT NOT THE ROOT
> > > FILESYSTEM***.  Think about it for a minute.  Gentoo reads modules off
> > > the disk.  If the code for the root filesystem is a module, Gentoo
would
> > > have to read the module off the disk to enable it to read the module
off
> > > the disk... OOPS.  This is a classic "chicken and egg" situation.
> >
> > On Solaris no problem with loadable modules - everything is
> > dynamically loaded.  ***YOU NEED A GRUB THAT UNDERSTANDS ZFS AND THAT
> > GIVES A ZFS INTERFACE TO THE KERNEL TO USE BEFORE ZFS WAS LOADED***.

I'm confused as to what this means. Grub reads a filesystem, loads a kernel
with options, and may give it an initrd. What happens from then on is none
of grub's business. The filesystem it reads from and the one the kernel
uses may be completely unrelated - this is why we have /boot filesystems.

At what point does grub "present a zfs interface for the kernel to use"?

[-- Attachment #2: Type: text/html, Size: 1479 bytes --]

Re: Integrated ZFS for Gentoo - WAS Re: [gentoo-user] Optional /usr merge in Gentoo

[Walter Dnes]

On Sun, Sep 01, 2013 at 01:41:30PM +0800, Mark David Dumlao wrote

> Case in point - do you enable all the ext4 options, like acls and
> whatnot? Let's say no.
> 
> What if you suddenly have to mount an external hard disk to
> recover some system on your server and the hard disk uses those ext4
> options? If ext4 is hard built into your kernel, your recompile will
> have to basically redo the whole thing, whereas if ext4 was a module
> you would only recompile ext4 itself.

  Have you ever actually done this?  I'd be very leery of pulling such a
stunt.  The clean way of switching module versions is to...
* unload the old module, and
* load the new module

  You obviously can't do this in your setup, because unloading the old
module would mean you could no longer access the file system to read in
the new module... OOPS.

  You could run a script that creates /dev/shm/lib/3.1.4.1.5.9-gentoo/
(easy as pie<G>) and copies the new module to that dir.  Then unload the
old module and load the new one, using modprobe with "-d /dev/shm/".

  That still looks impossible.  The problem is that you generally have a
whole bunch of files open at any time.  E.g. try...

lsof -d txt | grep -v "/proc/" | less

...and look at the output.  Shutting down all those open files would
be disastrous.  But that's not what you're saying.  You seem to imply
that file system code can be overwritten *IN PLACE, WHILE IN USE*,
without any problems.  Colour me skeptical about that one.

-- 
Walter Dnes <waltdnes@waltdnes.org>
I don't run "desktop environments"; I run useful applications

Re: Integrated ZFS for Gentoo - WAS Re: [gentoo-user] Optional /usr merge in Gentoo

[Joerg Schilling]

Mark David Dumlao <madumlao@gmail.com> wrote:

> > > > the disk... OOPS.  This is a classic "chicken and egg" situation.
> > >
> > > On Solaris no problem with loadable modules - everything is
> > > dynamically loaded.  ***YOU NEED A GRUB THAT UNDERSTANDS ZFS AND THAT
> > > GIVES A ZFS INTERFACE TO THE KERNEL TO USE BEFORE ZFS WAS LOADED***.
>
> I'm confused as to what this means. Grub reads a filesystem, loads a kernel
> with options, and may give it an initrd. What happens from then on is none
> of grub's business. The filesystem it reads from and the one the kernel
> uses may be completely unrelated - this is why we have /boot filesystems.
>
> At what point does grub "present a zfs interface for the kernel to use"?

After it booted the kernel

You may not know dynamic kernels as Linux is a static kernel that just may load 
additional modules _after_ it mounted the root fs.

Solaris is dynamic from the beginning:

-	no static loading at all

-	no predefined data sizes - everything is allocated

-	no predefined major device numbers - numbers are assigned at first load

Grub works this way:

1)	It loads /platform/i86pc/kernel/$ISADIR/unix

2)	It checks the file "unix" and sees ELF dependencies.

	It loads the ELF dependencies (genunix and dtracestubs) listed
	in the ELF headers from "unix".

3)	It loads /platform/i86pc/$ISADIR/boot_archive

The Kernel then uses the filesystem callbacks in grub to load modules from the 
filesystem in the boot archive.

After the kernel did mount the root filesystem, it switches to the normal 
kernel drivers just loaded and frees the memory space used by grub before.

Jörg

-- 
 EMail:joerg@schily.isdn.cs.tu-berlin.de (home) Jörg Schilling D-13353 Berlin
       js@cs.tu-berlin.de                (uni)  
       joerg.schilling@fokus.fraunhofer.de (work) Blog: http://schily.blogspot.com/
 URL:  http://cdrecord.berlios.de/private/ ftp://ftp.berlios.de/pub/schily

Re: Integrated ZFS for Gentoo - WAS Re: [gentoo-user] Optional /usr merge in Gentoo

[Walter Dnes]

On Mon, Sep 02, 2013 at 10:47:35AM +0200, Joerg Schilling wrote
> Mark David Dumlao <madumlao@gmail.com> wrote:
> > At what point does grub "present a zfs interface for the kernel to use"?
> 
> After it booted the kernel
> 
> You may not know dynamic kernels as Linux is a static kernel that
> just may load additional modules _after_ it mounted the root fs.
> 
> Solaris is dynamic from the beginning:
> 
> -	no static loading at all
> 
> -	no predefined data sizes - everything is allocated
> 
> -	no predefined major device numbers - numbers are assigned at first load
> 
> Grub works this way:
> 
> 1)	It loads /platform/i86pc/kernel/$ISADIR/unix

  Question... how does it read that file off a ZFS partition?  OK, so
ZFS code has to be installed statically into GRUB instead of statically
into the kernel.  Please stop the shell game.

  Note also that this is a Gentoo *LINUX* mailing list.  We're more
concerned about how Linux works.

-- 
Walter Dnes <waltdnes@waltdnes.org>
I don't run "desktop environments"; I run useful applications

Re: Integrated ZFS for Gentoo - WAS Re: [gentoo-user] Optional /usr merge in Gentoo

[Marc Stürmer]

Am 02.09.2013 10:47, schrieb Joerg Schilling:

> Solaris is dynamic from the beginning:

Well in my point of view it boils down to that: someone wants to use ZFS 
on Linux. Fine. This means you've got to be a good citizen and obey its 
license, of course.

It is for those legal reasons that ZFS is not included into the Linux 
kernel mainline source tree. It is also for those reasons you got to 
compile it as a module.

So somebody wants it being static into his kernel, modules being 
disabled on his machine because of security concerns. Unless he is going 
to do that stuff himself this is unlikely to ever happen.

So it boils down to those possible solutions:

a) writing that stuff himself (unlikely to happen),
b) just using the module and going to be happy (also unlikely to happen 
as it seems),
c) choosing another, native file system like Btrfs (which is still yet 
not production ready as a fast moving target) or going with something 
like XFS or Ext4 (and LVM),

or the most natural choice then, which is

d) choosing an operating system, which supports ZFS out of the box like 
FreeBSD and forget about all the rest of the problems.

I would go for d and forget about all of the rest of the problems. 
FreeBSD has been around long enough, and is stable and mature enough for 
most anything you can throw at and it is a nice, clean, well structured 
system anyway.

There's also Gentoo/FreeBSD around, but personally I would use the 
native ports system instead.

Re: Integrated ZFS for Gentoo - WAS Re: [gentoo-user] Optional /usr merge in Gentoo

[Joerg Schilling]

Walter Dnes <waltdnes@waltdnes.org> wrote:

> > Grub works this way:
> > 
> > 1)	It loads /platform/i86pc/kernel/$ISADIR/unix
>
>   Question... how does it read that file off a ZFS partition?  OK, so
> ZFS code has to be installed statically into GRUB instead of statically
> into the kernel.  Please stop the shell game.

Grub was enhanced by Sun to understand ZFS. You need such an enhanced grub if 
you like to boot off ZFS.

>   Note also that this is a Gentoo *LINUX* mailing list.  We're more
> concerned about how Linux works.

Linux does not contain code to boot AFAIK....

Jörg

-- 
 EMail:joerg@schily.isdn.cs.tu-berlin.de (home) Jörg Schilling D-13353 Berlin
       js@cs.tu-berlin.de                (uni)  
       joerg.schilling@fokus.fraunhofer.de (work) Blog: http://schily.blogspot.com/
 URL:  http://cdrecord.berlios.de/private/ ftp://ftp.berlios.de/pub/schily

Re: Integrated ZFS for Gentoo - WAS Re: [gentoo-user] Optional /usr merge in Gentoo

[Mark David Dumlao]

On Mon, Sep 2, 2013 at 4:47 PM, Joerg Schilling
<Joerg.Schilling@fokus.fraunhofer.de> wrote:
> Mark David Dumlao <madumlao@gmail.com> wrote:
>
>> > > > the disk... OOPS.  This is a classic "chicken and egg" situation.
>> > >
>> > > On Solaris no problem with loadable modules - everything is
>> > > dynamically loaded.  ***YOU NEED A GRUB THAT UNDERSTANDS ZFS AND THAT
>> > > GIVES A ZFS INTERFACE TO THE KERNEL TO USE BEFORE ZFS WAS LOADED***.
>>
>> I'm confused as to what this means. Grub reads a filesystem, loads a kernel
>> with options, and may give it an initrd. What happens from then on is none
>> of grub's business. The filesystem it reads from and the one the kernel
>> uses may be completely unrelated - this is why we have /boot filesystems.
>>
>> At what point does grub "present a zfs interface for the kernel to use"?
>
> After it booted the kernel
>
> You may not know dynamic kernels as Linux is a static kernel that just may load
> additional modules _after_ it mounted the root fs.
>
> Solaris is dynamic from the beginning:

Ah I see. But I think by default when we talk about "the kernel" on
this mailing list,
it's assumed that we're talking about Linux. And in the Linux case,
Grub does not do
anything like provide a filesystem interface to Linux. It just loads
the kernel into memory,
and passes it any arguments, like the initrd. So your grub needs to be
able to read the
filesystem containing the kernel and that's it. If the filesystem
containing the kernel is
also a zfs filesystem, then your grub needs a driver that can read
that filesystem.

Well sys-boot/grub-2.00 provides one. See /boot/grub/zfs.mod
-- 
This email is:    [ ] actionable   [x] fyi        [ ] social
Response needed:  [ ] yes          [ ] up to you  [x] no
Time-sensitive:   [ ] immediate    [ ] soon       [x] none

Re: Integrated ZFS for Gentoo - WAS Re: [gentoo-user] Optional /usr merge in Gentoo

[Joerg Schilling]

Mark David Dumlao <madumlao@gmail.com> wrote:

> containing the kernel is
> also a zfs filesystem, then your grub needs a driver that can read
> that filesystem.
>
> Well sys-boot/grub-2.00 provides one. See /boot/grub/zfs.mod

You don't need grub2, a capable older grub does it also, see:

http://hg.berlios.de/repos/schillix-on

for a related source.

Jörg

-- 
 EMail:joerg@schily.isdn.cs.tu-berlin.de (home) Jörg Schilling D-13353 Berlin
       js@cs.tu-berlin.de                (uni)  
       joerg.schilling@fokus.fraunhofer.de (work) Blog: http://schily.blogspot.com/
 URL:  http://cdrecord.berlios.de/private/ ftp://ftp.berlios.de/pub/schily

Re: Integrated ZFS for Gentoo - WAS Re:[gentoo-user] Optional /usr merge in Gentoo

[Nicolas Sebrecht]

The 04/09/13, Joerg Schilling wrote:

> Linux does not contain code to boot AFAIK....

Sure, it does. You can boot on the kernel directly without a boot
manager.

-- 
Nicolas Sebrecht

Re: Integrated ZFS for Gentoo - WAS Re: [gentoo-user] Optional /usr merge in Gentoo

[Tanstaafl]

On 2013-09-04 2:49 AM, Marc Stürmer <mail@marc-stuermer.de> wrote:
> Well in my point of view it boils down to that: someone wants to use ZFS
> on Linux. Fine. This means you've got to be a good citizen and obey its
> license, of course.
>
> It is for those legal reasons that ZFS is not included into the Linux
> kernel mainline source tree. It is also for those reasons you got to
> compile it as a module.

One of the points made was that this is FUD, and that there is NO logel 
reason that it cannot be included.

There is also the fact that it *could* be included, as long as it wasn't 
provided directly in the kernel sources, but as an overlay/patch type 
process, which could still be provided by the gentoo source repositories.