[gentoo-dev] baselayout-1.13 going into ~ARCH soon

[gentoo-dev] baselayout-1.13 going into ~ARCH soon

[Roy Marples]

Hi List!

This is a heads up to say that I'm going to be putting baselayout-1.13 into 
~ARCH soon as all the exciting new features I wanted are in - FreeBSD and 
vserver support, buffered and wrapped einfo/ewarn/eerror output, rc-depend 
for lightning fast dependency sorting, no more critical services, no more net 
service specific code.

So if you're concerned about any of the above features breaking your precious 
Gentoo, now is a very good time to test :)

However, one issue is a concern. All baselayouts defined svcdir 
in /etc/conf.d/rc which defines where we hold the state information of the 
running services. This defaulted to /var/lib/init.d - which is bad as /var 
could be on a different partition.

In 1.13, we've removed the variable from /etc/conf.d/rc and it's now forced 
to /lib/rcscripts/init.d which is safe as /lib is always on the same 
partition as /. The 1.13 ebuild will copy across existing state data, this is 
not the problem. However, downgrading back to 1.12 is a problem as services 
may have been stop, started etc in the middle.

One solution is to ensure that we only hold one copy of the state data and 
move it to the new location. However, this does require altering the stable 
ebuild as well.

Or we could just slap a very large warning on it.

Ideas are welcome :)

-- 
Roy Marples <uberlord@gentoo.org>
Gentoo/Linux/FreeBSD Developer (baselayout, networking)
-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] baselayout-1.13 going into ~ARCH soon

[Piotr Jaroszyński]

> So if you're concerned about any of the above features breaking your
> precious Gentoo, now is a very good time to test :)

Mon Oct  2 22:24:05 2006 >>> sys-apps/baselayout-1.13.0_alpha1-r1
^^ Using 1.13* for over a month and no problems whatsover.

-- 
Piotr Jaroszyński
Gentoo Developer

-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] baselayout-1.13 going into ~ARCH soon

[Bruno]

On Monday 06 November 2006 17:53, Roy Marples wrote:
> ...
>
> However, one issue is a concern. All baselayouts defined svcdir
> in /etc/conf.d/rc which defines where we hold the state information of the
> running services. This defaulted to /var/lib/init.d - which is bad as /var
> could be on a different partition.
>
> In 1.13, we've removed the variable from /etc/conf.d/rc and it's now forced
> to /lib/rcscripts/init.d which is safe as /lib is always on the same
> partition as /. The 1.13 ebuild will copy across existing state data, this
> is not the problem. However, downgrading back to 1.12 is a problem as
> services may have been stop, started etc in the middle.
>
> ...
How is the case where the / partition always remains ro handled? Is rc-state 
information put into a tmpfs partition on that location, is the location 
configured differently for those?

Bruno
-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] baselayout-1.13 going into ~ARCH soon

[Roy Marples]

On Monday 06 November 2006 17:12, Bruno wrote:
> How is the case where the / partition always remains ro handled? Is
> rc-state information put into a tmpfs partition on that location, is the
> location configured differently for those?

Good question!

/ is always ro at boot and the checkroot init script remounts it rw after 
checking it.

As such, we mount /lib/rcscripts/init.d/ as tmpfs (or ramfs or similar). Yes, 
this means that the kernel must be configured with a ramdisk or similar 
OR /lib/rscripts/init.d is writeable.

At the end of the rc process (ie end of boot runlevel) we tar up our state 
dir, unmount it and untar it back (taking good care with locking due to 
parallel starts and event driven services).

Admittedly, an always ro / isn't handled right now, but I'll ensure it will be 
for the next release :)

Thanks

-- 
Roy Marples <uberlord@gentoo.org>
Gentoo/Linux/FreeBSD Developer (baselayout, networking)
-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] baselayout-1.13 going into ~ARCH soon

[Roy Marples]

[-- Attachment #1: Type: text/plain, Size: 335 bytes --]

On Monday 06 November 2006 17:33, Roy Marples wrote:
> Admittedly, an always ro / isn't handled right now, but I'll ensure it will
> be for the next release :)

We handle it with the attached patch, just comitted to our svn repo :)

Thanks

-- 
Roy Marples <uberlord@gentoo.org>
Gentoo/Linux/FreeBSD Developer (baselayout, networking)

[-- Attachment #2: roroot.patch --]
[-- Type: text/x-diff, Size: 1083 bytes --]

Index: ChangeLog
===================================================================
--- ChangeLog	(revision 2364)
+++ ChangeLog	(working copy)
@@ -1,6 +1,11 @@
 # ChangeLog for Gentoo System Intialization ("rc") scripts
 # Copyright 1999-2006 Gentoo Foundation; Distributed under the GPLv2
 
+  06 Nov 2006; Roy Marples <uberlord@gentoo.org>:
+
+    Ensure that we only move $svcdir from tmpfs to disk if $svclib is rw.
+    Thanks to Bruno for the idea.
+
   05 Nov 2006; Roy Marples <uberlord@gentoo.org>:
 
     Don't create /proc on Linux as build scripts like to handle it now.
Index: sbin/rc
===================================================================
--- sbin/rc	(revision 2364)
+++ sbin/rc	(working copy)
@@ -442,7 +442,8 @@
 rm -rf "${svcdir}/failed" &>/dev/null
 
 # If $svcdir is mounted in ram, save it back to disk and unmount
-if [[ $'\n'$(get_mounts) =~ $'\n'${svcdir}\  ]] ; then
+# but only if $svclib is writeable.
+if [[ $'\n'$(get_mounts) =~ $'\n'${svcdir}\  && -w ${svclib} ]] ; then
 	# Function to show the timeout message
 	timeout=
 	do_timeout() {

Re: [gentoo-dev] baselayout-1.13 going into ~ARCH soon

[Francesco Riosa]

Roy Marples wrote:
[snip that change the meaning of the message ;]
> 
> Ideas are welcome :)
> 

need to jump net.lo in symlink tests fex as tested below:

for f in ${ROOT}etc/init.d/net.*; do
   [[ "${f}" == "${ROOT}etc/init.d/net.lo" || -L ${f} ]] && continue
   echo
   einfo "WARNING: You have older net.* files in ${ROOT}etc/init.d/"
[...]

there is a bug where to report gotchas ?
-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] baselayout-1.13 going into ~ARCH soon

[Roy Marples]

On Monday 06 November 2006 17:51, Francesco Riosa wrote:
> Roy Marples wrote:
> [snip that change the meaning of the message ;]
>
> > Ideas are welcome :)
>
> need to jump net.lo in symlink tests fex as tested below:
>
> for f in ${ROOT}etc/init.d/net.*; do
>    [[ "${f}" == "${ROOT}etc/init.d/net.lo" || -L ${f} ]] && continue
>    echo
>    einfo "WARNING: You have older net.* files in ${ROOT}etc/init.d/"
> [...]

No I don't - net.lo is now a symlink itself to /lib/rcscripts/sh/net.sh
This solves the issue of users not etc-updating net.lo, and then getting loads 
of errors. Which happens all to frequently.

>
> there is a bug where to report gotchas ?

Nope. Just file a normal bug.

-- 
Roy Marples <uberlord@gentoo.org>
Gentoo/Linux/FreeBSD Developer (baselayout, networking)
-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] baselayout-1.13 going into ~ARCH soon

[Matthew Snelham]

On 06 Nov 2006 04:53 PM or thereabouts, Roy Marples wrote:
> This is a heads up to say that I'm going to be putting baselayout-1.13 into 
> ~ARCH soon as all the exciting new features I wanted are in - FreeBSD and 
> vserver support, buffered and wrapped einfo/ewarn/eerror output, rc-depend 
> for lightning fast dependency sorting, no more critical services, no more net 
> service specific code.

Very nice!
 
> However, one issue is a concern. All baselayouts defined svcdir 
> in /etc/conf.d/rc which defines where we hold the state information of the 
> running services. This defaulted to /var/lib/init.d - which is bad as /var 
> could be on a different partition.
> 
> In 1.13, we've removed the variable from /etc/conf.d/rc and it's now forced 
> to /lib/rcscripts/init.d which is safe as /lib is always on the same 
> partition as /. 

>From a filesystem usage point of view though, storing actively changing
state data on /lib is ugly.  The tmpfs /lib/rcscripts/init.d overlay
solution for a ro / works, but as long as tmpfs magic is needed, can't it
be written to /var after localmount? 

--Matthew
zeypher@gentoo.org

-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] baselayout-1.13 going into ~ARCH soon

[Roy Marples]

On Monday 06 November 2006 18:27, Matthew Snelham wrote:
> > In 1.13, we've removed the variable from /etc/conf.d/rc and it's now
> > forced to /lib/rcscripts/init.d which is safe as /lib is always on the
> > same partition as /.
>
> From a filesystem usage point of view though, storing actively changing
> state data on /lib is ugly.  The tmpfs /lib/rcscripts/init.d overlay
> solution for a ro / works, but as long as tmpfs magic is needed, can't it
> be written to /var after localmount?

It could be written to /var at the end of an rc.
However, you then have to guarantee that /var is always available too and that 
may not always be the case as some people want to have /var net mounted or 
something equally silly :)

/lib may be ugly, but it's also guaranteed which is what I'm more interested 
in.

Thanks

-- 
Roy Marples <uberlord@gentoo.org>
Gentoo Developer (baselayout, networking)
-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] baselayout-1.13 going into ~ARCH soon

[Roy Marples]

On Monday 06 November 2006 22:06, Alec Warner wrote:
> Roy Marples wrote:
> > On Monday 06 November 2006 18:27, Matthew Snelham wrote:
> >>> In 1.13, we've removed the variable from /etc/conf.d/rc and it's now
> >>> forced to /lib/rcscripts/init.d which is safe as /lib is always on the
> >>> same partition as /.
> >>
> >> From a filesystem usage point of view though, storing actively changing
> >> state data on /lib is ugly.  The tmpfs /lib/rcscripts/init.d overlay
> >> solution for a ro / works, but as long as tmpfs magic is needed, can't
> >> it be written to /var after localmount?
> >
> > It could be written to /var at the end of an rc.
> > However, you then have to guarantee that /var is always available too and
> > that may not always be the case as some people want to have /var net
> > mounted or something equally silly :)
> >
> > /lib may be ugly, but it's also guaranteed which is what I'm more
> > interested in.
> >
> > Thanks
>
> This screams "vustomizable"
>
> Just give us a variable we can set to move it; obviously there is no
> "One True Location" for everyone.

If you want that level of flexability then simply symlink /lib/rcscripts 
to /var/rcscripts or where-ever you like.

>From my perspective, state data always has to be available and writeable, 
regardless of how simple or fancy the user has configured their layout.
So this means I have access to /bin, /dev, /etc, /lib, /sbin as those 
directories have to exist on /.

Why do we have to have everything configured by variables?

-- 
Roy Marples <uberlord@gentoo.org>
Gentoo Developer (baselayout, networking)

-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] baselayout-1.13 going into ~ARCH soon

[Alec Warner]

Roy Marples wrote:
> On Monday 06 November 2006 18:27, Matthew Snelham wrote:
>>> In 1.13, we've removed the variable from /etc/conf.d/rc and it's now
>>> forced to /lib/rcscripts/init.d which is safe as /lib is always on the
>>> same partition as /.
>> From a filesystem usage point of view though, storing actively changing
>> state data on /lib is ugly.  The tmpfs /lib/rcscripts/init.d overlay
>> solution for a ro / works, but as long as tmpfs magic is needed, can't it
>> be written to /var after localmount?
> 
> It could be written to /var at the end of an rc.
> However, you then have to guarantee that /var is always available too and that 
> may not always be the case as some people want to have /var net mounted or 
> something equally silly :)
> 
> /lib may be ugly, but it's also guaranteed which is what I'm more interested 
> in.
> 
> Thanks
> 

This screams "vustomizable"

Just give us a variable we can set to move it; obviously there is no 
"One True Location" for everyone.
-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] baselayout-1.13 going into ~ARCH soon

[Matthew Snelham]

On 06 Nov 2006 09:57 PM or thereabouts, Roy Marples wrote:
> On Monday 06 November 2006 22:06, Alec Warner wrote:
> > Roy Marples wrote:
> > > On Monday 06 November 2006 18:27, Matthew Snelham wrote:
> > >> From a filesystem usage point of view though, storing actively changing
> > >> state data on /lib is ugly.  The tmpfs /lib/rcscripts/init.d overlay
> > >> solution for a ro / works, but as long as tmpfs magic is needed, can't
> > >> it be written to /var after localmount?
> > >
> > > It could be written to /var at the end of an rc.
> > > However, you then have to guarantee that /var is always available too and
> > > that may not always be the case as some people want to have /var net
> > > mounted or something equally silly :)
> > >
> > > /lib may be ugly, but it's also guaranteed which is what I'm more
> > > interested in.
> >
> > This screams "vustomizable"
> >
> > Just give us a variable we can set to move it; obviously there is no
> > "One True Location" for everyone.
 
> If you want that level of flexability then simply symlink /lib/rcscripts 
> to /var/rcscripts or where-ever you like.

But then baselayout is still 'behaving badly' by sttempting to store
dynamic state information in /lib.  Something it has not done before, to
the best of my knowledge (with the exception of /dev state tarballs, which
are generally acceptable, since they don't change while the system is up).

UNIX filesystem usage patterns are older than a good chunk of gentoo devs,
so in the name of defaulting to expected behaviour, I think /lib should be
avoided.

> From my perspective, state data always has to be available and writeable, 
> regardless of how simple or fancy the user has configured their layout.
> So this means I have access to /bin, /dev, /etc, /lib, /sbin as those 
> directories have to exist on /.

/usr and /var have to be accessable for a working system... and if init
fails before those are availible, the system is non-functional regardless. 

So a tmp storage location is needed for state data early in the boot
process (before writeable filesystems can be assured), and can flip before
the boot runlevel is completed.  

(I've built a number of clusters with NFS root fs, but I've never even
heard of a disk backed root with an NFS /var.  Can we say that's
pathologically odd, and unsupported/unsupportable?)

> Why do we have to have everything configured by variables?

Eh, I'm not sure I see the need for this to be a variable.  I'd just like
to see it well behaved. 

--Matthew
zeypher@gentoo.org

-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] baselayout-1.13 going into ~ARCH soon

[Patrick McLean]

Matthew Snelham wrote:
>  
>> If you want that level of flexability then simply symlink /lib/rcscripts 
>> to /var/rcscripts or where-ever you like.
> 
> But then baselayout is still 'behaving badly' by sttempting to store
> dynamic state information in /lib.  Something it has not done before, to
> the best of my knowledge (with the exception of /dev state tarballs, which
> are generally acceptable, since they don't change while the system is up).
> 
> UNIX filesystem usage patterns are older than a good chunk of gentoo devs,
> so in the name of defaulting to expected behaviour, I think /lib should be
> avoided.

+1

This is a very good point, why are we breaking from accepted UNIX standards
uselessly? Generally, a live system should never need to write to /lib, but a
writable /var is pretty much standard. This new behavior breaks standards, if
/var is on a separate filesystem, maybe we can use a subdir in /tmp for the init
stuff until we get /var up, then move it over.
-- 
gentoo-dev@gentoo.org mailing list

[gentoo-dev] Re: baselayout-1.13 going into ~ARCH soon

[Sven Köhler]

[-- Attachment #1: Type: text/plain, Size: 724 bytes --]

>> In 1.13, we've removed the variable from /etc/conf.d/rc and it's now forced 
>> to /lib/rcscripts/init.d which is safe as /lib is always on the same 
>> partition as /. 
> 
> From a filesystem usage point of view though, storing actively changing
> state data on /lib is ugly.  The tmpfs /lib/rcscripts/init.d overlay
> solution for a ro / works, but as long as tmpfs magic is needed, can't it
> be written to /var after localmount? 

After reading all the concerns and doubt and things, i ask myself:

why not keep in a tmpfs?

Well, it can be swapped out too, and it isn't too much data anyway, is it?


The most disturbing thing to me, would be yet another entry when i watch
the list mountpoints.


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 249 bytes --]

Re: [gentoo-dev] baselayout-1.13 going into ~ARCH soon

[Mike Frysinger]

[-- Attachment #1: Type: text/plain, Size: 269 bytes --]

On Monday 06 November 2006 21:42, Matthew Snelham wrote:
> But then baselayout is still 'behaving badly' by sttempting to store
> dynamic state information in /lib.

it is and it isnt ... the dir is memory based so /lib could be read-only and 
that's fine
-mike

[-- Attachment #2: Type: application/pgp-signature, Size: 827 bytes --]

Re: [gentoo-dev] baselayout-1.13 going into ~ARCH soon

[Mike Frysinger]

[-- Attachment #1: Type: text/plain, Size: 266 bytes --]

On Monday 06 November 2006 22:22, Patrick McLean wrote:
> This is a very good point, why are we breaking from accepted UNIX standards
> uselessly?

did you even read the thread ?  the reasons listed certainly do not fall under 
the "uselessly" category
-mike

[-- Attachment #2: Type: application/pgp-signature, Size: 827 bytes --]

Re: [gentoo-dev] baselayout-1.13 going into ~ARCH soon

[Josh Saddler]

[-- Attachment #1: Type: text/plain, Size: 1367 bytes --]

Patrick McLean wrote:
> Matthew Snelham wrote:
>>  
>>> If you want that level of flexability then simply symlink /lib/rcscripts 
>>> to /var/rcscripts or where-ever you like.
>> But then baselayout is still 'behaving badly' by sttempting to store
>> dynamic state information in /lib.  Something it has not done before, to
>> the best of my knowledge (with the exception of /dev state tarballs, which
>> are generally acceptable, since they don't change while the system is up).
>>
>> UNIX filesystem usage patterns are older than a good chunk of gentoo devs,
>> so in the name of defaulting to expected behaviour, I think /lib should be
>> avoided.
> 
> +1
> 
> This is a very good point, why are we breaking from accepted UNIX standards
> uselessly? Generally, a live system should never need to write to /lib, but a
> writable /var is pretty much standard. This new behavior breaks standards, if
> /var is on a separate filesystem, maybe we can use a subdir in /tmp for the init
> stuff until we get /var up, then move it over.
Agreed, this is a good point. Writing to /lib will also cause security
complications for things like AIDE and other intrusion detection systems
that look for writes to certain directories. If they see /lib changing
quite often, it might confuse 'em and the sysadmin, who will get a rash
of spurious alerts.


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] Re: baselayout-1.13 going into ~ARCH soon

[Roy Marples]

On Tuesday 07 November 2006 03:23, Sven Köhler wrote:
> After reading all the concerns and doubt and things, i ask myself:
>
> why not keep in a tmpfs?
>
> Well, it can be swapped out too, and it isn't too much data anyway, is it?

Only linux has a non specific tmpfs - ie it just uses what it needs.
For the BSD's it has something similar, but you have to specify either it's 
size or it's maximum size so it's not as flexable.

> The most disturbing thing to me, would be yet another entry when i watch
> the list mountpoints.

One of the reasons I'm keen on rolling it back to disk after rc completes the 
boot runlevel.

-- 
Roy Marples <uberlord@gentoo.org>
Gentoo/Linux Developer (baselayout, networking)

-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] baselayout-1.13 going into ~ARCH soon

[Roy Marples]

On Tuesday 07 November 2006 04:06, Josh Saddler wrote:
> Agreed, this is a good point. Writing to /lib will also cause security
> complications for things like AIDE and other intrusion detection systems
> that look for writes to certain directories. If they see /lib changing
> quite often, it might confuse 'em and the sysadmin, who will get a rash
> of spurious alerts.

Hmmmm? It would only be updated when a service starts or stops or a user 
updates their config, which would imply a restart I suppose.

The state data is generally only written to when starting up and shutting 
down - in the meantime it should be minimal.

-- 
Roy Marples <uberlord@gentoo.org>
Gentoo/Linux Developer (baselayout, networking)
-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] baselayout-1.13 going into ~ARCH soon

[Roy Marples]

On Tuesday 07 November 2006 02:42, Matthew Snelham wrote:
> (I've built a number of clusters with NFS root fs, but I've never even
> heard of a disk backed root with an NFS /var.  Can we say that's
> pathologically odd, and unsupported/unsupportable?)

OK, I have /var mounted on an LVM. I need to run an fsck on it, so I unmount 
it and do the stuff I need to in single user mode.

In baselayout-1.13 I can bring the entire system back up by going back to the 
default runlevel (although there is one error in checkfs I have to address).

Now how can we do that if our state data no longer exists as /var is not 
available? Yes the user could remount it and it's back, but what happens if 
this is a laptop and the user plugs in their wifi card and they urgently need 
to get network running and there's a problem mounting /var?

> > Why do we have to have everything configured by variables?
>
> Eh, I'm not sure I see the need for this to be a variable.  I'd just like
> to see it well behaved.

Well behaved as to the LFS or well behaved as in coping for as many scenarios 
as we can? I'm all for the later. If you want the former then you're welcome 
to suggest alternative fixes for this too :)

-- 
Roy Marples <uberlord@gentoo.org>
Gentoo/Linux Developer (baselayout, networking)
-- 
gentoo-dev@gentoo.org mailing list

[gentoo-dev] Re: baselayout-1.13 going into ~ARCH soon

[Duncan]

Roy Marples <uberlord@gentoo.org> posted
200611070713.53479.uberlord@gentoo.org, excerpted below, on  Tue, 07 Nov
2006 07:13:53 +0000:

> On Tuesday 07 November 2006 03:23, Sven Köhler wrote:
>> After reading all the concerns and doubt and things, i ask myself:
>>
>> why not keep in a tmpfs?
>>
>> Well, it can be swapped out too, and it isn't too much data anyway, is it?
> 
> Only linux has a non specific tmpfs - ie it just uses what it needs.
> For the BSD's it has something similar, but you have to specify either it's 
> size or it's maximum size so it's not as flexable.

Getting a bit worried by comments so far.  You ARE planning to keep the
OPTION of keeping a tmpfs (or whatever) mounted svcdir, right (an option
to keep it mounted that way after the boot level, is how I guess it'd
work)? I'm using that now and hope to keep it. I went with the suggested
size=2m (tmpfs). df says 184KB used, so that's quite big enough and then
some, but on Linux the free space isn't actually allocated until it's no
longer free space, so no matter. Are you saying the BSDs would allocate
and therefore remove from further use the full 2MB, no way around it, even
if only 148KB is actually used?

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman

-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Re: baselayout-1.13 going into ~ARCH soon

[Roy Marples]

On Tuesday 07 November 2006 09:49, Duncan wrote:
> Getting a bit worried by comments so far.  You ARE planning to keep the
> OPTION of keeping a tmpfs (or whatever) mounted svcdir, right (an option
> to keep it mounted that way after the boot level, is how I guess it'd
> work)?

Actually we mount it in the sysinit runlevel

> I'm using that now and hope to keep it. I went with the suggested 
> size=2m (tmpfs). df says 184KB used, so that's quite big enough and then
> some, but on Linux the free space isn't actually allocated until it's no
> longer free space, so no matter. Are you saying the BSDs would allocate
> and therefore remove from further use the full 2MB, no way around it, even
> if only 148KB is actually used?

Which demonstrates that you don't know about tmpfs as you don't specify any 
size for it - it just uses what it needs.

And yes, the BSD's will use the full 2MB.

Actually I forgot about the option allowing you todo this. It will be 
supported in the next version, although the variable name specified 
in /etc/conf.d/rc will change to match out existing RC_ names, the old one 
will still work.

Thanks

-- 
Roy Marples <uberlord@gentoo.org>
Gentoo/Linux/FreeBSD Developer (baselayout, networking)
-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Re: baselayout-1.13 going into ~ARCH soon

[Roy Marples]

On Tuesday 07 November 2006 10:50, Roy Marples wrote:
> > I'm using that now and hope to keep it. I went with the suggested
> > size=2m (tmpfs). df says 184KB used, so that's quite big enough and then
> > some, but on Linux the free space isn't actually allocated until it's no
> > longer free space, so no matter. Are you saying the BSDs would allocate
> > and therefore remove from further use the full 2MB, no way around it,
> > even if only 148KB is actually used?
>
> Which demonstrates that you don't know about tmpfs as you don't specify any
> size for it - it just uses what it needs.
>
> And yes, the BSD's will use the full 2MB.
>
> Actually I forgot about the option allowing you todo this. It will be
> supported in the next version, although the variable name specified
> in /etc/conf.d/rc will change to match out existing RC_ names, the old one
> will still work.

Actually, before I do that, let me attack this from another angle.
What do you gain from keeping it mounted as a ramdisk?
If the answer is performance, well you loose performance at start time as 
you've lost the deptree.

So why would you want to keep it?

-- 
Roy Marples <uberlord@gentoo.org>
Gentoo/Linux/FreeBSD Developer (baselayout, networking)
-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Re: baselayout-1.13 going into ~ARCH soon

[Andrew Gaffney]

Roy Marples wrote:
> Actually, before I do that, let me attack this from another angle.
> What do you gain from keeping it mounted as a ramdisk?
> If the answer is performance, well you loose performance at start time as 
> you've lost the deptree.
> 
> So why would you want to keep it?

read-only nfsroot or any other root that needs to stay read-only

-- 
Andrew Gaffney                            http://dev.gentoo.org/~agaffney/
Gentoo Linux Developer                                   Installer Project
Today's lesson in political correctness:      "Go asphyxiate on a phallus"
-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Re: baselayout-1.13 going into ~ARCH soon

[Roy Marples]

On Tuesday 07 November 2006 12:29, Andrew Gaffney wrote:
> Roy Marples wrote:
> > Actually, before I do that, let me attack this from another angle.
> > What do you gain from keeping it mounted as a ramdisk?
> > If the answer is performance, well you loose performance at start time as
> > you've lost the deptree.
> >
> > So why would you want to keep it?
>
> read-only nfsroot or any other root that needs to stay read-only

We don't unmount it if root is ro (well, /lib/rcscripts, but it's the same 
thing really).

This is another argument to "keeping it simple and keeping $svcdir always 
mounted in a ramdisk" thread I've started - you may want to chime in on that.

-- 
Roy Marples <uberlord@gentoo.org>
Gentoo Developer (baselayout, networking)
-- 
gentoo-dev@gentoo.org mailing list

[gentoo-dev] Re: baselayout-1.13 going into ~ARCH soon

[Duncan]

Roy Marples <uberlord@gentoo.org> posted
200611071050.21313.uberlord@gentoo.org, excerpted below, on  Tue, 07 Nov
2006 10:50:21 +0000:

>> I'm using that now and hope to keep it. I went with the suggested 
>> size=2m (tmpfs). df says 184KB used, so that's quite big enough and then
>> some, but on Linux the free space isn't actually allocated until it's no
>> longer free space, so no matter. Are you saying the BSDs would allocate
>> and therefore remove from further use the full 2MB, no way around it, even
>> if only 148KB is actually used?
> 
> Which demonstrates that you don't know about tmpfs as you don't specify any 
> size for it - it just uses what it needs.

I think we talked past each other on this. =8^(  tmpfs uses what it needs,
yes, but the size specified is the max-size, according to mount's manpage.
In fact, I believe it was you that pointed out to me back during the
parallel thread on 1.12 or whatever that one of the advantages of tmpfs
over the "lighter" memory-fs types was precisely that -- that it allowed
such specifications, as I was using the lighter versions at the time, but
ultimately "saw the light" and decided that full tmpfs, with a size-limit,
was a bit more sensible in a scenario of restricting a run-away process
eating up all available file (and therefore memory if ramfs without such a
limit) space, if it's unrestricted.

BTW, having just looked it up, tmpfs default size is half of memory,
according to the mount manpage.  I hadn't realized/remembered that, and
had thought it was basically unlimited if unspecified, so I'm glad you
prompted me to look it up again.  =8^)  Anyway, I always set a (max) size
on my tmpfs mounts, 2m on svcdir as I said, 50m on /dev/shm (which has
PORTAGE_TMPFS pointed at it), and 5g on /tmp (out of 8 gig total real
memory so even in a runaway scenario I'd have 3 gig of real memory
reserved for system use, PORTAGE_TMPDIR and PKG_TMPDIR pointed at this one).

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman

-- 
gentoo-dev@gentoo.org mailing list

[gentoo-dev] Re: baselayout-1.13 going into ~ARCH soon

[Duncan]

Roy Marples <uberlord@gentoo.org> posted
200611071058.54068.uberlord@gentoo.org, excerpted below, on  Tue, 07 Nov
2006 10:58:53 +0000:

> Actually, before I do that, let me attack this from another angle.
> What do you gain from keeping it mounted as a ramdisk?
> If the answer is performance, well you loose performance at start time as 
> you've lost the deptree.
> 
> So why would you want to keep it?

Well, as you point out it's mixed from a performance angle.  One reason
I'm beginning to appreciate a bit more now that I have a tmpfs mounted
/tmp as well, however, is the clean start at every boot thing. 
Eliminating even the possibility of trouble from stale or corrupted
carryover has its own >0 benefits.

Back to performance, however, the deptree rebuild isn't as much of a
pull-down as one might expect, for a number of reasons.  Keep in mind that
perhaps the biggest boot-time bottleneck is disk read bandwidth anyway. 
Preloading all those scripts effectively at once therefore has three
overlapping effects.  1)  There's the direct disk read bandwidth, but that
would come in somewhere anyway when the scripts are actually executed.  2)
Having them all loaded at once gives the filesystem elevator scheduler a
bigger chance to do its thing, ordering the reads appropriately for better
efficency.  3) Once it comes time to actually run the scripts, they are
already loaded into cache, so no waiting to read them in then, simply
virtually instant execution.

Thus, while the performance benefit won't be great, it's likely to be
there, and there's the "system hygiene" (I believe that's a decently
accurate label) benefit as well.

So, keeping it at minimum an option is IMO a good thing.  If as proposed
we make it standard, there are other benefits including serious code
simplification as well.  There could well be negatives, especially on
certain archs I'm not familiar with, but for x86, amd64, and probably the
ppc folks, at least, it sounds very reasonable from my perspective.  From
what I've read, the particular challenges faced on MIPS may change the
time effects greatly there (basing this on remarks I recall from the
split-KDE debate, but I haven't the foggiest whether that applies here
or not), so I'd be interested in seeing someone address it from their
perspective.

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman

-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] baselayout-1.13 going into ~ARCH soon

[Aron Griffis]

Roy Marples wrote:  [Mon Nov 06 2006, 12:44:42PM EST]
> +if [[ $'\n'$(get_mounts) =~ $'\n'${svcdir}\  && -w ${svclib} ]] ; then

Shouldn't this be:

    if [[ $'\n'$(get_mounts) == $'\n'"${svcdir} " ...

because I don't think you want to treat the RHS as either a regex (=~)
or a glob (unquoted).

Aron
-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] baselayout-1.13 going into ~ARCH soon

[Roy Marples]

On Tuesday 07 November 2006 15:59, Aron Griffis wrote:
> > +if [[ $'\n'$(get_mounts) =~ $'\n'${svcdir}\  && -w ${svclib} ]] ; then
>
> Shouldn't this be:
>
>     if [[ $'\n'$(get_mounts) == $'\n'"${svcdir} " ...
>
> because I don't think you want to treat the RHS as either a regex (=~)
> or a glob (unquoted).

Needs to be regex so I can match $'\n' as iirc you loose that in globbing

-- 
Roy Marples <uberlord@gentoo.org>
Gentoo/Linux/FreeBSD Developer (baselayout, networking)
-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Re: baselayout-1.13 going into ~ARCH soon

[Mike Frysinger]

[-- Attachment #1: Type: text/plain, Size: 136 bytes --]

On Tuesday 07 November 2006 04:49, Duncan wrote:
> Getting a bit worried by comments so far.

sorry, but this just stinks of lame
-mike

[-- Attachment #2: Type: application/pgp-signature, Size: 827 bytes --]

Re: [gentoo-dev] baselayout-1.13 going into ~ARCH soon

[Drake Wyrm]

[-- Attachment #1: Type: text/plain, Size: 1105 bytes --]

Roy Marples <uberlord@gentoo.org> wrote:
> On Tuesday 07 November 2006 15:59, Aron Griffis wrote:
> > > +if [[ $'\n'$(get_mounts) =~ $'\n'${svcdir}\  && -w ${svclib} ]] ; then
> >
> > Shouldn't this be:
> >
> >     if [[ $'\n'$(get_mounts) == $'\n'"${svcdir} " ...
> >
> > because I don't think you want to treat the RHS as either a regex
> > (=~) or a glob (unquoted).
> 
> Needs to be regex so I can match $'\n' as iirc you loose that in
> globbing

I could be missing something, but:

[[ $'\nwombat' =~ $'wombat' ]] && \
echo "These compare as equal, with or without the leading \n"

They do not compare as equal with the == operator or the = operator. You
probably want the = operator, because the == operator _does_ interpret
the RHS as a glob. The = operator just uses simple string comparison,
without interpreting anything.



I am curious about why the space was included at the end of the in the
expression:

> > > $'\n'${svcdir}\ 

Does get_mounts add a space to the end of its output?

-- 
Every absurdity has a champion to defend it.
  -- Oliver Goldsmith

[-- Attachment #2: Type: application/pgp-signature, Size: 206 bytes --]

Re: [gentoo-dev] baselayout-1.13 going into ~ARCH soon

[Roy Marples]

On Tuesday 07 November 2006 20:32, Drake Wyrm wrote:
> I could be missing something, but:
>
> [[ $'\nwombat' =~ $'wombat' ]] && \
> echo "These compare as equal, with or without the leading \n"

A working example in bash-3.2 :)

[[ $(</proc/mounts) =~ $'\n'/dev/root\  ]] \
&& echo "Yay, I matched ^/dev/root "

I challenge you to get an exact match of /dev/root being on the first line 
using the == operator and/or quoting.
Remember, /dev/root/foo and /dev/foo /dev/root must not match either.

If you can you get credit in the ChangeLog :)

Yes, you could also write
grep -E "^/dev/root " /proc/mounts, but using bash like this is faster [1]

[1] http://roy.marples.name/node/267

Thanks

-- 
Roy Marples <uberlord@gentoo.org>
Gentoo/Linux Developer (baselayout, networking)
-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] baselayout-1.13 going into ~ARCH soon

[Roy Marples]

On Tuesday 07 November 2006 20:32, Drake Wyrm wrote:
> I could be missing something, but:
>
> [[ $'\nwombat' =~ $'wombat' ]] && \
> echo "These compare as equal, with or without the leading \n"
>
> They do not compare as equal with the == operator or the = operator. You
> probably want the = operator, because the == operator _does_ interpret
> the RHS as a glob. The = operator just uses simple string comparison,
> without interpreting anything.

Interesting. You just asked the regex(3) command if $'\nwombat' contains 
$'wombat'

Maybe you meant to write

[[ $'\n'wombat =~ wombat ]]
That will still return true

[[ wombat =~ $'\n'wombat ]]
Will return false, which is what we want as wombat isn't at the start of the 
line.

In my example, I'm trying to match something at the start of a line.

-- 
Roy Marples <uberlord@gentoo.org>
Gentoo/Linux Developer (baselayout, networking)
-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] baselayout-1.13 going into ~ARCH soon

[Francesco Riosa]

Roy Marples ha scritto:
> 
> [[ $(</proc/mounts) =~ $'\n'/dev/root\  ]] \
> && echo "Yay, I matched ^/dev/root "
> 
> I challenge you to get an exact match of /dev/root being on the first line 
> using the == operator and/or quoting.
> Remember, /dev/root/foo and /dev/foo /dev/root must not match either.

mounts=( $(</proc/mounts) )
for m in 0 6 12 ; do
  [[ "${mounts[${m}]}" == "/dev/root" ]] \
  && echo "Yay, I matched ^/dev/root "
done

don't know which is faster/safer (bet on ~=)
-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] baselayout-1.13 going into ~ARCH soon

[Roy Marples]

On Tuesday 07 November 2006 21:14, Francesco Riosa wrote:
> mounts=( $(</proc/mounts) )
> for m in 0 6 12 ; do
>   [[ "${mounts[${m}]}" == "/dev/root" ]] \
>   && echo "Yay, I matched ^/dev/root "
> done

Good, but you fail due to only matching 3 lines

mounts=( $(</proc/mounts) )
i=0
while [[ -n ${mounts[i]} ]] ; do
  [[ "${mounts[${m}]}" == "/dev/root" ]] \
  && echo "Yay, I matched ^/dev/root "
  ((i+=6))
done

Should work (typed into mail, not actually tested). However, this does rely on 
a fixed number of 6 words per line, otherwise it breaks ;)
But as you say,  =~ would be faster as bash is notorious for slow loops

-- 
Roy Marples <uberlord@gentoo.org>
Gentoo/Linux Developer (baselayout, networking)
-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] baselayout-1.13 going into ~ARCH soon

[Harald van Dijk]

On Tue, Nov 07, 2006 at 08:47:18PM +0000, Roy Marples wrote:
> On Tuesday 07 November 2006 20:32, Drake Wyrm wrote:
> > I could be missing something, but:
> >
> > [[ $'\nwombat' =~ $'wombat' ]] && \
> > echo "These compare as equal, with or without the leading \n"
> 
> A working example in bash-3.2 :)
> 
> [[ $(</proc/mounts) =~ $'\n'/dev/root\  ]] \
> && echo "Yay, I matched ^/dev/root "
> 
> I challenge you to get an exact match of /dev/root being on the first line 

You mean being first on a line, right?

> using the == operator and/or quoting.
> Remember, /dev/root/foo and /dev/foo /dev/root must not match either.
> 
> If you can you get credit in the ChangeLog :)

[[ "
$(</proc/mounts)" == *"
/dev/root "* ]] && echo "Yay, I matched ^/dev/root"

You can use $'\n' instead of actual newline characters, of course.
-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] baselayout-1.13 going into ~ARCH soon

[Roy Marples]

On Wednesday 08 November 2006 06:57, Harald van Dijk wrote:
> [[ "
> $(</proc/mounts)" == *"
> /dev/root "* ]] && echo "Yay, I matched ^/dev/root"
>
> You can use $'\n' instead of actual newline characters, of course.

You get credit :)

I couldn't get that to work with bash-3.1 for some reason though, as I prefer 
== over =~ myself.

Thanks

-- 
Roy Marples <uberlord@gentoo.org>
Gentoo/Linux Developer (baselayout, networking)

-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] baselayout-1.13 going into ~ARCH soon

[Roy Marples]

On Monday 06 November 2006 16:53, Roy Marples wrote:
> However, one issue is a concern. All baselayouts defined svcdir
> in /etc/conf.d/rc which defines where we hold the state information of the
> running services. This defaulted to /var/lib/init.d - which is bad as /var
> could be on a different partition.
>
> In 1.13, we've removed the variable from /etc/conf.d/rc and it's now forced
> to /lib/rcscripts/init.d which is safe as /lib is always on the same
> partition as /. The 1.13 ebuild will copy across existing state data, this
> is not the problem. However, downgrading back to 1.12 is a problem as
> services may have been stop, started etc in the middle.
>
> One solution is to ensure that we only hold one copy of the state data and
> move it to the new location. However, this does require altering the stable
> ebuild as well.

This is still an issue.

What do people think - hack the 1.12 ebuilds and try and get an 
upgrade/downgrade path or just slap a large warning on the ebuild?

You get upgraded for free :)

-- 
Roy Marples <uberlord@gentoo.org>
Gentoo/Linux Developer (baselayout, networking)
-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] baselayout-1.13 going into ~ARCH soon

[Richard Fish]

On 11/8/06, Roy Marples <uberlord@gentoo.org> wrote:
> On Monday 06 November 2006 16:53, Roy Marples wrote:
> > However, one issue is a concern. All baselayouts defined svcdir
> > in /etc/conf.d/rc which defines where we hold the state information of the
> > running services. This defaulted to /var/lib/init.d - which is bad as /var
> > could be on a different partition.
> >
> > In 1.13, we've removed the variable from /etc/conf.d/rc and it's now forced
> > to /lib/rcscripts/init.d which is safe as /lib is always on the same
> > partition as /. The 1.13 ebuild will copy across existing state data, this
> > is not the problem. However, downgrading back to 1.12 is a problem as
> > services may have been stop, started etc in the middle.
> >
> > One solution is to ensure that we only hold one copy of the state data and
> > move it to the new location. However, this does require altering the stable
> > ebuild as well.
>
> This is still an issue.
>
> What do people think - hack the 1.12 ebuilds and try and get an
> upgrade/downgrade path or just slap a large warning on the ebuild?

How about as part of the upgrade, create a symlink from
/var/lib/init.d to /lib/rcscripts/init.d?

-Richard
-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] baselayout-1.13 going into ~ARCH soon

[Mike Frysinger]

[-- Attachment #1: Type: text/plain, Size: 130 bytes --]

On Wednesday 08 November 2006 19:00, Roy Marples wrote:
> just slap a large warning on the ebuild? 

that's fine by me
-mike

[-- Attachment #2: Type: application/pgp-signature, Size: 827 bytes --]