[gentoo-user] xen on new install reboots by itself

[gentoo-user] xen on new install reboots by itself

[symack]

[-- Attachment #1: Type: text/plain, Size: 965 bytes --]

Hello Everyone,

New install, on a old server with raid 10 scsi... The normal installation
works fine,
the only thing is when we try to boot with xen, it gets to the prompt and
then reboots
by itself. The following message is what differs between normal gentoo and
xen kernel

Mar 31 06:32:18 test kernel: [    0.138644] ACPI Exception: AE_NOT_FOUND,
While evaluating Sleep State [\_S1_] (20140724/hwxface-580)
Mar 31 06:32:18 test kernel: [    0.138961] ACPI Exception: AE_NOT_FOUND,
While evaluating Sleep State [\_S2_] (20140724/hwxface-580)
Mar 31 06:32:18 test kernel: [    0.139267] ACPI Exception: AE_NOT_FOUND,
While evaluating Sleep State [\_S3_] (20140724/hwxface-580)


I'm never sure how to debug such errors. Some googling suggested adding
ACPI flags (ie, force, on, off), when I do that, the system just reboots
without getting to the login prompt.

The server is an X346 with hardward servraid 7K with raid 10.

Your help is greatly appreciated.

N.

[-- Attachment #2: Type: text/html, Size: 1278 bytes --]

Re: [gentoo-user] xen on new install reboots by itself

[hydra]

[-- Attachment #1: Type: text/plain, Size: 1174 bytes --]

On Tue, Mar 31, 2015 at 1:07 AM, symack <symack@gmail.com> wrote:

> Hello Everyone,
>
> New install, on a old server with raid 10 scsi... The normal installation
> works fine,
> the only thing is when we try to boot with xen, it gets to the prompt and
> then reboots
> by itself. The following message is what differs between normal gentoo and
> xen kernel
>
> Mar 31 06:32:18 test kernel: [    0.138644] ACPI Exception: AE_NOT_FOUND,
> While evaluating Sleep State [\_S1_] (20140724/hwxface-580)
> Mar 31 06:32:18 test kernel: [    0.138961] ACPI Exception: AE_NOT_FOUND,
> While evaluating Sleep State [\_S2_] (20140724/hwxface-580)
> Mar 31 06:32:18 test kernel: [    0.139267] ACPI Exception: AE_NOT_FOUND,
> While evaluating Sleep State [\_S3_] (20140724/hwxface-580)
>
>
> I'm never sure how to debug such errors. Some googling suggested adding
> ACPI flags (ie, force, on, off), when I do that, the system just reboots
> without getting to the login prompt.
>
> The server is an X346 with hardward servraid 7K with raid 10.
>
> Your help is greatly appreciated.
>
> N.
>

Please post your emerge --info information along with xen and xen-tools
versions / use flags.

[-- Attachment #2: Type: text/html, Size: 1700 bytes --]

Re: [gentoo-user] xen on new install reboots by itself

[symack]

[-- Attachment #1: Type: text/plain, Size: 4794 bytes --]

Hello Hydra,


Thank you for your response:

The commands are bellow. Also, the Bios is almost 10 years old on the
machine.
Not sure if that info helps for the ACPI related messages.

USE="app-emulation/xen-tools-4.3.1-r1 api hvm qenu screen -custom-cflags
-debug -doc -flask -xend"
USE="app-emulation/xen-4.3.3-r3 -custom-cflags -debug -flask -pae -xsm"



Portage 2.2.14 (python 2.7.9-final-0, hardened/linux/amd64/no-multilib,
gcc-4.8.3, glibc-2.19-r1, 3.17.7-gentoo x86_64)
=================================================================
System uname:
Linux-3.17.7-gentoo-x86_64-Intel-R-_Xeon-TM-_CPU_3.80GHz-with-gentoo-2.2
KiB Mem:     8177312 total,   8051276 free
KiB Swap:    2097148 total,   2097148 free
Timestamp of tree: Thu, 15 Jan 2015 17:45:01 +0000
ld GNU ld (Gentoo 2.24 p1.4) 2.24
app-shells/bash:          4.2_p53
dev-lang/perl:            5.18.2-r2
dev-lang/python:          2.7.9-r1, 3.3.5-r1
dev-util/cmake:           2.8.12.2-r1
dev-util/pkgconfig:       0.28-r1
sys-apps/baselayout:      2.2
sys-apps/openrc:          0.12.4
sys-apps/sandbox:         2.6-r1
sys-devel/autoconf:       2.69
sys-devel/automake:       1.13.4
sys-devel/binutils:       2.24-r3
sys-devel/gcc:            4.8.3
sys-devel/gcc-config:     1.7.3
sys-devel/libtool:        2.4.2-r1
sys-devel/make:           4.0-r1
sys-kernel/linux-headers: 3.16 (virtual/os-headers)
sys-libs/glibc:           2.19-r1
Repositories: gentoo
ACCEPT_KEYWORDS="amd64"
ACCEPT_LICENSE="* -@EULA"
CBUILD="x86_64-pc-linux-gnu"
CFLAGS="-O2 -pipe"
CHOST="x86_64-pc-linux-gnu"
CONFIG_PROTECT="/etc"
CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/env.d /etc/gconf
/etc/gentoo-release /etc/revdep-rebuild /etc/sandbox.d /etc/terminfo"
CXXFLAGS="-O2 -pipe"
DISTDIR="/usr/portage/distfiles"
FCFLAGS="-O2 -pipe"
FEATURES="assume-digests binpkg-logs config-protect-if-modified distlocks
ebuild-locks fixlafiles merge-sync news parallel-fetch preserve-libs
protect-owned sandbox sfperms strict unknown-features-warn unmerge-logs
unmerge-orphans userfetch userpriv usersandbox usersync xattr"
FFLAGS="-O2 -pipe"
GENTOO_MIRRORS="http://mirror.csclub.uwaterloo.ca/gentoo-distfiles/"
LANG="en_US"
LDFLAGS="-Wl,-O1 -Wl,--as-needed"
MAKEOPTS="-j2"
PKGDIR="/usr/portage/packages"
PORTAGE_CONFIGROOT="/"
PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times
--omit-dir-times --compress --force --whole-file --delete --stats
--human-readable --timeout=180 --exclude=/distfiles --exclude=/local
--exclude=/packages"
PORTAGE_TMPDIR="/var/tmp"
PORTDIR="/usr/portage"
PORTDIR_OVERLAY=""
SYNC="rsync://rsync4.ca.gentoo.org/gentoo-portage"
USE="acl amd64 app-emulation/xen-4.3.3-r3 berkdb bzip2 cli cracklib crypt
cxx dri gdbm hardened iconv ipv6 justify mmx modules ncurses nls nptl
openmp pam pax_kernel pcre readline session sse sse2 ssl tcpd unicode
urandom xattr xtpax zlib" ABI_X86="64" ALSA_CARDS="ali5451 als4000 atiixp
atiixp-modem bt87x ca0106 cmipci emu10k1x ens1370 ens1371 es1938 es1968
fm801 hda-intel intel8x0 intel8x0m maestro3 trident usb-audio via82xx
via82xx-modem ymfpci" APACHE2_MODULES="authn_core authz_core socache_shmcb
unixd actions alias auth_basic authn_alias authn_anon authn_dbm
authn_default authn_file authz_dbm authz_default authz_groupfile authz_host
authz_owner authz_user autoindex cache cgi cgid dav dav_fs dav_lock deflate
dir disk_cache env expires ext_filter file_cache filter headers include
info log_config logio mem_cache mime mime_magic negotiation rewrite
setenvif speling status unique_id userdir usertrack vhost_alias"
CALLIGRA_FEATURES="kexi words flow plan sheets stage tables krita karbon
braindump author" CAMERAS="ptp2" COLLECTD_PLUGINS="df interface irq load
memory rrdtool swap syslog" ELIBC="glibc" GPSD_PROTOCOLS="ashtech aivdm
earthmate evermore fv18 garmin garmintxt gpsclock itrax mtk3301 nmea ntrip
navcom oceanserver oldstyle oncore rtcm104v2 rtcm104v3 sirf superstar2
timing tsip tripmate tnt ublox ubx" INPUT_DEVICES="keyboard mouse evdev"
KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216
lcdm001 mtxorb ncurses text" LIBREOFFICE_EXTENSIONS="presenter-console
presenter-minimizer" OFFICE_IMPLEMENTATION="libreoffice"
PHP_TARGETS="php5-5" PYTHON_SINGLE_TARGET="python2_7"
PYTHON_TARGETS="python2_7 python3_3" RUBY_TARGETS="ruby19 ruby20"
USERLAND="GNU" VIDEO_CARDS="fbdev glint intel mach64 mga nouveau nv r128
radeon savage sis tdfx trident vesa via vmware dummy v4l"
XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipset ipp2p
iface geoip fuzzy condition tee tarpit sysrq steal rawnat logmark ipmark
dhcpmac delude chaos account"
Unset:  CPPFLAGS, CTARGET, EMERGE_DEFAULT_OPTS, INSTALL_MASK, LC_ALL,
PORTAGE_BUNZIP2_COMMAND, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS,
PORTAGE_RSYNC_EXTRA_OPTS, USE_PYTHON



Kind Regards,

Nick.

[-- Attachment #2: Type: text/html, Size: 6444 bytes --]

Re: [gentoo-user] xen on new install reboots by itself

[hydra]

[-- Attachment #1: Type: text/plain, Size: 5245 bytes --]

On Wed, Apr 1, 2015 at 3:52 AM, symack <symack@gmail.com> wrote:

> Hello Hydra,
>
>
> Thank you for your response:
>
> The commands are bellow. Also, the Bios is almost 10 years old on the
> machine.
> Not sure if that info helps for the ACPI related messages.
>
> USE="app-emulation/xen-tools-4.3.1-r1 api hvm qenu screen -custom-cflags
> -debug -doc -flask -xend"
> USE="app-emulation/xen-4.3.3-r3 -custom-cflags -debug -flask -pae -xsm"
>
>
>
> Portage 2.2.14 (python 2.7.9-final-0, hardened/linux/amd64/no-multilib,
> gcc-4.8.3, glibc-2.19-r1, 3.17.7-gentoo x86_64)
> =================================================================
> System uname:
> Linux-3.17.7-gentoo-x86_64-Intel-R-_Xeon-TM-_CPU_3.80GHz-with-gentoo-2.2
> KiB Mem:     8177312 total,   8051276 free
> KiB Swap:    2097148 total,   2097148 free
> Timestamp of tree: Thu, 15 Jan 2015 17:45:01 +0000
> ld GNU ld (Gentoo 2.24 p1.4) 2.24
> app-shells/bash:          4.2_p53
> dev-lang/perl:            5.18.2-r2
> dev-lang/python:          2.7.9-r1, 3.3.5-r1
> dev-util/cmake:           2.8.12.2-r1
> dev-util/pkgconfig:       0.28-r1
> sys-apps/baselayout:      2.2
> sys-apps/openrc:          0.12.4
> sys-apps/sandbox:         2.6-r1
> sys-devel/autoconf:       2.69
> sys-devel/automake:       1.13.4
> sys-devel/binutils:       2.24-r3
> sys-devel/gcc:            4.8.3
> sys-devel/gcc-config:     1.7.3
> sys-devel/libtool:        2.4.2-r1
> sys-devel/make:           4.0-r1
> sys-kernel/linux-headers: 3.16 (virtual/os-headers)
> sys-libs/glibc:           2.19-r1
> Repositories: gentoo
> ACCEPT_KEYWORDS="amd64"
> ACCEPT_LICENSE="* -@EULA"
> CBUILD="x86_64-pc-linux-gnu"
> CFLAGS="-O2 -pipe"
> CHOST="x86_64-pc-linux-gnu"
> CONFIG_PROTECT="/etc"
> CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/env.d /etc/gconf
> /etc/gentoo-release /etc/revdep-rebuild /etc/sandbox.d /etc/terminfo"
> CXXFLAGS="-O2 -pipe"
> DISTDIR="/usr/portage/distfiles"
> FCFLAGS="-O2 -pipe"
> FEATURES="assume-digests binpkg-logs config-protect-if-modified distlocks
> ebuild-locks fixlafiles merge-sync news parallel-fetch preserve-libs
> protect-owned sandbox sfperms strict unknown-features-warn unmerge-logs
> unmerge-orphans userfetch userpriv usersandbox usersync xattr"
> FFLAGS="-O2 -pipe"
> GENTOO_MIRRORS="http://mirror.csclub.uwaterloo.ca/gentoo-distfiles/"
> LANG="en_US"
> LDFLAGS="-Wl,-O1 -Wl,--as-needed"
> MAKEOPTS="-j2"
> PKGDIR="/usr/portage/packages"
> PORTAGE_CONFIGROOT="/"
> PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times
> --omit-dir-times --compress --force --whole-file --delete --stats
> --human-readable --timeout=180 --exclude=/distfiles --exclude=/local
> --exclude=/packages"
> PORTAGE_TMPDIR="/var/tmp"
> PORTDIR="/usr/portage"
> PORTDIR_OVERLAY=""
> SYNC="rsync://rsync4.ca.gentoo.org/gentoo-portage"
> USE="acl amd64 app-emulation/xen-4.3.3-r3 berkdb bzip2 cli cracklib crypt
> cxx dri gdbm hardened iconv ipv6 justify mmx modules ncurses nls nptl
> openmp pam pax_kernel pcre readline session sse sse2 ssl tcpd unicode
> urandom xattr xtpax zlib" ABI_X86="64" ALSA_CARDS="ali5451 als4000 atiixp
> atiixp-modem bt87x ca0106 cmipci emu10k1x ens1370 ens1371 es1938 es1968
> fm801 hda-intel intel8x0 intel8x0m maestro3 trident usb-audio via82xx
> via82xx-modem ymfpci" APACHE2_MODULES="authn_core authz_core socache_shmcb
> unixd actions alias auth_basic authn_alias authn_anon authn_dbm
> authn_default authn_file authz_dbm authz_default authz_groupfile authz_host
> authz_owner authz_user autoindex cache cgi cgid dav dav_fs dav_lock deflate
> dir disk_cache env expires ext_filter file_cache filter headers include
> info log_config logio mem_cache mime mime_magic negotiation rewrite
> setenvif speling status unique_id userdir usertrack vhost_alias"
> CALLIGRA_FEATURES="kexi words flow plan sheets stage tables krita karbon
> braindump author" CAMERAS="ptp2" COLLECTD_PLUGINS="df interface irq load
> memory rrdtool swap syslog" ELIBC="glibc" GPSD_PROTOCOLS="ashtech aivdm
> earthmate evermore fv18 garmin garmintxt gpsclock itrax mtk3301 nmea ntrip
> navcom oceanserver oldstyle oncore rtcm104v2 rtcm104v3 sirf superstar2
> timing tsip tripmate tnt ublox ubx" INPUT_DEVICES="keyboard mouse evdev"
> KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216
> lcdm001 mtxorb ncurses text" LIBREOFFICE_EXTENSIONS="presenter-console
> presenter-minimizer" OFFICE_IMPLEMENTATION="libreoffice"
> PHP_TARGETS="php5-5" PYTHON_SINGLE_TARGET="python2_7"
> PYTHON_TARGETS="python2_7 python3_3" RUBY_TARGETS="ruby19 ruby20"
> USERLAND="GNU" VIDEO_CARDS="fbdev glint intel mach64 mga nouveau nv r128
> radeon savage sis tdfx trident vesa via vmware dummy v4l"
> XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipset ipp2p
> iface geoip fuzzy condition tee tarpit sysrq steal rawnat logmark ipmark
> dhcpmac delude chaos account"
> Unset:  CPPFLAGS, CTARGET, EMERGE_DEFAULT_OPTS, INSTALL_MASK, LC_ALL,
> PORTAGE_BUNZIP2_COMMAND, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS,
> PORTAGE_RSYNC_EXTRA_OPTS, USE_PYTHON
>
>
>
> Kind Regards,
>
> Nick.
>
>
>
>
>
>
Did xen ever work on that machine?

I don't know if xen works with hardened - I always use the default profile.
Have you also tried using older/newer xen versions?

[-- Attachment #2: Type: text/html, Size: 6967 bytes --]

Re: [gentoo-user] xen on new install reboots by itself

[symack]

[-- Attachment #1: Type: text/plain, Size: 306 bytes --]

Hello Hydra,

I'm totally new to Xen and was following this tutorial very compressed
tutorial https://wiki.gentoo.org/wiki/Xen
I purchased the machine as the test machine to see how stable/secure I can
get Xen doms to run on it.

Should I change the profile and reinstall xen/xen-tools


N
​

[-- Attachment #2: Type: text/html, Size: 526 bytes --]

Re: [gentoo-user] xen on new install reboots by itself

[hydra]

[-- Attachment #1: Type: text/plain, Size: 518 bytes --]

On Wed, Apr 1, 2015 at 2:52 PM, symack <symack@gmail.com> wrote:

> Hello Hydra,
>
> I'm totally new to Xen and was following this tutorial very compressed
> tutorial https://wiki.gentoo.org/wiki/Xen
> I purchased the machine as the test machine to see how stable/secure I can
> get Xen doms to run on it.
>
> Should I change the profile and reinstall xen/xen-tools
>
>
> N
> ​
>

I have no idea if it helps, but if you can play with the machine, try the
default profile and/or other xen versions.

[-- Attachment #2: Type: text/html, Size: 1002 bytes --]

Re: [gentoo-user] xen on new install reboots by itself

[symack]

[-- Attachment #1: Type: text/plain, Size: 358 bytes --]

Hello Hydra,

Thank you so much. I was literally about to use the default profile and
reinstall the same
versions. If I need to go back in time, which version is considered Stable
and not too old
for xen and xen-tools. Finally, as I am new to gentoo, can you please tell
me how to emerge
an older xen version.

Thanks for Everything!

Nick
​

[-- Attachment #2: Type: text/html, Size: 515 bytes --]

Re: [gentoo-user] xen on new install reboots by itself

[hydra]

[-- Attachment #1: Type: text/plain, Size: 1048 bytes --]

On Wed, Apr 1, 2015 at 6:24 PM, symack <symack@gmail.com> wrote:

> Hello Hydra,
>
> Thank you so much. I was literally about to use the default profile and
> reinstall the same
> versions. If I need to go back in time, which version is considered Stable
> and not too old
> for xen and xen-tools. Finally, as I am new to gentoo, can you please tell
> me how to emerge
> an older xen version.
>
> Thanks for Everything!
>
> Nick
> ​
>

You can try those in portage:
https://packages.gentoo.org/package/app-emulation/xen
https://packages.gentoo.org/package/app-emulation/xen-tools

Currently you have 4.3 branch (which was removed from portage a week ago or
so), try 4.2, 4.4 or 4.5, you'll see what you get.

If you want to emerge a specific version:
emerge =app-emulation/xen-4.2.5-r7 =app-emulation/xen-tools-4.2.5-r3

Don't mix xen and xen-tools branches (4.2 with 4.3) - always keep both the
same branch. The revisions don't have to match (like the above r7 for xen
and r3 for xen-tools), but the branches yes.

[-- Attachment #2: Type: text/html, Size: 1643 bytes --]

Re: [gentoo-user] xen on new install reboots by itself

[J. Roeleveld]

On Monday, March 30, 2015 07:07:39 PM symack wrote:
> Hello Everyone,
> 
> New install, on a old server with raid 10 scsi... The normal installation
> works fine,
> the only thing is when we try to boot with xen, it gets to the prompt and
> then reboots
> by itself. The following message is what differs between normal gentoo and
> xen kernel
> 
> Mar 31 06:32:18 test kernel: [    0.138644] ACPI Exception: AE_NOT_FOUND,
> While evaluating Sleep State [\_S1_] (20140724/hwxface-580)
> Mar 31 06:32:18 test kernel: [    0.138961] ACPI Exception: AE_NOT_FOUND,
> While evaluating Sleep State [\_S2_] (20140724/hwxface-580)
> Mar 31 06:32:18 test kernel: [    0.139267] ACPI Exception: AE_NOT_FOUND,
> While evaluating Sleep State [\_S3_] (20140724/hwxface-580)
> 
> 
> I'm never sure how to debug such errors. Some googling suggested adding
> ACPI flags (ie, force, on, off), when I do that, the system just reboots
> without getting to the login prompt.
> 
> The server is an X346 with hardward servraid 7K with raid 10.
> 
> Your help is greatly appreciated.
> 
> N.

Do you have the XEN dom0 support compiled into the kernel?

I have the following options on one of my servers and don't recall the issue 
you are facing:
# zcat /proc/config.gz | grep -i xen
CONFIG_XEN=y
CONFIG_XEN_DOM0=y
CONFIG_XEN_PVHVM=y
CONFIG_XEN_MAX_DOMAIN_MEMORY=500
CONFIG_XEN_SAVE_RESTORE=y
# CONFIG_XEN_PVH is not set
CONFIG_PCI_XEN=y
CONFIG_XEN_PCIDEV_FRONTEND=y
CONFIG_XEN_BLKDEV_FRONTEND=y
CONFIG_XEN_BLKDEV_BACKEND=y
CONFIG_XEN_NETDEV_FRONTEND=y
CONFIG_XEN_NETDEV_BACKEND=y
CONFIG_INPUT_XEN_KBDDEV_FRONTEND=y
CONFIG_HVC_XEN=y
CONFIG_HVC_XEN_FRONTEND=y
CONFIG_XEN_FBDEV_FRONTEND=y
# Xen driver support
CONFIG_XEN_BALLOON=y
# CONFIG_XEN_SELFBALLOONING is not set
CONFIG_XEN_SCRUB_PAGES=y
CONFIG_XEN_DEV_EVTCHN=y
CONFIG_XEN_BACKEND=y
CONFIG_XENFS=y
CONFIG_XEN_COMPAT_XENFS=y
CONFIG_XEN_SYS_HYPERVISOR=y
CONFIG_XEN_XENBUS_FRONTEND=y
CONFIG_XEN_GNTDEV=y
CONFIG_XEN_GRANT_DEV_ALLOC=m
CONFIG_SWIOTLB_XEN=y
CONFIG_XEN_TMEM=m
CONFIG_XEN_PCIDEV_BACKEND=m
CONFIG_XEN_PRIVCMD=y
CONFIG_XEN_ACPI_PROCESSOR=m
# CONFIG_XEN_MCE_LOG is not set
CONFIG_XEN_HAVE_PVMMU=y


I, currently, have the following versions for Xen running:
app-emulation/xen-4.3.3-r3
app-emulation/xen-tools-4.3.3-r1

These are scheduled to be upgraded during the next maintenance window.

--
Joost

Re: [gentoo-user] xen on new install reboots by itself

[lee]

symack <symack@gmail.com> writes:

> the only thing is when we try to boot with xen, it gets to the prompt and
> then reboots
> by itself. The following message is what differs between normal gentoo and
> xen kernel
>
> Mar 31 06:32:18 test kernel: [    0.138644] ACPI Exception: AE_NOT_FOUND,
> While evaluating Sleep State [\_S1_] (20140724/hwxface-580)
> Mar 31 06:32:18 test kernel: [    0.138961] ACPI Exception: AE_NOT_FOUND,
> While evaluating Sleep State [\_S2_] (20140724/hwxface-580)
> Mar 31 06:32:18 test kernel: [    0.139267] ACPI Exception: AE_NOT_FOUND,
> While evaluating Sleep State [\_S3_] (20140724/hwxface-580)

There is a kernel boot parameter which you can use to prevent the xen
kernel from attempting power management --- though IIRC that was more
related to frequency settings.

As far as I was able to figure out, you have two options: Either xen
sets frequencies, or dom0 does.  If the latter, all CPUs must be
available to dom0.  However, currently nobody seems to know exactly what
xen does towards frequency setting.


And if I'm not horribly mistaken, there is a way to do something about
what xen considers as usable, or available, cpu states.  It was
something along the lines that xen might not detect all possible cpu
states, and you could do something to tell it that there are more states
than it detects.  Perhaps it's also possible to limit xen to S0.

First I would look into updating the BIOS.  If that doesn't help, I'd
try to limit xen to S0.

Other than that, unless you really do need full virtualization: I'm
finding Linux containers to be far more manageable than virtual
machines, and much more efficient.


-- 
Again we must be afraid of speaking of daemons for fear that daemons
might swallow us.  Finally, this fear has become reasonable.

Re: [gentoo-user] xen on new install reboots by itself

[hydra]

[-- Attachment #1: Type: text/plain, Size: 2688 bytes --]

On Thu, Apr 2, 2015 at 8:20 AM, J. Roeleveld <joost@antarean.org> wrote:

> On Monday, March 30, 2015 07:07:39 PM symack wrote:
> > Hello Everyone,
> >
> > New install, on a old server with raid 10 scsi... The normal installation
> > works fine,
> > the only thing is when we try to boot with xen, it gets to the prompt and
> > then reboots
> > by itself. The following message is what differs between normal gentoo
> and
> > xen kernel
> >
> > Mar 31 06:32:18 test kernel: [    0.138644] ACPI Exception: AE_NOT_FOUND,
> > While evaluating Sleep State [\_S1_] (20140724/hwxface-580)
> > Mar 31 06:32:18 test kernel: [    0.138961] ACPI Exception: AE_NOT_FOUND,
> > While evaluating Sleep State [\_S2_] (20140724/hwxface-580)
> > Mar 31 06:32:18 test kernel: [    0.139267] ACPI Exception: AE_NOT_FOUND,
> > While evaluating Sleep State [\_S3_] (20140724/hwxface-580)
> >
> >
> > I'm never sure how to debug such errors. Some googling suggested adding
> > ACPI flags (ie, force, on, off), when I do that, the system just reboots
> > without getting to the login prompt.
> >
> > The server is an X346 with hardward servraid 7K with raid 10.
> >
> > Your help is greatly appreciated.
> >
> > N.
>
> Do you have the XEN dom0 support compiled into the kernel?
>
> I have the following options on one of my servers and don't recall the
> issue
> you are facing:
> # zcat /proc/config.gz | grep -i xen
> CONFIG_XEN=y
> CONFIG_XEN_DOM0=y
> CONFIG_XEN_PVHVM=y
> CONFIG_XEN_MAX_DOMAIN_MEMORY=500
> CONFIG_XEN_SAVE_RESTORE=y
> # CONFIG_XEN_PVH is not set
> CONFIG_PCI_XEN=y
> CONFIG_XEN_PCIDEV_FRONTEND=y
> CONFIG_XEN_BLKDEV_FRONTEND=y
> CONFIG_XEN_BLKDEV_BACKEND=y
> CONFIG_XEN_NETDEV_FRONTEND=y
> CONFIG_XEN_NETDEV_BACKEND=y
> CONFIG_INPUT_XEN_KBDDEV_FRONTEND=y
> CONFIG_HVC_XEN=y
> CONFIG_HVC_XEN_FRONTEND=y
> CONFIG_XEN_FBDEV_FRONTEND=y
> # Xen driver support
> CONFIG_XEN_BALLOON=y
> # CONFIG_XEN_SELFBALLOONING is not set
> CONFIG_XEN_SCRUB_PAGES=y
> CONFIG_XEN_DEV_EVTCHN=y
> CONFIG_XEN_BACKEND=y
> CONFIG_XENFS=y
> CONFIG_XEN_COMPAT_XENFS=y
> CONFIG_XEN_SYS_HYPERVISOR=y
> CONFIG_XEN_XENBUS_FRONTEND=y
> CONFIG_XEN_GNTDEV=y
> CONFIG_XEN_GRANT_DEV_ALLOC=m
> CONFIG_SWIOTLB_XEN=y
> CONFIG_XEN_TMEM=m
> CONFIG_XEN_PCIDEV_BACKEND=m
> CONFIG_XEN_PRIVCMD=y
> CONFIG_XEN_ACPI_PROCESSOR=m
> # CONFIG_XEN_MCE_LOG is not set
> CONFIG_XEN_HAVE_PVMMU=y
>
>
> I, currently, have the following versions for Xen running:
> app-emulation/xen-4.3.3-r3
> app-emulation/xen-tools-4.3.3-r1
>
> These are scheduled to be upgraded during the next maintenance window.
>
> --
> Joost
>


True, maybe we should look for problem here. Maybe you can attach your
kernel configuration or at least `grep -i xen .config`.

[-- Attachment #2: Type: text/html, Size: 3391 bytes --]

Re: [gentoo-user] xen on new install reboots by itself

[hydra]

[-- Attachment #1: Type: text/plain, Size: 316 bytes --]

On Sat, Apr 4, 2015 at 3:20 PM, lee <lee@yagibdah.de> wrote:

> symack <symack@gmail.com> writes:
>
> Other than that, unless you really do need full virtualization: I'm
> finding Linux containers to be far more manageable than virtual
> machines, and much more efficient.
>
>
Can you please post some more details?

[-- Attachment #2: Type: text/html, Size: 760 bytes --]

Re: [gentoo-user] xen on new install reboots by itself

[lee]

hydra <hydrapolic@gmail.com> writes:

> On Sat, Apr 4, 2015 at 3:20 PM, lee <lee@yagibdah.de> wrote:
>
>> symack <symack@gmail.com> writes:
>>
>> Other than that, unless you really do need full virtualization: I'm
>> finding Linux containers to be far more manageable than virtual
>> machines, and much more efficient.
>>
>>
> Can you please post some more details?

About containers?

There's very useful documentation about them like
https://wiki.gentoo.org/wiki/LXC ...

What can I say?  Virtualization with xen is like juggling with a set of
black boxes each of which aren't exactly accessible; the
documentation sucks, it's hard work to get it running and likewise hard
to maintain.

Virtualization with containers is basically as simple as running just
another daemon.

Which the "better" tool, or combination of tools is, depends on what you
want to accomplish.  You could use containers in a VM, too, or use
virtualbox along with containers to run the odd VMs that require full
virtualzation.


-- 
Again we must be afraid of speaking of daemons for fear that daemons
might swallow us.  Finally, this fear has become reasonable.

Re: [gentoo-user] xen on new install reboots by itself

[hydra]

[-- Attachment #1: Type: text/plain, Size: 1503 bytes --]

On Wed, Apr 8, 2015 at 11:43 PM, lee <lee@yagibdah.de> wrote:

> hydra <hydrapolic@gmail.com> writes:
>
> > On Sat, Apr 4, 2015 at 3:20 PM, lee <lee@yagibdah.de> wrote:
> >
> >> symack <symack@gmail.com> writes:
> >>
> >> Other than that, unless you really do need full virtualization: I'm
> >> finding Linux containers to be far more manageable than virtual
> >> machines, and much more efficient.
> >>
> >>
> > Can you please post some more details?
>
> About containers?
>
> There's very useful documentation about them like
> https://wiki.gentoo.org/wiki/LXC ...
>
> What can I say?  Virtualization with xen is like juggling with a set of
> black boxes each of which aren't exactly accessible; the
> documentation sucks, it's hard work to get it running and likewise hard
> to maintain.
>
> Virtualization with containers is basically as simple as running just
> another daemon.
>
> Which the "better" tool, or combination of tools is, depends on what you
> want to accomplish.  You could use containers in a VM, too, or use
> virtualbox along with containers to run the odd VMs that require full
> virtualzation.
>
>
> --
> Again we must be afraid of speaking of daemons for fear that daemons
> might swallow us.  Finally, this fear has become reasonable.
>
>
 You mean the documentation at Gentoo about Xen sucks or the upstream
documentation? What information are you missing from there? Maybe we can
add  the missing pieces for Xen being more accessible and easier to use,
what do you think? :)

[-- Attachment #2: Type: text/html, Size: 2317 bytes --]

Re: [gentoo-user] xen on new install reboots by itself

[J. Roeleveld]

On 8 April 2015 14:43:02 GMT-07:00, lee <lee@yagibdah.de> wrote:
>hydra <hydrapolic@gmail.com> writes:
>
>> On Sat, Apr 4, 2015 at 3:20 PM, lee <lee@yagibdah.de> wrote:
>>
>>> symack <symack@gmail.com> writes:
>>>
>>> Other than that, unless you really do need full virtualization: I'm
>>> finding Linux containers to be far more manageable than virtual
>>> machines, and much more efficient.
>>>
>>>
>> Can you please post some more details?
>
>About containers?
>
>There's very useful documentation about them like
>https://wiki.gentoo.org/wiki/LXC ...
>
>What can I say?  Virtualization with xen is like juggling with a set of
>black boxes each of which aren't exactly accessible; the
>documentation sucks, it's hard work to get it running and likewise hard
>to maintain.

I disagree. Been using Xen for over 10 years now and find it very easy to use.
The documentation could be better on the Xen site itself, but there is plenty of decent documentation available via Google. 

>Virtualization with containers is basically as simple as running just
>another daemon.

Not quite. I use virtualization to minimizer the physical hardware. Xen is easy for that. Containers are what chroot jails should have been.
But there is no simple method to set these up when security isolation is your goal.

>Which the "better" tool, or combination of tools is, depends on what
>you
>want to accomplish.  You could use containers in a VM, too, or use
>virtualbox along with containers to run the odd VMs that require full
>virtualzation.

Virtualbox is nice for a quick test. I wouldn't use it for production.

--
Joost 


-- 
Sent from my Android device with K-9 Mail. Please excuse my brevity.

Re: [gentoo-user] xen on new install reboots by itself

[symack]

[-- Attachment #1: Type: text/plain, Size: 1448 bytes --]

Hello Everyone,

Sorry for the delayed response. Flased the bios to the latest version 1.1.7
dated 2007.
Bellow is xen kernel built features:

CONFIG_XEN=y
CONFIG_XEN_DOM0=y
CONFIG_XEN_PVHVM=y
CONFIG_XEN_MAX_DOMAIN_MEMORY=500
CONFIG_XEN_SAVE_RESTORE=y
# CONFIG_XEN_DEBUG_FS is not set
# CONFIG_XEN_PVH is not set
CONFIG_PCI_XEN=y
CONFIG_XEN_PCIDEV_FRONTEND=y
CONFIG_XEN_BLKDEV_FRONTEND=y
CONFIG_XEN_BLKDEV_BACKEND=y
CONFIG_XEN_NETDEV_FRONTEND=y
CONFIG_XEN_NETDEV_BACKEND=y
CONFIG_INPUT_XEN_KBDDEV_FRONTEND=y
CONFIG_HVC_XEN=y
CONFIG_HVC_XEN_FRONTEND=y
# CONFIG_XEN_WDT is not set
CONFIG_XEN_FBDEV_FRONTEND=y
# Xen driver support
CONFIG_XEN_BALLOON=y
CONFIG_XEN_SCRUB_PAGES=y
CONFIG_XEN_DEV_EVTCHN=y
CONFIG_XEN_BACKEND=y
CONFIG_XENFS=y
CONFIG_XEN_COMPAT_XENFS=y
CONFIG_XEN_SYS_HYPERVISOR=y
CONFIG_XEN_XENBUS_FRONTEND=y
CONFIG_XEN_GNTDEV=y
CONFIG_XEN_GRANT_DEV_ALLOC=y
CONFIG_SWIOTLB_XEN=y
CONFIG_XEN_PCIDEV_BACKEND=y
CONFIG_XEN_PRIVCMD=y
CONFIG_XEN_ACPI_PROCESSOR=y
# CONFIG_XEN_MCE_LOG is not set
CONFIG_XEN_HAVE_PVMMU=y



The machine is still rebooting when booting in the Xen built kernel. I know
these machines have a thing
called VRM (Voltage Regulator Module). This module is needed for dual cpu
configurations. The machine
has only on CPU and so no VRM. I read somewhere there was a issue with VRM
equipped machine.

Your help is greatly appreciate as I am stumped over this.

Nick.
​

[-- Attachment #2: Type: text/html, Size: 2009 bytes --]

Re: [gentoo-user] xen on new install reboots by itself

[lee]

hydra <hydrapolic@gmail.com> writes:

>  You mean the documentation at Gentoo about Xen sucks or the upstream
> documentation? What information are you missing from there? Maybe we can
> add  the missing pieces for Xen being more accessible and easier to use,
> what do you think? :)

I mean the documentation they have on their wiki.  It's a confusing mess
referring to various version with which things are being done
differently.

Could you add missing pieces about why power management --- as in
frequency scaling --- doesn't work and what to do about keeping the time
in sync between all VMs when you find out that this doesn't work as the
documentation would have you think it does?


-- 
Again we must be afraid of speaking of daemons for fear that daemons
might swallow us.  Finally, this fear has become reasonable.

Re: [gentoo-user] xen on new install reboots by itself

[lee]

"J. Roeleveld" <joost@antarean.org> writes:

> On 8 April 2015 14:43:02 GMT-07:00, lee <lee@yagibdah.de> wrote:
>>hydra <hydrapolic@gmail.com> writes:
>>
>>> On Sat, Apr 4, 2015 at 3:20 PM, lee <lee@yagibdah.de> wrote:
>>>
>>>> symack <symack@gmail.com> writes:
>>>>
>>>> Other than that, unless you really do need full virtualization: I'm
>>>> finding Linux containers to be far more manageable than virtual
>>>> machines, and much more efficient.
>>>>
>>>>
>>> Can you please post some more details?
>>
>>About containers?
>>
>>There's very useful documentation about them like
>>https://wiki.gentoo.org/wiki/LXC ...
>>
>>What can I say?  Virtualization with xen is like juggling with a set of
>>black boxes each of which aren't exactly accessible; the
>>documentation sucks, it's hard work to get it running and likewise hard
>>to maintain.
>
> I disagree. Been using Xen for over 10 years now and find it very easy to use.
> The documentation could be better on the Xen site itself, but there is plenty of decent documentation available via Google. 

Then we just disagree about this.

>>Virtualization with containers is basically as simple as running just
>>another daemon.
>
> Not quite. I use virtualization to minimizer the physical hardware. Xen is easy for that. Containers are what chroot jails should have been.
> But there is no simple method to set these up when security isolation is your goal.

Containers or chroots?

>>Which the "better" tool, or combination of tools is, depends on what
>>you
>>want to accomplish.  You could use containers in a VM, too, or use
>>virtualbox along with containers to run the odd VMs that require full
>>virtualzation.
>
> Virtualbox is nice for a quick test. I wouldn't use it for production.

Why not?


-- 
Again we must be afraid of speaking of daemons for fear that daemons
might swallow us.  Finally, this fear has become reasonable.

Re: [gentoo-user] xen on new install reboots by itself

[J. Roeleveld]

On Thursday, April 23, 2015 11:02:24 PM lee wrote:
> hydra <hydrapolic@gmail.com> writes:
> >  You mean the documentation at Gentoo about Xen sucks or the upstream
> > 
> > documentation? What information are you missing from there? Maybe we can
> > add  the missing pieces for Xen being more accessible and easier to use,
> > what do you think? :)
> 
> I mean the documentation they have on their wiki.  It's a confusing mess
> referring to various version with which things are being done
> differently.

The problem here is the different "implementations" that exist:
- Xen (install and configure yourself, toolset: 'xl' , 'xm' is deprecated)
- Citrix and XCP (pre-configured, install on dedicated server, toolset: 'xcp')
- OVM (Oracle's implementation, not sure which toolset they use)

> Could you add missing pieces about why power management --- as in
> frequency scaling --- doesn't work

What doesn't work with this?
The following seems quite detailed:
http://wiki.xen.org/wiki/Xen_power_management

And the commands listed there (for the hypervisor based option) work on my 
server.

> and what to do about keeping the time
> in sync between all VMs when you find out that this doesn't work as the
> documentation would have you think it does?

In what way doesn't it work?
The clocks are all synchronized and I don't need to use anything like 'ntpd'

--
Joost

Re: [gentoo-user] xen on new install reboots by itself

[J. Roeleveld]

On Thursday, April 23, 2015 11:03:53 PM lee wrote:
> "J. Roeleveld" <joost@antarean.org> writes:
> > On 8 April 2015 14:43:02 GMT-07:00, lee <lee@yagibdah.de> wrote:
> >>hydra <hydrapolic@gmail.com> writes:
> >>> On Sat, Apr 4, 2015 at 3:20 PM, lee <lee@yagibdah.de> wrote:
> >>>> symack <symack@gmail.com> writes:
> >>>> 
> >>>> Other than that, unless you really do need full virtualization: I'm
> >>>> finding Linux containers to be far more manageable than virtual
> >>>> machines, and much more efficient.
> >>> 
> >>> Can you please post some more details?
> >>
> >>About containers?
> >>
> >>There's very useful documentation about them like
> >>https://wiki.gentoo.org/wiki/LXC ...
> >>
> >>What can I say?  Virtualization with xen is like juggling with a set of
> >>black boxes each of which aren't exactly accessible; the
> >>documentation sucks, it's hard work to get it running and likewise hard
> >>to maintain.
> >>
> > I disagree. Been using Xen for over 10 years now and find it very easy to
> > use. The documentation could be better on the Xen site itself, but there
> > is plenty of decent documentation available via Google.
> Then we just disagree about this.

Do you have anything that you find insufficiently documented or is too difficult?

> >>Virtualization with containers is basically as simple as running just
> >>another daemon.
> >>
> > Not quite. I use virtualization to minimizer the physical hardware. Xen is
> > easy for that. Containers are what chroot jails should have been. But
> > there is no simple method to set these up when security isolation is your
> > goal.
> Containers or chroots?

Containers.
Chroots don't have much when it comes to isolation.

> >>Which the "better" tool, or combination of tools is, depends on what
> >>you
> >>want to accomplish.  You could use containers in a VM, too, or use
> >>virtualbox along with containers to run the odd VMs that require full
> >>virtualzation.
> >>
> > Virtualbox is nice for a quick test. I wouldn't use it for production.
> 
> Why not?

Several reasons:

1) I wouldn't trust a desktop application for a server

2) The overhead from Virtualbox is quite high (still better then VMWare's 
desktop versions though)

--
Joost

Re: [gentoo-user] xen on new install reboots by itself

[J. Roeleveld]

On Thursday, April 16, 2015 06:38:08 PM symack wrote:
> Hello Everyone,
> 
> Sorry for the delayed response. Flased the bios to the latest version 1.1.7
> dated 2007.
> Bellow is xen kernel built features:
> 
> CONFIG_XEN=y
> CONFIG_XEN_DOM0=y
> CONFIG_XEN_PVHVM=y
> CONFIG_XEN_MAX_DOMAIN_MEMORY=500
> CONFIG_XEN_SAVE_RESTORE=y
> # CONFIG_XEN_DEBUG_FS is not set
> # CONFIG_XEN_PVH is not set
> CONFIG_PCI_XEN=y
> CONFIG_XEN_PCIDEV_FRONTEND=y
> CONFIG_XEN_BLKDEV_FRONTEND=y
> CONFIG_XEN_BLKDEV_BACKEND=y
> CONFIG_XEN_NETDEV_FRONTEND=y
> CONFIG_XEN_NETDEV_BACKEND=y
> CONFIG_INPUT_XEN_KBDDEV_FRONTEND=y
> CONFIG_HVC_XEN=y
> CONFIG_HVC_XEN_FRONTEND=y
> # CONFIG_XEN_WDT is not set
> CONFIG_XEN_FBDEV_FRONTEND=y
> # Xen driver support
> CONFIG_XEN_BALLOON=y
> CONFIG_XEN_SCRUB_PAGES=y
> CONFIG_XEN_DEV_EVTCHN=y
> CONFIG_XEN_BACKEND=y
> CONFIG_XENFS=y
> CONFIG_XEN_COMPAT_XENFS=y
> CONFIG_XEN_SYS_HYPERVISOR=y
> CONFIG_XEN_XENBUS_FRONTEND=y
> CONFIG_XEN_GNTDEV=y
> CONFIG_XEN_GRANT_DEV_ALLOC=y
> CONFIG_SWIOTLB_XEN=y
> CONFIG_XEN_PCIDEV_BACKEND=y
> CONFIG_XEN_PRIVCMD=y
> CONFIG_XEN_ACPI_PROCESSOR=y
> # CONFIG_XEN_MCE_LOG is not set
> CONFIG_XEN_HAVE_PVMMU=y

Looks ok.

> The machine is still rebooting when booting in the Xen built kernel. I know
> these machines have a thing
> called VRM (Voltage Regulator Module). This module is needed for dual cpu
> configurations. The machine
> has only on CPU and so no VRM. I read somewhere there was a issue with VRM
> equipped machine.

Hmm.... can't help you with this.
I only have one machine with 2 CPU-sockets and both are filled with a real CPU.

But, have a read through:
http://lists.xenproject.org/archives/html/xen-devel/2005-12/msg00710.html

Maybe the VRM you have is dodgy, or maybe you need to disable APIC like the 
person in the above email.

> Your help is greatly appreciate as I am stumped over this.
> 
> Nick.
> ​

Which bootloader do you use?
Please provide the boot-line/command-line options you use for Xen and the 
kernel.

My grub.conf looks like:
********
title Xen + Gentoo Linux 3.16.5 - Xen auto-starting

root (hd0,1)

kernel /xen.gz dom0_mem=4GB,max:4G console=vga dom0_max_vcpus=1 dom0_vcpus_pin

module /kernel-3.16.5-gentoo dolvm root=/dev/vg_artemis_system/artemis_root 
rootfstype=ext4 softlevel=xen

module /initramfs-genkernel-x86_64-3.16.5-gentoo
*******

ignore the "softlevel=xen" part.
I have a xen-bootlevel with loading of all Xen domains (VMs) enabled.

--
Joost

Re: [gentoo-user] xen on new install reboots by itself

[lee]

"J. Roeleveld" <joost@antarean.org> writes:

> On Thursday, April 23, 2015 11:03:53 PM lee wrote:
>> "J. Roeleveld" <joost@antarean.org> writes:
>> >>
>> > I disagree. Been using Xen for over 10 years now and find it very easy to
>> > use. The documentation could be better on the Xen site itself, but there
>> > is plenty of decent documentation available via Google.
>> Then we just disagree about this.
>
> Do you have anything that you find insufficiently documented or is too difficult?

sure, lots

>> >>Virtualization with containers is basically as simple as running just
>> >>another daemon.
>> >>
>> > Not quite. I use virtualization to minimizer the physical hardware. Xen is
>> > easy for that. Containers are what chroot jails should have been. But
>> > there is no simple method to set these up when security isolation is your
>> > goal.
>> Containers or chroots?
>
> Containers.
> Chroots don't have much when it comes to isolation.

What exactly are the issues with containers?  Ppl seem to work on them
and to manage to make them more secure over time.

>> >>Which the "better" tool, or combination of tools is, depends on what
>> >>you
>> >>want to accomplish.  You could use containers in a VM, too, or use
>> >>virtualbox along with containers to run the odd VMs that require full
>> >>virtualzation.
>> >>
>> > Virtualbox is nice for a quick test. I wouldn't use it for production.
>> 
>> Why not?
>
> Several reasons:
>
> 1) I wouldn't trust a desktop application for a server

So that's a gut feeling?

> 2) The overhead from Virtualbox is quite high (still better then VMWare's 
> desktop versions though)

Overhead in which way?  I haven't done much with virtualbox yet and
merely found it rather easy to use, very useful and to just work fine.

Compared to containers, the overhead xen requires is enormous, and it
doesn't give you a stable system to run VMs on because dom0 is already
virtualized itself.  I don't know how that compares to virtualbox --- I
didn't have time to look into it and it just worked, allowing me to run
a VM on the fly on the same machine I'm working on without any ado.

That VM was simply a copy of a VM taken from a vmware server, and the
copy could be used without any conversion or anything.  You can't do
that with xen because you'll be having lots of trouble to convert the
VM, to convert the machine you're working on to xen and to get it to
work, to work around all the problems xen brings about ...  Some days
later you might finally have it working --- which is out of the question
because the VM is needed right away.  And virtualbox does just that.

I was really surprised that virtualbox worked that well.  Maybe xen will
get there some time.


-- 
Again we must be afraid of speaking of daemons for fear that daemons
might swallow us.  Finally, this fear has become reasonable.

Re: [gentoo-user] xen on new install reboots by itself

[lee]

"J. Roeleveld" <joost@antarean.org> writes:

> On Thursday, April 23, 2015 11:02:24 PM lee wrote:
>> hydra <hydrapolic@gmail.com> writes:
>> >  You mean the documentation at Gentoo about Xen sucks or the upstream
>> > 
>> > documentation? What information are you missing from there? Maybe we can
>> > add  the missing pieces for Xen being more accessible and easier to use,
>> > what do you think? :)
>> 
>> I mean the documentation they have on their wiki.  It's a confusing mess
>> referring to various version with which things are being done
>> differently.
>
> The problem here is the different "implementations" that exist:
> - Xen (install and configure yourself, toolset: 'xl' , 'xm' is deprecated)
> - Citrix and XCP (pre-configured, install on dedicated server, toolset: 'xcp')
> - OVM (Oracle's implementation, not sure which toolset they use)

Maybe, maybe not; the documentation is so confusing that I can't really
tell what it is talking about.

>> Could you add missing pieces about why power management --- as in
>> frequency scaling --- doesn't work
>
> What doesn't work with this?
> The following seems quite detailed:
> http://wiki.xen.org/wiki/Xen_power_management

There was some command to query what frequencies the CPUs are running
on, and it didn't give any output.  Documentation seems to claim that
xen can do power management automagically, yet there was no way to
verify what it actually does.

> And the commands listed there (for the hypervisor based option) work on my 
> server.
>
>> and what to do about keeping the time
>> in sync between all VMs when you find out that this doesn't work as the
>> documentation would have you think it does?
>
> In what way doesn't it work?
> The clocks are all synchronized and I don't need to use anything like 'ntpd'

The clocks were off by quite a bit after a while, and I had to use ntp
to get them in sync.  Some documentation claims you don't need ntp or
anything; some other documentation apparently tries to explain that
keeping the clocks in sync cannot work unless the CPU(s) have some
features having to do with clock consistency while they are in sleep
states, and yet other documentation seems to say that using ntp cannot
work because xen screws it off.  In the end, it was recommended to me to
use ntp, which I found to work.  There was no way to figure out what xen
was actually doing or not doing towards this, and nobody seemed to know
how to keep the clocks in sync, other than using ntp, which appears to
be deprecated.


-- 
Again we must be afraid of speaking of daemons for fear that daemons
might swallow us.  Finally, this fear has become reasonable.

Re: [gentoo-user] xen on new install reboots by itself

[J. Roeleveld]

On Friday, April 24, 2015 10:23:01 PM lee wrote:
> "J. Roeleveld" <joost@antarean.org> writes:
> > On Thursday, April 23, 2015 11:03:53 PM lee wrote:
> > Do you have anything that you find insufficiently documented or is too
> > difficult?
> sure, lots

Have you contacted the Xen project with this?

> > Containers.
> > Chroots don't have much when it comes to isolation.
> 
> What exactly are the issues with containers?  Ppl seem to work on them
> and to manage to make them more secure over time.

Lack of clear documentation on how to use them. All the examples online refer 
to systemd-only commands.

> >> >>Which the "better" tool, or combination of tools is, depends on what
> >> >>you
> >> >>want to accomplish.  You could use containers in a VM, too, or use
> >> >>virtualbox along with containers to run the odd VMs that require full
> >> >>virtualzation.
> >> >>
> >> > Virtualbox is nice for a quick test. I wouldn't use it for production.
> >> 
> >> Why not?
> > 
> > Several reasons:
> > 
> > 1) I wouldn't trust a desktop application for a server
> 
> So that's a gut feeling?

No, a combination of experience and common sense.
A desktop application dies when the desktop dies.

> > 2) The overhead from Virtualbox is quite high (still better then VMWare's
> > desktop versions though)
> 
> Overhead in which way?  I haven't done much with virtualbox yet and
> merely found it rather easy to use, very useful and to just work fine.

Virtualbox is easy when all you want is to quickly run a VM for a quick test.
It isn't designed to run multiple VMs with maximum performance.
In my experience I get on average 80% of the performance inside a Virtualbox 
VM when compared to running them on the machine directly. With Xen, I can get 
95%.
(This is using normal work-loads, lets not talk about 3D inside a VM)

> Compared to containers, the overhead xen requires is enormous,

Hardly comparable. Containers run inside the same kernel. With Xen, or any 
other virtualisation technology, you run a full OS.

> and it
> doesn't give you a stable system to run VMs on because dom0 is already
> virtualized itself.

Why doesn't it provide a stable system?
The dom0 has 1 task and 1 task only: Manage the VMs and resources provided to 
the VMs. That part can be made extremely stable.
My Lab machine (which only runs VMs for testing and development) currently has 
an uptime of over a year. In that time I've had VMs crashing because of bad 
code inside the VM. Not noticing any issues there. Neither with stability nor 
with performance.
My only interaction with the dom0 there is the create/destroy/start/stop/... 
VMs.

> I don't know how that compares to virtualbox --- I
> didn't have time to look into it and it just worked, allowing me to run
> a VM on the fly on the same machine I'm working on without any ado.

For that scenario, VirtualBox is quite well suited. I wouldn't run Xen on my 
desktop or laptop.

> That VM was simply a copy of a VM taken from a vmware server, and the
> copy could be used without any conversion or anything.

Good luck doing that when you installed the VMWare client tools and drivers 
inside a MS Windows VM.

> You can't do
> that with xen because you'll be having lots of trouble to convert the
> VM, to convert the machine you're working on to xen and to get it to
> work, to work around all the problems xen brings about ...  Some days
> later you might finally have it working --- which is out of the question
> because the VM is needed right away. And virtualbox does just that.

Look into the pre-configured versions of Xen, like what Citrix offers.
I can import VMs from VMWare as well without issue. (Apart from the VMWare 
client tools as mentioned, but Virtualbox has the same issues)
 
> I was really surprised that virtualbox worked that well.  Maybe xen will
> get there some time.

Xen already is there.

Please understand that Xen and Virtualbox have their own usecases:

Xen is for dedicated hosts running VMs 24/7

Virtualbox is for testing stuff quickly on a laptop/desktop

The only common part is that they both run VMs.

--
Joost

Re: [gentoo-user] xen on new install reboots by itself

[J. Roeleveld]

On Friday, April 24, 2015 10:24:06 PM lee wrote:
> "J. Roeleveld" <joost@antarean.org> writes:
> > On Thursday, April 23, 2015 11:02:24 PM lee wrote:
> >> hydra <hydrapolic@gmail.com> writes:
> >> >  You mean the documentation at Gentoo about Xen sucks or the upstream
> >> > 
> >> > documentation? What information are you missing from there? Maybe we
> >> > can
> >> > add  the missing pieces for Xen being more accessible and easier to
> >> > use,
> >> > what do you think? :)
> >> 
> >> I mean the documentation they have on their wiki.  It's a confusing mess
> >> referring to various version with which things are being done
> >> differently.
> > 
> > The problem here is the different "implementations" that exist:
> > - Xen (install and configure yourself, toolset: 'xl' , 'xm' is deprecated)
> > - Citrix and XCP (pre-configured, install on dedicated server, toolset:
> > 'xcp') - OVM (Oracle's implementation, not sure which toolset they use)
> 
> Maybe, maybe not; the documentation is so confusing that I can't really
> tell what it is talking about.

Where did you look?

> >> Could you add missing pieces about why power management --- as in
> >> frequency scaling --- doesn't work
> > 
> > What doesn't work with this?
> > The following seems quite detailed:
> > http://wiki.xen.org/wiki/Xen_power_management
> 
> There was some command to query what frequencies the CPUs are running
> on, and it didn't give any output.  Documentation seems to claim that
> xen can do power management automagically, yet there was no way to
> verify what it actually does.

It works here:
# xenpm get-cpufreq-para all
cpu id               : 0
affected_cpus        : 0
cpuinfo frequency    : max [3101000] min [1600000] cur [1600000]
scaling_driver       : acpi-cpufreq
scaling_avail_gov    : userspace performance powersave ondemand
current_governor     : ondemand
  ondemand specific  :
    sampling_rate    : max [10000000] min [10000] cur [20000]
    up_threshold     : 80
scaling_avail_freq   : 3101000 3100000 2900000 2700000 2500000 2300000 2100000 
1900000 1700000 *1600000
scaling frequency    : max [3101000] min [1600000] cur [1600000]
turbo mode           : enabled

<snipped identical results for other CPU-cores>

Looks like it's actually working and I never configured this.

> > And the commands listed there (for the hypervisor based option) work on my
> > server.
> > 
> >> and what to do about keeping the time
> >> in sync between all VMs when you find out that this doesn't work as the
> >> documentation would have you think it does?
> > 
> > In what way doesn't it work?
> > The clocks are all synchronized and I don't need to use anything like
> > 'ntpd'
> The clocks were off by quite a bit after a while, and I had to use ntp
> to get them in sync.  Some documentation claims you don't need ntp or
> anything; some other documentation apparently tries to explain that
> keeping the clocks in sync cannot work unless the CPU(s) have some
> features having to do with clock consistency while they are in sleep
> states, and yet other documentation seems to say that using ntp cannot
> work because xen screws it off.  In the end, it was recommended to me to
> use ntp, which I found to work.  There was no way to figure out what xen
> was actually doing or not doing towards this, and nobody seemed to know
> how to keep the clocks in sync, other than using ntp, which appears to
> be deprecated.

Which version did you try?
I remember having had clock-issues requiring ntp when I first started using Xen 
over 10 years ago.

--
Joost

Re: [gentoo-user] xen on new install reboots by itself

[symack]

[-- Attachment #1: Type: text/plain, Size: 1139 bytes --]

Hello Joost,

We are running Grub2. Bellow is my grub.cfg for Xen

menuentry 'Gentoo GNU/Linux, with Xen hypervisor' --class gentoo --class
gnu-linux --class gnu --class os --class xen $menuentry_id_option
'xen-gnulinux-simple-43fa46d6-a602-4281-9493-66faec5c096f' {
        insmod part_msdos
        insmod ext2
        set root='hd0,msdos1'
        if [ x$feature_platform_search_hint = xy ]; then
          search --no-floppy --fs-uuid --set=root --hint-bios=hd0,msdos1
--hint-efi=hd0,msdos1 --hint-baremetal=ahci0,msdos1
 66cdce01-cc3f-45ad-bea3-a64dff6db724
        else
          search --no-floppy --fs-uuid --set=root
66cdce01-cc3f-45ad-bea3-a64dff6db724
        fi
        echo    'Loading Xen xen ...'
        if [ "$grub_platform" = "pc" -o "$grub_platform" = "" ]; then
            xen_rm_opts=
        else
            xen_rm_opts="no-real-mode edd=off"
        fi
        multiboot       /xen.gz placeholder   ${xen_rm_opts}
        echo    'Loading Linux 3.17.7-gentoo ...'
        module  /vmlinuz-3.17.7-gentoo placeholder root=/dev/sda3 ro
 rootwait
​

This is starting to hurt....

N.

[-- Attachment #2: Type: text/html, Size: 1615 bytes --]

Re: [gentoo-user] xen on new install reboots by itself

[J. Roeleveld]

On 29 April 2015 00:34:10 CEST, symack <symack@gmail.com> wrote:
>Hello Joost,
>
>We are running Grub2. Bellow is my grub.cfg for Xen
>
>menuentry 'Gentoo GNU/Linux, with Xen hypervisor' --class gentoo
>--class
>gnu-linux --class gnu --class os --class xen $menuentry_id_option
>'xen-gnulinux-simple-43fa46d6-a602-4281-9493-66faec5c096f' {
>        insmod part_msdos
>        insmod ext2
>        set root='hd0,msdos1'
>        if [ x$feature_platform_search_hint = xy ]; then
>         search --no-floppy --fs-uuid --set=root --hint-bios=hd0,msdos1
>--hint-efi=hd0,msdos1 --hint-baremetal=ahci0,msdos1
> 66cdce01-cc3f-45ad-bea3-a64dff6db724
>        else
>          search --no-floppy --fs-uuid --set=root
>66cdce01-cc3f-45ad-bea3-a64dff6db724
>        fi
>        echo    'Loading Xen xen ...'
>        if [ "$grub_platform" = "pc" -o "$grub_platform" = "" ]; then
>            xen_rm_opts=
>        else
>            xen_rm_opts="no-real-mode edd=off"
>        fi
>        multiboot       /xen.gz placeholder   ${xen_rm_opts}
>        echo    'Loading Linux 3.17.7-gentoo ...'
>        module  /vmlinuz-3.17.7-gentoo placeholder root=/dev/sda3 ro
> rootwait
>​
>
>This is starting to hurt....
>
>N.

Symack,

I never got grub2 to play nice with Xen.
My Xen servers all use grub1.

--
Joost
-- 
Sent from my Android device with K-9 Mail. Please excuse my brevity.

Re: [gentoo-user] xen on new install reboots by itself

[hydra]

[-- Attachment #1: Type: text/plain, Size: 1637 bytes --]

On Wed, Apr 29, 2015 at 5:38 AM, J. Roeleveld <joost@antarean.org> wrote:

> On 29 April 2015 00:34:10 CEST, symack <symack@gmail.com> wrote:
> >Hello Joost,
> >
> >We are running Grub2. Bellow is my grub.cfg for Xen
> >
> >menuentry 'Gentoo GNU/Linux, with Xen hypervisor' --class gentoo
> >--class
> >gnu-linux --class gnu --class os --class xen $menuentry_id_option
> >'xen-gnulinux-simple-43fa46d6-a602-4281-9493-66faec5c096f' {
> >        insmod part_msdos
> >        insmod ext2
> >        set root='hd0,msdos1'
> >        if [ x$feature_platform_search_hint = xy ]; then
> >         search --no-floppy --fs-uuid --set=root --hint-bios=hd0,msdos1
> >--hint-efi=hd0,msdos1 --hint-baremetal=ahci0,msdos1
> > 66cdce01-cc3f-45ad-bea3-a64dff6db724
> >        else
> >          search --no-floppy --fs-uuid --set=root
> >66cdce01-cc3f-45ad-bea3-a64dff6db724
> >        fi
> >        echo    'Loading Xen xen ...'
> >        if [ "$grub_platform" = "pc" -o "$grub_platform" = "" ]; then
> >            xen_rm_opts=
> >        else
> >            xen_rm_opts="no-real-mode edd=off"
> >        fi
> >        multiboot       /xen.gz placeholder   ${xen_rm_opts}
> >        echo    'Loading Linux 3.17.7-gentoo ...'
> >        module  /vmlinuz-3.17.7-gentoo placeholder root=/dev/sda3 ro
> > rootwait
> >​
> >
> >This is starting to hurt....
> >
> >N.
>
> Symack,
>
> I never got grub2 to play nice with Xen.
> My Xen servers all use grub1.
>
> --
> Joost
> --
> Sent from my Android device with K-9 Mail. Please excuse my brevity.
>
>
I run like 20 xen machines on grub2 without problems.

[-- Attachment #2: Type: text/html, Size: 2469 bytes --]

Re: [gentoo-user] xen on new install reboots by itself

[J. Roeleveld]

On 29 April 2015 07:11:55 CEST, hydra <hydrapolic@gmail.com> wrote:
>On Wed, Apr 29, 2015 at 5:38 AM, J. Roeleveld <joost@antarean.org>
>wrote:
>
>> On 29 April 2015 00:34:10 CEST, symack <symack@gmail.com> wrote:
>> >Hello Joost,
>> >
>> >We are running Grub2. Bellow is my grub.cfg for Xen
>> >
>> >menuentry 'Gentoo GNU/Linux, with Xen hypervisor' --class gentoo
>> >--class
>> >gnu-linux --class gnu --class os --class xen $menuentry_id_option
>> >'xen-gnulinux-simple-43fa46d6-a602-4281-9493-66faec5c096f' {
>> >        insmod part_msdos
>> >        insmod ext2
>> >        set root='hd0,msdos1'
>> >        if [ x$feature_platform_search_hint = xy ]; then
>> >         search --no-floppy --fs-uuid --set=root
>--hint-bios=hd0,msdos1
>> >--hint-efi=hd0,msdos1 --hint-baremetal=ahci0,msdos1
>> > 66cdce01-cc3f-45ad-bea3-a64dff6db724
>> >        else
>> >          search --no-floppy --fs-uuid --set=root
>> >66cdce01-cc3f-45ad-bea3-a64dff6db724
>> >        fi
>> >        echo    'Loading Xen xen ...'
>> >        if [ "$grub_platform" = "pc" -o "$grub_platform" = "" ];
>then
>> >            xen_rm_opts=
>> >        else
>> >            xen_rm_opts="no-real-mode edd=off"
>> >        fi
>> >        multiboot       /xen.gz placeholder   ${xen_rm_opts}
>> >        echo    'Loading Linux 3.17.7-gentoo ...'
>> >        module  /vmlinuz-3.17.7-gentoo placeholder root=/dev/sda3 ro
>> > rootwait
>> >​
>> >
>> >This is starting to hurt....
>> >
>> >N.
>>
>> Symack,
>>
>> I never got grub2 to play nice with Xen.
>> My Xen servers all use grub1.
>>
>> --
>> Joost
>> --
>> Sent from my Android device with K-9 Mail. Please excuse my brevity.
>>
>>
>I run like 20 xen machines on grub2 without problems.

Can you provide your grub.cfg for comparison?

--
Joost
-- 
Sent from my Android device with K-9 Mail. Please excuse my brevity.

Re: [gentoo-user] xen on new install reboots by itself

[symack]

[-- Attachment #1: Type: text/plain, Size: 38 bytes --]

Yes please! grub2 cfg.

Nick.
​

[-- Attachment #2: Type: text/html, Size: 85 bytes --]

Re: [gentoo-user] xen on new install reboots by itself

[hydra]

[-- Attachment #1: Type: text/plain, Size: 1735 bytes --]

On Wed, Apr 29, 2015 at 2:37 PM, symack <symack@gmail.com> wrote:

> Yes please! grub2 cfg.
>
> Nick.
> ​
>

On a HP server with HW raid having /dev/sda with disklabel type dos:
/dev/sda1 /boot ext2
/dev/sda2 /root ext4
/dev/sda3 lvm

/etc/default/grub
GRUB_DEFAULT=saved
GRUB_CMDLINE_LINUX="panic=30 net.ifnames=0"
GRUB_CMDLINE_XEN="dom0_mem=4G"

/boot/grub/grubenv
saved_entry=Gentoo GNU/Linux, with Xen hypervisor

/boot has:
config-3.14.37
kernel-3.14.37
xen-4.4.2.gz
xen-4.4.gz -> xen-4.4.2.gz
xen-4.gz -> xen-4.4.2.gz
xen-syms-4.4.2
xen.gz -> xen-4.4.2.gz

grub.cfg looks like:
### BEGIN /etc/grub.d/20_linux_xen ###
menuentry 'Gentoo GNU/Linux, with Xen hypervisor' --class gentoo --class
gnu-linux --class gnu --class os --class xen $menuentry_id_option
'xen-gnulinux-simple-0316b3f3-07ca-4e07-ab1e-42cba9c6763d' {
        insmod part_msdos
        insmod ext2
        set root='hd0,msdos1'
        if [ x$feature_platform_search_hint = xy ]; then
          search --no-floppy --fs-uuid --set=root --hint-bios=hd0,msdos1
--hint-efi=hd0,msdos1 --hint-baremetal=ahci0,msdos1
be00dffd-2b2a-4a7f-831d-b1a728a51e4d
        else
          search --no-floppy --fs-uuid --set=root
be00dffd-2b2a-4a7f-831d-b1a728a51e4d
        fi
        echo    'Loading Xen xen ...'
        if [ "$grub_platform" = "pc" -o "$grub_platform" = "" ]; then
            xen_rm_opts=
        else
            xen_rm_opts="no-real-mode edd=off"
        fi
        multiboot       /xen.gz placeholder dom0_mem=4G ${xen_rm_opts}
        echo    'Loading Linux 3.14.37 ...'
        module  /kernel-3.14.37 placeholder root=/dev/sda2 ro panic=30
net.ifnames=0
}

This was generated by grub2-mkconfig.

[-- Attachment #2: Type: text/html, Size: 2423 bytes --]

Re: [gentoo-user] xen on new install reboots by itself

[lee]

"J. Roeleveld" <joost@antarean.org> writes:

> On Friday, April 24, 2015 10:24:06 PM lee wrote:
>> "J. Roeleveld" <joost@antarean.org> writes:
>> > On Thursday, April 23, 2015 11:02:24 PM lee wrote:
>> >> hydra <hydrapolic@gmail.com> writes:
>> >> >  You mean the documentation at Gentoo about Xen sucks or the upstream
>> >> > 
>> >> > documentation? What information are you missing from there? Maybe we
>> >> > can
>> >> > add  the missing pieces for Xen being more accessible and easier to
>> >> > use,
>> >> > what do you think? :)
>> >> 
>> >> I mean the documentation they have on their wiki.  It's a confusing mess
>> >> referring to various version with which things are being done
>> >> differently.
>> > 
>> > The problem here is the different "implementations" that exist:
>> > - Xen (install and configure yourself, toolset: 'xl' , 'xm' is deprecated)
>> > - Citrix and XCP (pre-configured, install on dedicated server, toolset:
>> > 'xcp') - OVM (Oracle's implementation, not sure which toolset they use)
>> 
>> Maybe, maybe not; the documentation is so confusing that I can't really
>> tell what it is talking about.
>
> Where did you look?

Everywhere I could find.  The xen wiki is particularly messy.

>> >> Could you add missing pieces about why power management --- as in
>> >> frequency scaling --- doesn't work
>> > 
>> > What doesn't work with this?
>> > The following seems quite detailed:
>> > http://wiki.xen.org/wiki/Xen_power_management
>> 
>> There was some command to query what frequencies the CPUs are running
>> on, and it didn't give any output.  Documentation seems to claim that
>> xen can do power management automagically, yet there was no way to
>> verify what it actually does.
>
> It works here:
> # xenpm get-cpufreq-para all
> cpu id               : 0
> affected_cpus        : 0
> cpuinfo frequency    : max [3101000] min [1600000] cur [1600000]
> scaling_driver       : acpi-cpufreq
> scaling_avail_gov    : userspace performance powersave ondemand
> current_governor     : ondemand
>   ondemand specific  :
>     sampling_rate    : max [10000000] min [10000] cur [20000]
>     up_threshold     : 80
> scaling_avail_freq   : 3101000 3100000 2900000 2700000 2500000 2300000 2100000 
> 1900000 1700000 *1600000
> scaling frequency    : max [3101000] min [1600000] cur [1600000]
> turbo mode           : enabled
>
> <snipped identical results for other CPU-cores>
>
> Looks like it's actually working and I never configured this.

It didn't work for me.

>> > And the commands listed there (for the hypervisor based option) work on my
>> > server.
>> > 
>> >> and what to do about keeping the time
>> >> in sync between all VMs when you find out that this doesn't work as the
>> >> documentation would have you think it does?
>> > 
>> > In what way doesn't it work?
>> > The clocks are all synchronized and I don't need to use anything like
>> > 'ntpd'
>> The clocks were off by quite a bit after a while, and I had to use ntp
>> to get them in sync.  Some documentation claims you don't need ntp or
>> anything; some other documentation apparently tries to explain that
>> keeping the clocks in sync cannot work unless the CPU(s) have some
>> features having to do with clock consistency while they are in sleep
>> states, and yet other documentation seems to say that using ntp cannot
>> work because xen screws it off.  In the end, it was recommended to me to
>> use ntp, which I found to work.  There was no way to figure out what xen
>> was actually doing or not doing towards this, and nobody seemed to know
>> how to keep the clocks in sync, other than using ntp, which appears to
>> be deprecated.
>
> Which version did you try?
> I remember having had clock-issues requiring ntp when I first started using Xen 
> over 10 years ago.

The version in Debian --- I don't remember which one it was.  Debian was
the only distribution I could get it to work with at all, and the VMs
also were Debian because there isn't a good way to install an operating
system in a VM.


-- 
Again we must be afraid of speaking of daemons for fear that daemons
might swallow us.  Finally, this fear has become reasonable.

Re: [gentoo-user] xen on new install reboots by itself

[lee]

"J. Roeleveld" <joost@antarean.org> writes:

> On Friday, April 24, 2015 10:23:01 PM lee wrote:
>> "J. Roeleveld" <joost@antarean.org> writes:
>> > On Thursday, April 23, 2015 11:03:53 PM lee wrote:
>> > Do you have anything that you find insufficiently documented or is too
>> > difficult?
>> sure, lots
>
> Have you contacted the Xen project with this?

I've been asking questions on mailing lists.  What do you expect?  I
could tell them "your documentation sucks" and they might say "go ahead
and improve it then".  I tried to improve it the little bit I could;
it's on the wiki, if it's still there.

>> > Containers.
>> > Chroots don't have much when it comes to isolation.
>> 
>> What exactly are the issues with containers?  Ppl seem to work on them
>> and to manage to make them more secure over time.
>
> Lack of clear documentation on how to use them. All the examples online refer 
> to systemd-only commands.

True, there isn't much, if any, clear documentation.  I followed the
Gentoo wiki and it's working fine, though.

>> >> > Virtualbox is nice for a quick test. I wouldn't use it for production.
>> >> 
>> >> Why not?
>> > 
>> > Several reasons:
>> > 
>> > 1) I wouldn't trust a desktop application for a server
>> 
>> So that's a gut feeling?
>
> No, a combination of experience and common sense.
> A desktop application dies when the desktop dies.

You cannot run it from the command line?  It only runs in an X session?
If that is so, I'm going to need something else.

>> > 2) The overhead from Virtualbox is quite high (still better then VMWare's
>> > desktop versions though)
>> 
>> Overhead in which way?  I haven't done much with virtualbox yet and
>> merely found it rather easy to use, very useful and to just work fine.
>
> Virtualbox is easy when all you want is to quickly run a VM for a quick test.
> It isn't designed to run multiple VMs with maximum performance.
> In my experience I get on average 80% of the performance inside a Virtualbox 
> VM when compared to running them on the machine directly. With Xen, I can get 
> 95%.
> (This is using normal work-loads, lets not talk about 3D inside a VM)

Someone told me that you may find xen reducing the performance by up to
40%.

>> Compared to containers, the overhead xen requires is enormous,
>
> Hardly comparable. Containers run inside the same kernel. With Xen, or any 
> other virtualisation technology, you run a full OS.

How is that not comparable?

You don't need to run a full OS, and you're not stuck with fixed memory
assignments without even the ability to overcommit when you use
containers.  With xen, you're stuck with what you initially assigned,
may your VM currently use it or not.

If my mail server was a xen VM, I'd have assigned 2GB to it; as a
container, it uses less than one.  If the machine I'm working on was a
xen VM, I'd have assigned at least 16GB to it; as the host of the
container, it costs me nothing.

So obviously, the overhead required by xen is enormous.

>
>> and it
>> doesn't give you a stable system to run VMs on because dom0 is already
>> virtualized itself.
>
> Why doesn't it provide a stable system?
> The dom0 has 1 task and 1 task only: Manage the VMs and resources provided to 
> the VMs. That part can be made extremely stable.

It's already virtualized itself.  With containers, I have a
non-virtualized system as usual, as stable as they are.  A container is
like just another service I can start or stop, and I can access it
easily because it simply resides under /etc/lxc while I can use the host
for whatever else I'm doing.

With xen, I have a virtualized system to begin with, which is wasted
because it's only purpose is to provide a way to maintain other VMs.  I
can't fully use any of these VMs because for what I'm doing, I'd have to
pass through my NVDIA card to one of them.  IIUC, I wouldn't even be
able to log in to the host because it won't have a graphics card,
provided that I actually could pass the graphics card through, which
appears to be pretty much impossible.

The VMs would reside on LVM volumes and be hardly accessible --- though
now I'd use ZFS subvolumes, making that as easy as with containers.

Power management wouldn't work.  The xen documentation sucks. Everything
would be difficult and troublesome.  It's extremely difficult to install
a VM --- I never figured out how to actually do that.  I'd be hugely
wasting resources.

I'd never have the feeling that there is a stable platform to work with,
and it's not something I would want to have to maintain.

> My Lab machine (which only runs VMs for testing and development) currently has 
> an uptime of over a year. In that time I've had VMs crashing because of bad 
> code inside the VM. Not noticing any issues there. Neither with stability nor 
> with performance.
> My only interaction with the dom0 there is the create/destroy/start/stop/... 
> VMs.

How can you use it for testing when it's so ridiculously difficult to
install a VM?  How do you do updates without rebooting when a new kernel
version comes along?  How do you adjust resource allocations depending
on changing workloads?

>> I don't know how that compares to virtualbox --- I
>> didn't have time to look into it and it just worked, allowing me to run
>> a VM on the fly on the same machine I'm working on without any ado.
>
> For that scenario, VirtualBox is quite well suited. I wouldn't run Xen on my 
> desktop or laptop.
>
>> That VM was simply a copy of a VM taken from a vmware server, and the
>> copy could be used without any conversion or anything.
>
> Good luck doing that when you installed the VMWare client tools and drivers 
> inside a MS Windows VM.

That's what it was.

It gets troublesome when the VM is distributed across multiple files ---
I started trying to convert that and never had the time to finish.

>> You can't do
>> that with xen because you'll be having lots of trouble to convert the
>> VM, to convert the machine you're working on to xen and to get it to
>> work, to work around all the problems xen brings about ...  Some days
>> later you might finally have it working --- which is out of the question
>> because the VM is needed right away. And virtualbox does just that.
>
> Look into the pre-configured versions of Xen, like what Citrix offers.
> I can import VMs from VMWare as well without issue. (Apart from the VMWare 
> client tools as mentioned, but Virtualbox has the same issues)

Citrix is commercial, isn't it?

IIRC, I tried to get xen to work with Centos (no chance), something else
which I don't remember and finally Debian.  Debian worked, but it is a
total mess with too much ancient software and their backports, requiring
backports kernels for xen and because of kernel bugs and problems with
dracut/initramfs-tools (which are supposed to be fixed now).

Add to that the sucking documentation of xen, things like power
management not working and all the quirks like the clocks being offset
despite the docs saying that they will be synced automagically, the
impossibility to install an OS in a VM and the enormous waste of
resources, and I really don't want to use xen --- particularly not for
production.  It's simply too troublesome.

>> I was really surprised that virtualbox worked that well.  Maybe xen will
>> get there some time.
>
> Xen already is there.

Not by far, it doesn't even have good documentation yet and is too
troublesome and too difficult to use.

> Please understand that Xen and Virtualbox have their own usecases:
>
> Xen is for dedicated hosts running VMs 24/7

If you can get it to work and if you have the time to get it there, it's
a nice solution for VMs.  That's very big ifs, not even to mention to
work around the quirks.

> Virtualbox is for testing stuff quickly on a laptop/desktop

If it works only for that, what's the alternative?  Sooner than later,
I'll have to set up some Windoze VMs at work, and I really don't want to
use xen.


-- 
Again we must be afraid of speaking of daemons for fear that daemons
might swallow us.  Finally, this fear has become reasonable.