* [gentoo-user] QEMU on a partition @ 2018-03-02 11:12 Helmut Jarausch 2018-03-02 11:34 ` Peter Humphrey 0 siblings, 1 reply; 12+ messages in thread From: Helmut Jarausch @ 2018-03-02 11:12 UTC (permalink / raw To: gentoo-user Hi, I'd like to install a second Gentoo system on a partition by running QEMU using that partition (directly) - this is to create and update a Gentoo system with different CFLAGS (for an older machine). Having no experience in such setups my initial problem is how to install grub2 on that partition (only). I don't want to modify the MBR of the whole drive containing that partition. Is is possible? Many thanks for some hints or pointers, Helmut ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [gentoo-user] QEMU on a partition 2018-03-02 11:12 [gentoo-user] QEMU on a partition Helmut Jarausch @ 2018-03-02 11:34 ` Peter Humphrey 2018-03-02 15:00 ` Mick 0 siblings, 1 reply; 12+ messages in thread From: Peter Humphrey @ 2018-03-02 11:34 UTC (permalink / raw To: gentoo-user On Friday, 2 March 2018 11:12:36 GMT Helmut Jarausch wrote: > Hi, > > I'd like to install a second Gentoo system on a partition by running > QEMU using that partition (directly) - this is to create and update a > Gentoo > system with different CFLAGS (for an older machine). > > Having no experience in such setups my initial problem is how to > install grub2 on that partition (only). I don't want to modify the MBR > of the whole drive containing that partition. I do the same for my Atom machine by NFS-exporting its /usr/portage to a chroot on my main machine. The question of booting doesn't arise; all that's needed is a copy of /etc/portage* and the world file. If that sounds interesting I can show you some more detail. * Things like -march and --jobs differ to suit the host machine, but that's about all. -- Regards, Peter. ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [gentoo-user] QEMU on a partition 2018-03-02 11:34 ` Peter Humphrey @ 2018-03-02 15:00 ` Mick 2018-03-02 15:33 ` R0b0t1 0 siblings, 1 reply; 12+ messages in thread From: Mick @ 2018-03-02 15:00 UTC (permalink / raw To: gentoo-user [-- Attachment #1: Type: text/plain, Size: 1571 bytes --] On Friday, 2 March 2018 11:34:09 GMT Peter Humphrey wrote: > On Friday, 2 March 2018 11:12:36 GMT Helmut Jarausch wrote: > > Hi, > > > > I'd like to install a second Gentoo system on a partition by running > > QEMU using that partition (directly) - this is to create and update a > > Gentoo > > system with different CFLAGS (for an older machine). The QEMU guest's complete filesystem is contained within a *virtual* disk image. As far as the host OS is concerned, the guest's disk with all partitions on it, is a file. You can store this image file wherever you want and map the QEMU on the host to launch it. > > Having no experience in such setups my initial problem is how to > > install grub2 on that partition (only). I don't want to modify the MBR > > of the whole drive containing that partition. You do not install the guest's GRUB or any other boot loader on the host's partition. You install it within the virtual disk after you have launched the guest having attached a LiveCD to it, using QEMU. > I do the same for my Atom machine by NFS-exporting its /usr/portage to a > chroot on my main machine. The question of booting doesn't arise; all that's > needed is a copy of /etc/portage* and the world file. If that sounds > interesting I can show you some more detail. > > * Things like -march and --jobs differ to suit the host machine, but that's > about all. As noted above you'll need to set up CFLAGS in the guest's make.conf file to suit the *guest* platform and its CPU. Setting up "-march=native" won't work here. -- Regards, Mick [-- Attachment #2: This is a digitally signed message part. --] [-- Type: application/pgp-signature, Size: 833 bytes --] ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [gentoo-user] QEMU on a partition 2018-03-02 15:00 ` Mick @ 2018-03-02 15:33 ` R0b0t1 2018-03-02 16:45 ` Mick 2018-03-02 23:51 ` [gentoo-user] " Grant Taylor 0 siblings, 2 replies; 12+ messages in thread From: R0b0t1 @ 2018-03-02 15:33 UTC (permalink / raw To: gentoo-user On Fri, Mar 2, 2018 at 9:00 AM, Mick <michaelkintzios@gmail.com> wrote: > On Friday, 2 March 2018 11:34:09 GMT Peter Humphrey wrote: >> On Friday, 2 March 2018 11:12:36 GMT Helmut Jarausch wrote: >> > Hi, >> > >> > I'd like to install a second Gentoo system on a partition by running >> > QEMU using that partition (directly) - this is to create and update a >> > Gentoo >> > system with different CFLAGS (for an older machine). > > The QEMU guest's complete filesystem is contained within a *virtual* disk > image. As far as the host OS is concerned, the guest's disk with all > partitions on it, is a file. You can store this image file wherever you want > and map the QEMU on the host to launch it. > You can pass a block device directly to QEMU, and this is recommended for performance reasons. I have a Windows 10 VM that was passed an entire SSD; it runs fine, and you can take the disk and plug it into other computers. Passing a partition is a little different, if you wish to load it directly, you would need to chainload it with GRUB, as the MBR/GPT information would be duplicated. All OP needs to do is pass something like "-drive file=/dev/block,if=virtio". There should be more options, such as AIO implementation, but you likely won't need to mess with them. > >> > Having no experience in such setups my initial problem is how to >> > install grub2 on that partition (only). I don't want to modify the MBR >> > of the whole drive containing that partition. > > You do not install the guest's GRUB or any other boot loader on the host's > partition. You install it within the virtual disk after you have launched the > guest having attached a LiveCD to it, using QEMU. > If you pass a block device the MBR/GPT information will be stored there. In the case of passing a partition, this means you can't boot it "directly" because the BIOS/EFI firmware can't read it. > >> I do the same for my Atom machine by NFS-exporting its /usr/portage to a >> chroot on my main machine. The question of booting doesn't arise; all that's >> needed is a copy of /etc/portage* and the world file. If that sounds >> interesting I can show you some more detail. >> >> * Things like -march and --jobs differ to suit the host machine, but that's >> about all. > > As noted above you'll need to set up CFLAGS in the guest's make.conf file to > suit the *guest* platform and its CPU. Setting up "-march=native" won't work > here. > QEMU should, by default, emulate the host processor exactly if your machine has VT-x. The VM runs directly on the processor and all instructions are available except the virtualization ones, but you can enable nested VT-x or the AMD equivalent. Cheers, R0b0t1 ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [gentoo-user] QEMU on a partition 2018-03-02 15:33 ` R0b0t1 @ 2018-03-02 16:45 ` Mick 2018-03-02 17:22 ` R0b0t1 2018-03-02 23:56 ` Grant Taylor 2018-03-02 23:51 ` [gentoo-user] " Grant Taylor 1 sibling, 2 replies; 12+ messages in thread From: Mick @ 2018-03-02 16:45 UTC (permalink / raw To: gentoo-user [-- Attachment #1: Type: text/plain, Size: 271 bytes --] On Friday, 2 March 2018 15:33:02 GMT R0b0t1 wrote: > You can pass a block device directly to QEMU, and this is recommended > for performance reasons. Does it make a measurable difference, after the guest OS has booted? I'll need to try this out. :-) -- Regards, Mick [-- Attachment #2: This is a digitally signed message part. --] [-- Type: application/pgp-signature, Size: 833 bytes --] ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [gentoo-user] QEMU on a partition 2018-03-02 16:45 ` Mick @ 2018-03-02 17:22 ` R0b0t1 2018-03-02 23:56 ` Grant Taylor 1 sibling, 0 replies; 12+ messages in thread From: R0b0t1 @ 2018-03-02 17:22 UTC (permalink / raw To: gentoo-user On Fri, Mar 2, 2018 at 10:45 AM, Mick <michaelkintzios@gmail.com> wrote: > On Friday, 2 March 2018 15:33:02 GMT R0b0t1 wrote: >> You can pass a block device directly to QEMU, and this is recommended >> for performance reasons. > > Does it make a measurable difference, after the guest OS has booted? > > I'll need to try this out. :-) > If I remember right it makes a huge difference in raw IO speed, but one of the most noticeable benefits is that there are no delays as filesystem cache is flushed to service VM IO operations (which would be noticed outside of the VM as well). ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [gentoo-user] QEMU on a partition 2018-03-02 16:45 ` Mick 2018-03-02 17:22 ` R0b0t1 @ 2018-03-02 23:56 ` Grant Taylor 2018-03-03 2:12 ` R0b0t1 1 sibling, 1 reply; 12+ messages in thread From: Grant Taylor @ 2018-03-02 23:56 UTC (permalink / raw To: gentoo-user On 03/02/2018 09:45 AM, Mick wrote: > Does it make a measurable difference, after the guest OS has booted? IMHO, /bin/yes > I'll need to try this out. :-) Yes, you should try it out for yourself. I've found that removing not-strictly-needed layers of abstraction reduces complexity and makes things faster. I only use partitions (when using the entire disk) on PCs because so many recovery tools assume that they are there and work a LOT better than with raw disks. -- Grant. . . . unix || die ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [gentoo-user] QEMU on a partition 2018-03-02 23:56 ` Grant Taylor @ 2018-03-03 2:12 ` R0b0t1 2018-03-03 3:09 ` [gentoo-user] " Ian Zimmerman 0 siblings, 1 reply; 12+ messages in thread From: R0b0t1 @ 2018-03-03 2:12 UTC (permalink / raw To: gentoo-user On Fri, Mar 2, 2018 at 5:56 PM, Grant Taylor <gtaylor@gentoo.tnetconsulting.net> wrote: > I've found that removing not-strictly-needed layers of abstraction reduces > complexity and makes things faster. > I can't find it again, but there was a neat writeup investigating the TCP over TCP "tunnel collapse" phenomena. When two layers are doing the same thing, there is a tendency for both to behave poorly. I'm not sure any deeper explanation was or can be offered, but it is something that holds true not only for network traffic, but disk IO and databases as well. Cheers, R0b0t1 ^ permalink raw reply [flat|nested] 12+ messages in thread
* [gentoo-user] Re: QEMU on a partition 2018-03-03 2:12 ` R0b0t1 @ 2018-03-03 3:09 ` Ian Zimmerman 2018-03-03 12:54 ` Mick 0 siblings, 1 reply; 12+ messages in thread From: Ian Zimmerman @ 2018-03-03 3:09 UTC (permalink / raw To: gentoo-user On 2018-03-02 20:12, R0b0t1 wrote: > I can't find it again, but there was a neat writeup investigating the > TCP over TCP "tunnel collapse" phenomena. When two layers are doing > the same thing, there is a tendency for both to behave poorly. I'm not > sure any deeper explanation was or can be offered, but it is something > that holds true not only for network traffic, but disk IO and > databases as well. I think I've seen that too, and it was when I decided to install and learn openvpn in place of the everything-over-ssh setup I had before. -- Please don't Cc: me privately on mailing lists and Usenet, if you also post the followup to the list or newsgroup. To reply privately _only_ on Usenet and on broken lists which rewrite From, fetch the TXT record for no-use.mooo.com. ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [gentoo-user] Re: QEMU on a partition 2018-03-03 3:09 ` [gentoo-user] " Ian Zimmerman @ 2018-03-03 12:54 ` Mick 2018-03-03 18:52 ` Grant Taylor 0 siblings, 1 reply; 12+ messages in thread From: Mick @ 2018-03-03 12:54 UTC (permalink / raw To: gentoo-user [-- Attachment #1: Type: text/plain, Size: 1512 bytes --] On Saturday, 3 March 2018 03:09:25 GMT Ian Zimmerman wrote: > On 2018-03-02 20:12, R0b0t1 wrote: > > I can't find it again, but there was a neat writeup investigating the > > TCP over TCP "tunnel collapse" phenomena. When two layers are doing > > the same thing, there is a tendency for both to behave poorly. I'm not > > sure any deeper explanation was or can be offered, but it is something > > that holds true not only for network traffic, but disk IO and > > databases as well. > > I think I've seen that too, and it was when I decided to install and > learn openvpn in place of the everything-over-ssh setup I had before. I think the problem you mention refers to TCP retransmission timeouts, when you stack one TCP packet within another. RFC3439 warns against TCP layering: https://tools.ietf.org/html/rfc3439#page-7 UDP encapsulation as used for e.g. VPN does not suffer with the same problem because it does not use the same transmission quality control mechanism as TCP. I have used SSH within IPSec VPN tunnels without retransmission problems (both with and without UDP encapsulation). I am not sure if block device I/O protocols suffer the same problem - I don't really know how the read/write SCSI commands are queued and processed between host and guest OS. What I have noticed is abstraction layers relating to partitioning schemes, e.g. good ol' primary Vs logical partitions, make a difference *only* when the partition is initially mounted, but not thereafter. -- Regards, Mick [-- Attachment #2: This is a digitally signed message part. --] [-- Type: application/pgp-signature, Size: 833 bytes --] ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [gentoo-user] Re: QEMU on a partition 2018-03-03 12:54 ` Mick @ 2018-03-03 18:52 ` Grant Taylor 0 siblings, 0 replies; 12+ messages in thread From: Grant Taylor @ 2018-03-03 18:52 UTC (permalink / raw To: gentoo-user On 03/03/2018 05:54 AM, Mick wrote: > UDP encapsulation as used for e.g. VPN does not suffer with the same > problem because it does not use the same transmission quality control > mechanism as TCP. I think it's fair to say that it doesn't suffer at the protocol (TCP / UDP) level. There is nothing to prevent higher application layer retransmissions from compounding things. > I am not sure if block device I/O protocols suffer the same problem - > I don't really know how the read/write SCSI commands are queued and > processed between host and guest OS. What I have noticed is abstraction > layers relating to partitioning schemes, e.g. good ol' primary Vs logical > partitions, make a difference *only* when the partition is initially > mounted, but not thereafter. I've always operated under the assumption that there was additional logic ~> complexity, thus it must be slightly slower. That being said, I've long held that the performance overhead is extremely likely negligible and can be ignored. At least unless you are trying to squeeze every bit of performance out of something. I.e. HPC or low power / low speed devices. -- Grant. . . . unix || die ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [gentoo-user] QEMU on a partition 2018-03-02 15:33 ` R0b0t1 2018-03-02 16:45 ` Mick @ 2018-03-02 23:51 ` Grant Taylor 1 sibling, 0 replies; 12+ messages in thread From: Grant Taylor @ 2018-03-02 23:51 UTC (permalink / raw To: gentoo-user On 03/02/2018 08:33 AM, R0b0t1 wrote: > You can pass a block device directly to QEMU, and this is recommended > for performance reasons. I have a Windows 10 VM that was passed an > entire SSD; it runs fine, and you can take the disk and plug it into > other computers. Passing a partition is a little different, if you wish > to load it directly, you would need to chainload it with GRUB, as the > MBR/GPT information would be duplicated. Agreed on all accounts. > All OP needs to do is pass something like "-drive > file=/dev/block,if=virtio". There should be more options, such as AIO > implementation, but you likely won't need to mess with them. > If you pass a block device the MBR/GPT information will be stored > there. In the case of passing a partition, this means you can't boot it > "directly" because the BIOS/EFI firmware can't read it. I think that it might be possible to pass the partitions (FS & swap) as individual drives to the guest VM. Make sure that the guest VM mounts them by the UUID and not by path as the path in the VM and bare metal will be different. I've not tried this, but I think that it will work. Guest would "mkfs /dev/sda" and "mkswap /dev/sdb" -- Grant. . . . unix || die ^ permalink raw reply [flat|nested] 12+ messages in thread
end of thread, other threads:[~2018-03-03 18:52 UTC | newest] Thread overview: 12+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2018-03-02 11:12 [gentoo-user] QEMU on a partition Helmut Jarausch 2018-03-02 11:34 ` Peter Humphrey 2018-03-02 15:00 ` Mick 2018-03-02 15:33 ` R0b0t1 2018-03-02 16:45 ` Mick 2018-03-02 17:22 ` R0b0t1 2018-03-02 23:56 ` Grant Taylor 2018-03-03 2:12 ` R0b0t1 2018-03-03 3:09 ` [gentoo-user] " Ian Zimmerman 2018-03-03 12:54 ` Mick 2018-03-03 18:52 ` Grant Taylor 2018-03-02 23:51 ` [gentoo-user] " Grant Taylor
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox