* Re: [gentoo-user] Best filesystem for virtualized gentoo mail server - WAS: vmWare HowTo / best practices
2013-04-20 15:00 ` [gentoo-user] Best filesystem for virtualized gentoo mail server - WAS: " Tanstaafl
@ 2013-04-20 15:33 ` Alan McKinnon
2013-04-21 9:41 ` J. Roeleveld
2013-04-21 14:33 ` Tanstaafl
2013-04-20 15:38 ` Jarry
` (3 subsequent siblings)
4 siblings, 2 replies; 46+ messages in thread
From: Alan McKinnon @ 2013-04-20 15:33 UTC (permalink / raw
To: gentoo-user
On 20/04/2013 17:00, Tanstaafl wrote:
> Thanks for the responses so far...
>
> Another question - are there any caveats as to which filesystem to use
> for a mail server, for virtualized systems? Ir do the same
> issues/questions apply (ie, does the fact that it is virtualized not
> change anything)?
>
> If there are none, I'm curious what others prefer.
>
> I've been using reiserfs on my old mail server since it was first set up
> (over 8 years ago). I have had no issues with it whatsoever, and even
> had one scare with a bad UPS causing the system to experienc an unclean
> shutdown - but it came back up, auto fsck'd, and there was no 'apparent'
> data loss (this was a very long time ago, so if there had been any
> serious problems, I'd have known about it long go).
>
> I've been considering using XFS, but have never used it before.
>
> So, anyway, opinions are welcome...
Virtualization can change things, and it's not really intuitive.
Regardless of what optimizations you apply to the VM, and regardless of
what kind of virtualization is in use on the host, you are still going
to be bound by the disk and fs behaviour of the host. If VMWare gives
you a really shitty host driver, then something really shitty is going
to be the best you can achieve.
Disks aren't like eg NICs, you can't easily virtualize them and give the
guest exclusive access in the style of para-virtualization (I can't
imagine how that would even be done).
You also didn't mention what mail server you use - implementations vary
a great deal. Gut feel tells me that unless you are dealing with many
1000s of mails in a short period you won't really need XFS's aggressive
caching. But I'm happy to be proved wrong and numbers tell the truth :-)
I think the best you will get here is a list of combinations that are
unlikely to suit, and you will have to do your own extensive testing to
find what works best in your area.
FWIW, I have two mail relays (no mail storage) running old postfix
versions on FreeBSD. I expected throughput to differ when virtualized on
ESXi, but in practice I couldn't see a difference at all - maybe the
mail servers were very under-utilized. Considering this pair deal with
anything between 500,000 to a million mails a day total, I would not
have considered them "under-utilized". Just goes to show how opinions
are often worthless but numbers buys the whiskey :-)
--
Alan McKinnon
alan.mckinnon@gmail.com
^ permalink raw reply [flat|nested] 46+ messages in thread* Re: [gentoo-user] Best filesystem for virtualized gentoo mail server - WAS: vmWare HowTo / best practices
2013-04-20 15:33 ` Alan McKinnon
@ 2013-04-21 9:41 ` J. Roeleveld
2013-04-21 14:33 ` Tanstaafl
1 sibling, 0 replies; 46+ messages in thread
From: J. Roeleveld @ 2013-04-21 9:41 UTC (permalink / raw
To: gentoo-user
On Sat, April 20, 2013 17:33, Alan McKinnon wrote:
> On 20/04/2013 17:00, Tanstaafl wrote:
>> Thanks for the responses so far...
>>
>> Another question - are there any caveats as to which filesystem to use
>> for a mail server, for virtualized systems? Ir do the same
>> issues/questions apply (ie, does the fact that it is virtualized not
>> change anything)?
>>
>> If there are none, I'm curious what others prefer.
>>
>> I've been using reiserfs on my old mail server since it was first set up
>> (over 8 years ago). I have had no issues with it whatsoever, and even
>> had one scare with a bad UPS causing the system to experienc an unclean
>> shutdown - but it came back up, auto fsck'd, and there was no 'apparent'
>> data loss (this was a very long time ago, so if there had been any
>> serious problems, I'd have known about it long go).
>>
>> I've been considering using XFS, but have never used it before.
>>
>> So, anyway, opinions are welcome...
>
>
> Virtualization can change things, and it's not really intuitive.
>
> Regardless of what optimizations you apply to the VM, and regardless of
> what kind of virtualization is in use on the host, you are still going
> to be bound by the disk and fs behaviour of the host. If VMWare gives
> you a really shitty host driver, then something really shitty is going
> to be the best you can achieve.
This can be improved by not using file-backed disk devices.
Not sure if ESXi can do this. XenServer supports LVM-backed disk devices,
which reduces the overhead for the FS significantly.
> Disks aren't like eg NICs, you can't easily virtualize them and give the
> guest exclusive access in the style of para-virtualization (I can't
> imagine how that would even be done).
There is one option for this, but then you need to push the whole
disk-device for the VM. In other words you pass "sda" to the VM, instead
of a file or partition.
To avoid lousy I/O driver on the host, you could try passing the
disk-controller directly to the VM. But then the VM has it's own set of
disks that can not be used for other VMs.
> FWIW, I have two mail relays (no mail storage) running old postfix
> versions on FreeBSD. I expected throughput to differ when virtualized on
> ESXi, but in practice I couldn't see a difference at all - maybe the
> mail servers were very under-utilized. Considering this pair deal with
> anything between 500,000 to a million mails a day total, I would not
> have considered them "under-utilized". Just goes to show how opinions
> are often worthless but numbers buys the whiskey :-)
If Postfix only passes emails through, then it only uses the mail-spool
for temporary storage. For that, it doesn't require a lot of disk I/O.
Most filtering is done in memory, afaik.
My experience with ESXi is that it can have issues with networking when
the versions of the host don't match perfectly or the time is not synched
correctly or VMs are moved around a lot by an Admin that likes to play
with that...
Doing large multi-system installations on an ESXi cluster while the VMs
are moved around can occasionally fail because of a bad network-layer.
--
Joost
>
>
> --
> Alan McKinnon
> alan.mckinnon@gmail.com
>
>
>
>
^ permalink raw reply [flat|nested] 46+ messages in thread
* Re: [gentoo-user] Best filesystem for virtualized gentoo mail server - WAS: vmWare HowTo / best practices
2013-04-20 15:33 ` Alan McKinnon
2013-04-21 9:41 ` J. Roeleveld
@ 2013-04-21 14:33 ` Tanstaafl
2013-04-21 20:20 ` Alan McKinnon
2013-04-22 12:56 ` Andre Lucas Falco
1 sibling, 2 replies; 46+ messages in thread
From: Tanstaafl @ 2013-04-21 14:33 UTC (permalink / raw
To: gentoo-user
Thanks for the reply Alan...
On 2013-04-20 11:33 AM, Alan McKinnon <alan.mckinnon@gmail.com> wrote:
> If VMWare gives you a really shitty host driver, then something
> really shitty is going to be the best you can achieve.
The host is a Dell R515 with the Perc H700.
Windows VMs see get an 'LSI Logic SAS', and my gentoo VM gets an 'LSI
Logic Parallel' controller.
All disks are 15k rpm SAS (6G) drives.
> Disks aren't like eg NICs, you can't easily virtualize them and give the
> guest exclusive access in the style of para-virtualization (I can't
> imagine how that would even be done).
>
> You also didn't mention what mail server you use - implementations vary
> a great deal. Gut feel tells me that unless you are dealing with many
> 1000s of mails in a short period you won't really need XFS's aggressive
> caching.
Postfix+dovecot, still debating between maildir or mdbox, leaning toward
mdbox.
^ permalink raw reply [flat|nested] 46+ messages in thread
* Re: [gentoo-user] Best filesystem for virtualized gentoo mail server - WAS: vmWare HowTo / best practices
2013-04-21 14:33 ` Tanstaafl
@ 2013-04-21 20:20 ` Alan McKinnon
2013-04-22 12:56 ` Andre Lucas Falco
1 sibling, 0 replies; 46+ messages in thread
From: Alan McKinnon @ 2013-04-21 20:20 UTC (permalink / raw
To: gentoo-user
On 21/04/2013 16:33, Tanstaafl wrote:
> Thanks for the reply Alan...
>
> On 2013-04-20 11:33 AM, Alan McKinnon <alan.mckinnon@gmail.com> wrote:
>> If VMWare gives you a really shitty host driver, then something
>> really shitty is going to be the best you can achieve.
>
> The host is a Dell R515 with the Perc H700.
>
> Windows VMs see get an 'LSI Logic SAS', and my gentoo VM gets an 'LSI
> Logic Parallel' controller.
>
> All disks are 15k rpm SAS (6G) drives.
That's very similar to our setups with the exception of the host
hardware - we have R700 series hosts.
I'd be interested to keep track of what you find out and feed that back
to the sysadmins
>
>> Disks aren't like eg NICs, you can't easily virtualize them and give the
>> guest exclusive access in the style of para-virtualization (I can't
>> imagine how that would even be done).
>>
>> You also didn't mention what mail server you use - implementations vary
>> a great deal. Gut feel tells me that unless you are dealing with many
>> 1000s of mails in a short period you won't really need XFS's aggressive
>> caching.
>
> Postfix+dovecot, still debating between maildir or mdbox, leaning toward
> mdbox.
I always found postfix to be the best choice for an MTA in the large
picture (mostly because everyone else can also figure out what's going
on). I'd also be testing mdbox first just because maildir always leaves
me feeling a bit like there's a little too much disk IO going on and it
could be better
--
Alan McKinnon
alan.mckinnon@gmail.com
^ permalink raw reply [flat|nested] 46+ messages in thread
* Re: [gentoo-user] Best filesystem for virtualized gentoo mail server - WAS: vmWare HowTo / best practices
2013-04-21 14:33 ` Tanstaafl
2013-04-21 20:20 ` Alan McKinnon
@ 2013-04-22 12:56 ` Andre Lucas Falco
2013-04-23 19:28 ` PVSCSI vs LSI Logic Parallel/SAS - WAS: " Tanstaafl
1 sibling, 1 reply; 46+ messages in thread
From: Andre Lucas Falco @ 2013-04-22 12:56 UTC (permalink / raw
To: gentoo-user
[-- Attachment #1: Type: text/plain, Size: 238 bytes --]
2013/4/21 Tanstaafl <tanstaafl@libertytrek.org>
> Windows VMs see get an 'LSI Logic SAS', and my gentoo VM gets an 'LSI
> Logic Parallel' controller.
>
Did you tested using pvscsi? It's improve performance with less cost to CPU
usage.
[-- Attachment #2: Type: text/html, Size: 618 bytes --]
^ permalink raw reply [flat|nested] 46+ messages in thread
* PVSCSI vs LSI Logic Parallel/SAS - WAS: Re: [gentoo-user] Best filesystem for virtualized gentoo mail server - WAS: vmWare HowTo / best practices
2013-04-22 12:56 ` Andre Lucas Falco
@ 2013-04-23 19:28 ` Tanstaafl
2013-04-24 16:45 ` Tanstaafl
2013-04-25 2:23 ` Pandu Poluan
0 siblings, 2 replies; 46+ messages in thread
From: Tanstaafl @ 2013-04-23 19:28 UTC (permalink / raw
To: gentoo-user
On 2013-04-22 8:56 AM, Andre Lucas Falco <alfalco@gmail.com> wrote:
> 2013/4/21 Tanstaafl <tanstaafl@libertytrek.org> wrote:
>> Windows VMs see get an 'LSI Logic SAS', and my gentoo VM gets an
>> 'LSI Logic Parallel' controller.
> Did you tested using pvscsi? It's improve performance with less cost to
> CPU usage.
No, I didn't...
It appears there is pvscsi support in the mainline linux kernel, but is
it rock-solid? Anyone else here running gentoo linux with this driver
for their primary/boot disk controller?
Also, for my windows server 2008r2 vms, I used the default, which was
the LSI SAS... I did search and found the knowledgebase article
describing how to change them, but is the gain really worth the trouble
(and more importantly, the risk)?
^ permalink raw reply [flat|nested] 46+ messages in thread
* Re: PVSCSI vs LSI Logic Parallel/SAS - WAS: Re: [gentoo-user] Best filesystem for virtualized gentoo mail server - WAS: vmWare HowTo / best practices
2013-04-23 19:28 ` PVSCSI vs LSI Logic Parallel/SAS - WAS: " Tanstaafl
@ 2013-04-24 16:45 ` Tanstaafl
2013-04-25 2:23 ` Pandu Poluan
1 sibling, 0 replies; 46+ messages in thread
From: Tanstaafl @ 2013-04-24 16:45 UTC (permalink / raw
To: gentoo-user
Anyone?
On 2013-04-23 3:28 PM, Tanstaafl <tanstaafl@libertytrek.org> wrote:
> On 2013-04-22 8:56 AM, Andre Lucas Falco <alfalco@gmail.com> wrote:
>> 2013/4/21 Tanstaafl <tanstaafl@libertytrek.org> wrote:
>>> Windows VMs see get an 'LSI Logic SAS', and my gentoo VM gets an
>>> 'LSI Logic Parallel' controller.
>
>> Did you tested using pvscsi? It's improve performance with less cost to
>> CPU usage.
>
> No, I didn't...
>
> It appears there is pvscsi support in the mainline linux kernel, but is
> it rock-solid? Anyone else here running gentoo linux with this driver
> for their primary/boot disk controller?
>
> Also, for my windows server 2008r2 vms, I used the default, which was
> the LSI SAS... I did search and found the knowledgebase article
> describing how to change them, but is the gain really worth the trouble
> (and more importantly, the risk)?
>
^ permalink raw reply [flat|nested] 46+ messages in thread
* Re: PVSCSI vs LSI Logic Parallel/SAS - WAS: Re: [gentoo-user] Best filesystem for virtualized gentoo mail server - WAS: vmWare HowTo / best practices
2013-04-23 19:28 ` PVSCSI vs LSI Logic Parallel/SAS - WAS: " Tanstaafl
2013-04-24 16:45 ` Tanstaafl
@ 2013-04-25 2:23 ` Pandu Poluan
2013-04-25 10:53 ` Tanstaafl
1 sibling, 1 reply; 46+ messages in thread
From: Pandu Poluan @ 2013-04-25 2:23 UTC (permalink / raw
To: gentoo-user
[-- Attachment #1: Type: text/plain, Size: 1125 bytes --]
On Apr 24, 2013 2:29 AM, "Tanstaafl" <tanstaafl@libertytrek.org> wrote:
>
> On 2013-04-22 8:56 AM, Andre Lucas Falco <alfalco@gmail.com> wrote:
>>
>> 2013/4/21 Tanstaafl <tanstaafl@libertytrek.org> wrote:
>>>
>>> Windows VMs see get an 'LSI Logic SAS', and my gentoo VM gets an
>>> 'LSI Logic Parallel' controller.
>
>
>> Did you tested using pvscsi? It's improve performance with less cost to
>> CPU usage.
>
>
> No, I didn't...
>
> It appears there is pvscsi support in the mainline linux kernel, but is
it rock-solid? Anyone else here running gentoo linux with this driver for
their primary/boot disk controller?
>
> Also, for my windows server 2008r2 vms, I used the default, which was the
LSI SAS... I did search and found the knowledgebase article describing how
to change them, but is the gain really worth the trouble (and more
importantly, the risk)?
>
My Gentoo VMs in the cloud (using VMware's vCloud) uses PV-SCSI. It's
stable... but kind of sensitive: Everytime the cloud provider do something
with their storage, my VMs become Read-Only.
Other than that, performance is good, no fs corruption, etc.
Rgds,
--
[-- Attachment #2: Type: text/html, Size: 1526 bytes --]
^ permalink raw reply [flat|nested] 46+ messages in thread
* Re: PVSCSI vs LSI Logic Parallel/SAS - WAS: Re: [gentoo-user] Best filesystem for virtualized gentoo mail server - WAS: vmWare HowTo / best practices
2013-04-25 2:23 ` Pandu Poluan
@ 2013-04-25 10:53 ` Tanstaafl
2013-04-25 11:44 ` Pandu Poluan
0 siblings, 1 reply; 46+ messages in thread
From: Tanstaafl @ 2013-04-25 10:53 UTC (permalink / raw
To: gentoo-user
On 2013-04-24 10:23 PM, Pandu Poluan <pandu@poluan.info> wrote:
> My Gentoo VMs in the cloud (using VMware's vCloud) uses PV-SCSI. It's
> stable... but kind of sensitive: Everytime the cloud provider do
> something with their storage, my VMs become Read-Only.
Ouch... so, how do you fix it?
^ permalink raw reply [flat|nested] 46+ messages in thread
* Re: PVSCSI vs LSI Logic Parallel/SAS - WAS: Re: [gentoo-user] Best filesystem for virtualized gentoo mail server - WAS: vmWare HowTo / best practices
2013-04-25 10:53 ` Tanstaafl
@ 2013-04-25 11:44 ` Pandu Poluan
2013-04-25 12:18 ` Tanstaafl
0 siblings, 1 reply; 46+ messages in thread
From: Pandu Poluan @ 2013-04-25 11:44 UTC (permalink / raw
To: gentoo-user
[-- Attachment #1: Type: text/plain, Size: 1404 bytes --]
On Apr 25, 2013 5:54 PM, "Tanstaafl" <tanstaafl@libertytrek.org> wrote:
>
> On 2013-04-24 10:23 PM, Pandu Poluan <pandu@poluan.info> wrote:
>>
>> My Gentoo VMs in the cloud (using VMware's vCloud) uses PV-SCSI. It's
>> stable... but kind of sensitive: Everytime the cloud provider do
>> something with their storage, my VMs become Read-Only.
>
>
> Ouch... so, how do you fix it?
>
For the Read-Only problem, a simple reboot suffices.
But I made sure to fire an angry email to the Customer Support, telling
them in no uncertain terms that the next time they want to do something re:
their storage, they contact me first.
So, when the time comes for then to do some storage management things, they
tell me exactly at what time. I scheduled the VMs to shut down at the
agreed time, and turn them back on as soon as they text me that their
maintenance had finished. Only one VM I left on, the "gatewall" VM. And
every single time the gatewall VM detected that something is being done on
the storage level, and changed to a Read-Only mode.
I consider myself lucky that when the 'problem' manifested itself, the VMs
are not yet in Production. And no corruption happened.
(PS: Strangely enough, the RO switch happens only on Gentoo VMs; the
FreeBSD VMs and Debian VMs are unaffected. Maybe because I've pared down
each and every Gentoo VM to their bare minimum, so they are much more
responsive)
Rgds,
--
[-- Attachment #2: Type: text/html, Size: 1705 bytes --]
^ permalink raw reply [flat|nested] 46+ messages in thread
* Re: PVSCSI vs LSI Logic Parallel/SAS - WAS: Re: [gentoo-user] Best filesystem for virtualized gentoo mail server - WAS: vmWare HowTo / best practices
2013-04-25 11:44 ` Pandu Poluan
@ 2013-04-25 12:18 ` Tanstaafl
0 siblings, 0 replies; 46+ messages in thread
From: Tanstaafl @ 2013-04-25 12:18 UTC (permalink / raw
To: gentoo-user
On 2013-04-25 7:44 AM, Pandu Poluan <pandu@poluan.info> wrote:
> On Apr 25, 2013 5:54 PM, "Tanstaafl" wrote:
>> On 2013-04-24 10:23 PM, Pandu Poluan wrote:
>>> My Gentoo VMs in the cloud (using VMware's vCloud) uses PV-SCSI. It's
>>> stable... but kind of sensitive: Everytime the cloud provider do
>>> something with their storage, my VMs become Read-Only.
>> Ouch... so, how do you fix it?
> For the Read-Only problem, a simple reboot suffices.
>
> But I made sure to fire an angry email to the Customer Support, telling
> them in no uncertain terms that the next time they want to do something
> re: their storage, they contact me first.
Interesting... thanks for the heads up, but since this is on my own
host, I won't need to worry about this.
^ permalink raw reply [flat|nested] 46+ messages in thread
* Re: [gentoo-user] Best filesystem for virtualized gentoo mail server - WAS: vmWare HowTo / best practices
2013-04-20 15:00 ` [gentoo-user] Best filesystem for virtualized gentoo mail server - WAS: " Tanstaafl
2013-04-20 15:33 ` Alan McKinnon
@ 2013-04-20 15:38 ` Jarry
2013-04-21 9:47 ` J. Roeleveld
2013-04-20 16:22 ` Pandu Poluan
` (2 subsequent siblings)
4 siblings, 1 reply; 46+ messages in thread
From: Jarry @ 2013-04-20 15:38 UTC (permalink / raw
To: gentoo-user
On 20-Apr-13 17:00, Tanstaafl wrote:
> Another question - are there any caveats as to which filesystem to use
> for a mail server, for virtualized systems? Ir do the same
> issues/questions apply (ie, does the fact that it is virtualized not
> change anything)?
Problem of virtualized filesystem is not that it is virtualized,
but that it is located on datastore with more virtual systems,
all of them competing for the same i/o. *That* is the bottleneck.
If you switch reiser for xfs or btrfs, you might win (or loose)
a few %. If you optimize your esxi-datastore design, you might
win much more than what you have ever dreamed of.
I have 8 VMs (out of them 6 are Gentoo) hosted on ESXi, intended
for various tasks (mail, dns, mysql, web, etc), moderately loaded.
I used hw-raid controller with 2x sata-hdd in raid1 but performance
was quite dissapointing and I experienced all sorts of i/o jams.
Then I switched hdd for ssd (yes I use 2 of them in raid1, even
if this is not generally recommended) and performance rocks now!
I can start now kernel compilation on all 6 VMs at the same time,
with near-zero performance penalty (depending on cpu/vcpu ratio
and number of threads used). Unthinkable with hdd-based datastore.
I would definitely recommend using SSD. Either directly as
datastore for VMs, or at least as EXSi host-cache. There is
also possibility of "hybrid-raid" (1xSSD and 1xHDD in raid1)
on some raid-controllers. Or if your pocket is really deep,
you could grab one of those FusionIO-cards to avoid being
limited by rather slow sata-interface (SSD for PCIe)...
Jarry
--
_______________________________________________________________
This mailbox accepts e-mails only from selected mailing-lists!
Everything else is considered to be spam and therefore deleted.
^ permalink raw reply [flat|nested] 46+ messages in thread* Re: [gentoo-user] Best filesystem for virtualized gentoo mail server - WAS: vmWare HowTo / best practices
2013-04-20 15:38 ` Jarry
@ 2013-04-21 9:47 ` J. Roeleveld
2013-04-21 15:09 ` Tanstaafl
0 siblings, 1 reply; 46+ messages in thread
From: J. Roeleveld @ 2013-04-21 9:47 UTC (permalink / raw
To: gentoo-user
On Sat, April 20, 2013 17:38, Jarry wrote:
> On 20-Apr-13 17:00, Tanstaafl wrote:
>
>> Another question - are there any caveats as to which filesystem to use
>> for a mail server, for virtualized systems? Ir do the same
>> issues/questions apply (ie, does the fact that it is virtualized not
>> change anything)?
>
> Problem of virtualized filesystem is not that it is virtualized,
> but that it is located on datastore with more virtual systems,
> all of them competing for the same i/o. *That* is the bottleneck.
> If you switch reiser for xfs or btrfs, you might win (or loose)
> a few %. If you optimize your esxi-datastore design, you might
> win much more than what you have ever dreamed of.
If the underlying I/O is fast enough with low seek-times and high
throughput, that handling multiple VMs using a lot of disk I/O
simultaneously isn't a problem. Provided the Host has sufficient resources
(think memory and dedicated CPU) to handle it.
> I have 8 VMs (out of them 6 are Gentoo) hosted on ESXi, intended
> for various tasks (mail, dns, mysql, web, etc), moderately loaded.
> I used hw-raid controller with 2x sata-hdd in raid1 but performance
> was quite dissapointing and I experienced all sorts of i/o jams.
Which hw-raid controller did you use?
RAID-1 (mirroring) isn't actually known for high performance.
> Then I switched hdd for ssd (yes I use 2 of them in raid1, even
> if this is not generally recommended) and performance rocks now!
> I can start now kernel compilation on all 6 VMs at the same time,
> with near-zero performance penalty (depending on cpu/vcpu ratio
> and number of threads used). Unthinkable with hdd-based datastore.
I have HDD-based datastores and can do this on 4 VMs (single quad-core
CPU) without any penalty.
> I would definitely recommend using SSD. Either directly as
> datastore for VMs, or at least as EXSi host-cache. There is
> also possibility of "hybrid-raid" (1xSSD and 1xHDD in raid1)
> on some raid-controllers. Or if your pocket is really deep,
> you could grab one of those FusionIO-cards to avoid being
> limited by rather slow sata-interface (SSD for PCIe)...
A decent hardware raid-controller with multiple disks running in a higher
raid version is cheaper then the same storage capacity in SSDs.
--
Joost
^ permalink raw reply [flat|nested] 46+ messages in thread
* Re: [gentoo-user] Best filesystem for virtualized gentoo mail server - WAS: vmWare HowTo / best practices
2013-04-21 9:47 ` J. Roeleveld
@ 2013-04-21 15:09 ` Tanstaafl
2013-04-22 5:42 ` J. Roeleveld
0 siblings, 1 reply; 46+ messages in thread
From: Tanstaafl @ 2013-04-21 15:09 UTC (permalink / raw
To: gentoo-user
On 2013-04-21 5:47 AM, J. Roeleveld <joost@antarean.org> wrote:
> On Sat, April 20, 2013 17:38, Jarry wrote:
>> Problem of virtualized filesystem is not that it is virtualized,
>> but that it is located on datastore with more virtual systems,
>> all of them competing for the same i/o. *That* is the bottleneck.
>> If you switch reiser for xfs or btrfs, you might win (or loose)
>> a few %. If you optimize your esxi-datastore design, you might
>> win much more than what you have ever dreamed of.
>
> If the underlying I/O is fast enough with low seek-times and high
> throughput, that handling multiple VMs using a lot of disk I/O
> simultaneously isn't a problem. Provided the Host has sufficient resources
> (think memory and dedicated CPU) to handle it.
My host specs:
Dual AMD Opteron 4180 (6-core, 2.6Ghz)
128GB RAM
2x internal SSDs in RAID1 for Host OS
6x 300G SAS 6Gb 15k hard drives in RAID10 for Guest OSs
I allocate each Guest 1 virtual CPU with 2 cores
> A decent hardware raid-controller with multiple disks running in a higher
> raid version is cheaper then the same storage capacity in SSDs.
Yep... I toyed with the idea of SSDs, but the cost was considerably more
as compared to even these SAS drives...
^ permalink raw reply [flat|nested] 46+ messages in thread
* Re: [gentoo-user] Best filesystem for virtualized gentoo mail server - WAS: vmWare HowTo / best practices
2013-04-21 15:09 ` Tanstaafl
@ 2013-04-22 5:42 ` J. Roeleveld
0 siblings, 0 replies; 46+ messages in thread
From: J. Roeleveld @ 2013-04-22 5:42 UTC (permalink / raw
To: gentoo-user
On Sun, April 21, 2013 17:09, Tanstaafl wrote:
> On 2013-04-21 5:47 AM, J. Roeleveld <joost@antarean.org> wrote:
>> On Sat, April 20, 2013 17:38, Jarry wrote:
>>> Problem of virtualized filesystem is not that it is virtualized,
>>> but that it is located on datastore with more virtual systems,
>>> all of them competing for the same i/o. *That* is the bottleneck.
>>> If you switch reiser for xfs or btrfs, you might win (or loose)
>>> a few %. If you optimize your esxi-datastore design, you might
>>> win much more than what you have ever dreamed of.
>>
>> If the underlying I/O is fast enough with low seek-times and high
>> throughput, that handling multiple VMs using a lot of disk I/O
>> simultaneously isn't a problem. Provided the Host has sufficient
>> resources
>> (think memory and dedicated CPU) to handle it.
>
> My host specs:
>
> Dual AMD Opteron 4180 (6-core, 2.6Ghz)
> 128GB RAM
> 2x internal SSDs in RAID1 for Host OS
> 6x 300G SAS 6Gb 15k hard drives in RAID10 for Guest OSs
Sounds like a nice machine for testing :)
> I allocate each Guest 1 virtual CPU with 2 cores
Do you limit the Guest to use any of 2 specific cores? Or are you giving 2
vCPUs to each Guest?
>> A decent hardware raid-controller with multiple disks running in a
>> higher
>> raid version is cheaper then the same storage capacity in SSDs.
>
> Yep... I toyed with the idea of SSDs, but the cost was considerably more
> as compared to even these SAS drives...
I am planning on using SSDs when getting new desktops, but for servers I
prefer spinning disks. They're higher capacity and cheaper.
For speed, I just put a bunch of them together with hardware raid.
--
Joost
^ permalink raw reply [flat|nested] 46+ messages in thread
* Re: [gentoo-user] Best filesystem for virtualized gentoo mail server - WAS: vmWare HowTo / best practices
2013-04-20 15:00 ` [gentoo-user] Best filesystem for virtualized gentoo mail server - WAS: " Tanstaafl
2013-04-20 15:33 ` Alan McKinnon
2013-04-20 15:38 ` Jarry
@ 2013-04-20 16:22 ` Pandu Poluan
2013-04-21 9:51 ` J. Roeleveld
2013-04-21 16:15 ` [gentoo-user] LVM on VM or not? - WAS " Tanstaafl
2013-04-21 16:53 ` [gentoo-user] Partition layout questions " Tanstaafl
4 siblings, 1 reply; 46+ messages in thread
From: Pandu Poluan @ 2013-04-20 16:22 UTC (permalink / raw
To: gentoo-user
[-- Attachment #1: Type: text/plain, Size: 1841 bytes --]
On Apr 20, 2013 10:01 PM, "Tanstaafl" <tanstaafl@libertytrek.org> wrote:
>
> Thanks for the responses so far...
>
> Another question - are there any caveats as to which filesystem to use
for a mail server, for virtualized systems? Ir do the same issues/questions
apply (ie, does the fact that it is virtualized not change anything)?
>
> If there are none, I'm curious what others prefer.
>
> I've been using reiserfs on my old mail server since it was first set up
(over 8 years ago). I have had no issues with it whatsoever, and even had
one scare with a bad UPS causing the system to experienc an unclean
shutdown - but it came back up, auto fsck'd, and there was no 'apparent'
data loss (this was a very long time ago, so if there had been any serious
problems, I'd have known about it long go).
>
> I've been considering using XFS, but have never used it before.
>
> So, anyway, opinions are welcome...
>
> Thanks again
>
> Charles
>
Reiterating what others have said, in a virtualized environment, it's how
you build the underlying storage that will have the greatest effect on
performance.
Just an illustration: in my current employment, we have a very heavily used
database (SQL Server). To ensure good performance, I dedicated a RAID array
of 8 drives (15k RPM each), ensure that the space allocation is 'thick' not
'thin', and dedicate the whole RAID array to just that one VM. Performance
went through the roof with that one... especially since it was originally a
physical server running on top of 4 x 7200 RPM drives ;-)
If you have the budget, you really should invest in a SAN Storage solution
that can provide "tiered storage", in which frequently used blocks will be
'cached' in SSD, while less frequently used blocks are migrated first to
slower SAS drives, and later on (if 'cold') to even slower SATA drives.
Rgds,
--
[-- Attachment #2: Type: text/html, Size: 2175 bytes --]
^ permalink raw reply [flat|nested] 46+ messages in thread
* Re: [gentoo-user] Best filesystem for virtualized gentoo mail server - WAS: vmWare HowTo / best practices
2013-04-20 16:22 ` Pandu Poluan
@ 2013-04-21 9:51 ` J. Roeleveld
2013-04-21 10:06 ` Pandu Poluan
0 siblings, 1 reply; 46+ messages in thread
From: J. Roeleveld @ 2013-04-21 9:51 UTC (permalink / raw
To: gentoo-user
On Sat, April 20, 2013 18:22, Pandu Poluan wrote:
> On Apr 20, 2013 10:01 PM, "Tanstaafl" <tanstaafl@libertytrek.org> wrote:
>>
>> Thanks for the responses so far...
>>
>> Another question - are there any caveats as to which filesystem to use
> for a mail server, for virtualized systems? Ir do the same
> issues/questions
> apply (ie, does the fact that it is virtualized not change anything)?
>>
>> If there are none, I'm curious what others prefer.
>>
>> I've been using reiserfs on my old mail server since it was first set up
> (over 8 years ago). I have had no issues with it whatsoever, and even had
> one scare with a bad UPS causing the system to experienc an unclean
> shutdown - but it came back up, auto fsck'd, and there was no 'apparent'
> data loss (this was a very long time ago, so if there had been any serious
> problems, I'd have known about it long go).
>>
>> I've been considering using XFS, but have never used it before.
>>
>> So, anyway, opinions are welcome...
>>
>> Thanks again
>>
>> Charles
>>
>
> Reiterating what others have said, in a virtualized environment, it's how
> you build the underlying storage that will have the greatest effect on
> performance.
>
> Just an illustration: in my current employment, we have a very heavily
> used
> database (SQL Server). To ensure good performance, I dedicated a RAID
> array
> of 8 drives (15k RPM each), ensure that the space allocation is 'thick'
> not
> 'thin', and dedicate the whole RAID array to just that one VM. Performance
> went through the roof with that one... especially since it was originally
> a
> physical server running on top of 4 x 7200 RPM drives ;-)
>
> If you have the budget, you really should invest in a SAN Storage solution
> that can provide "tiered storage", in which frequently used blocks will be
> 'cached' in SSD, while less frequently used blocks are migrated first to
> slower SAS drives, and later on (if 'cold') to even slower SATA drives.
4-tier sounds nicer: 1 TB in high speed RAM for the high-speed layer, with
dedicated UPS to ensure this is backed up to disk on shutdown.
--
Joost
^ permalink raw reply [flat|nested] 46+ messages in thread
* Re: [gentoo-user] Best filesystem for virtualized gentoo mail server - WAS: vmWare HowTo / best practices
2013-04-21 9:51 ` J. Roeleveld
@ 2013-04-21 10:06 ` Pandu Poluan
2013-04-22 5:54 ` J. Roeleveld
0 siblings, 1 reply; 46+ messages in thread
From: Pandu Poluan @ 2013-04-21 10:06 UTC (permalink / raw
To: gentoo-user
[-- Attachment #1: Type: text/plain, Size: 2654 bytes --]
On Apr 21, 2013 4:51 PM, "J. Roeleveld" <joost@antarean.org> wrote:
>
> On Sat, April 20, 2013 18:22, Pandu Poluan wrote:
> > On Apr 20, 2013 10:01 PM, "Tanstaafl" <tanstaafl@libertytrek.org> wrote:
> >>
> >> Thanks for the responses so far...
> >>
> >> Another question - are there any caveats as to which filesystem to use
> > for a mail server, for virtualized systems? Ir do the same
> > issues/questions
> > apply (ie, does the fact that it is virtualized not change anything)?
> >>
> >> If there are none, I'm curious what others prefer.
> >>
> >> I've been using reiserfs on my old mail server since it was first set
up
> > (over 8 years ago). I have had no issues with it whatsoever, and even
had
> > one scare with a bad UPS causing the system to experienc an unclean
> > shutdown - but it came back up, auto fsck'd, and there was no 'apparent'
> > data loss (this was a very long time ago, so if there had been any
serious
> > problems, I'd have known about it long go).
> >>
> >> I've been considering using XFS, but have never used it before.
> >>
> >> So, anyway, opinions are welcome...
> >>
> >> Thanks again
> >>
> >> Charles
> >>
> >
> > Reiterating what others have said, in a virtualized environment, it's
how
> > you build the underlying storage that will have the greatest effect on
> > performance.
> >
> > Just an illustration: in my current employment, we have a very heavily
> > used
> > database (SQL Server). To ensure good performance, I dedicated a RAID
> > array
> > of 8 drives (15k RPM each), ensure that the space allocation is 'thick'
> > not
> > 'thin', and dedicate the whole RAID array to just that one VM.
Performance
> > went through the roof with that one... especially since it was
originally
> > a
> > physical server running on top of 4 x 7200 RPM drives ;-)
> >
> > If you have the budget, you really should invest in a SAN Storage
solution
> > that can provide "tiered storage", in which frequently used blocks will
be
> > 'cached' in SSD, while less frequently used blocks are migrated first to
> > slower SAS drives, and later on (if 'cold') to even slower SATA drives.
>
> 4-tier sounds nicer: 1 TB in high speed RAM for the high-speed layer, with
> dedicated UPS to ensure this is backed up to disk on shutdown.
>
Indeed! But 1 TB is kind of overkill, if you ask me... :-D
VMware and XenServer can 'talk' with some Storage controllers, where they
conspire in the background to provide 'victim cache' on the virtualization
host. Not sure about Hyper-V.
I myself had had good experience relying on EMC VNX's internal 8 GB cache;
apparently the workload is not high enough to stress the system.
Rgds,
--
[-- Attachment #2: Type: text/html, Size: 3465 bytes --]
^ permalink raw reply [flat|nested] 46+ messages in thread
* Re: [gentoo-user] Best filesystem for virtualized gentoo mail server - WAS: vmWare HowTo / best practices
2013-04-21 10:06 ` Pandu Poluan
@ 2013-04-22 5:54 ` J. Roeleveld
0 siblings, 0 replies; 46+ messages in thread
From: J. Roeleveld @ 2013-04-22 5:54 UTC (permalink / raw
To: gentoo-user
On Sun, April 21, 2013 12:06, Pandu Poluan wrote:
> On Apr 21, 2013 4:51 PM, "J. Roeleveld" <joost@antarean.org> wrote:
>>
>> On Sat, April 20, 2013 18:22, Pandu Poluan wrote:
> > > If you have the budget, you really should invest in a SAN
> > > Storage solution that can provide "tiered storage", in which
> > > frequently used blocks will be 'cached' in SSD, while less
> > > frequently used blocks are migrated first to slower SAS
> > > drives, and later on (if 'cold') to even slower SATA
> > > drives.
> >
> > 4-tier sounds nicer: 1 TB in high speed RAM for the high-speed
> > layer, with dedicated UPS to ensure this is backed up to disk
> > on shutdown.
>
> Indeed! But 1 TB is kind of overkill, if you ask me... :-D
Maybe, but when using that for VMs in a lab environment where you want to
create snapshots quickly.
> VMware and XenServer can 'talk' with some Storage controllers, where they
> conspire in the background to provide 'victim cache' on the virtualization
> host. Not sure about Hyper-V.
>
> I myself had had good experience relying on EMC VNX's internal 8 GB cache;
> apparently the workload is not high enough to stress the system.
When the time comes to upgrade the hardware, I will look into that. By
then, this technology should be more common as well.
--
Joost
^ permalink raw reply [flat|nested] 46+ messages in thread
* [gentoo-user] LVM on VM or not? - WAS Re: Best filesystem for virtualized gentoo mail server - WAS: vmWare HowTo / best practices
2013-04-20 15:00 ` [gentoo-user] Best filesystem for virtualized gentoo mail server - WAS: " Tanstaafl
` (2 preceding siblings ...)
2013-04-20 16:22 ` Pandu Poluan
@ 2013-04-21 16:15 ` Tanstaafl
2013-04-21 16:27 ` Jarry
2013-04-21 16:32 ` Randy Barlow
2013-04-21 16:53 ` [gentoo-user] Partition layout questions " Tanstaafl
4 siblings, 2 replies; 46+ messages in thread
From: Tanstaafl @ 2013-04-21 16:15 UTC (permalink / raw
To: gentoo-user
On 2013-04-20 11:00 AM, Tanstaafl <tanstaafl@libertytrek.org> wrote:
> Another question - are there any caveats as to which filesystem to use
> for a mail server, for virtualized systems?
Ok, googling reveals lots of conflicting opinions about using LVM in a
VM environment.
I was wanting to use it mainly for its snapshot ability (to get
consistent backups of my mailstore and mysql DBs).
Also it would be very nice to be able to resize things if needed (I have
adequate storage available).
But I've found lots of opinions that using LVM in a virtualized
environment can lead to data corruption, and if this is true, I'd rather
not risk it...
So, LVM or not?
^ permalink raw reply [flat|nested] 46+ messages in thread* Re: [gentoo-user] LVM on VM or not? - WAS Re: Best filesystem for virtualized gentoo mail server - WAS: vmWare HowTo / best practices
2013-04-21 16:15 ` [gentoo-user] LVM on VM or not? - WAS " Tanstaafl
@ 2013-04-21 16:27 ` Jarry
2013-04-21 16:56 ` Tanstaafl
2013-04-21 16:32 ` Randy Barlow
1 sibling, 1 reply; 46+ messages in thread
From: Jarry @ 2013-04-21 16:27 UTC (permalink / raw
To: gentoo-user
On 21-Apr-13 18:15, Tanstaafl wrote:
>
> Ok, googling reveals lots of conflicting opinions about using LVM in a
> VM environment.
>
> I was wanting to use it mainly for its snapshot ability (to get
> consistent backups of my mailstore and mysql DBs).
>
> Also it would be very nice to be able to resize things if needed (I have
> adequate storage available).
>
> But I've found lots of opinions that using LVM in a virtualized
> environment can lead to data corruption, and if this is true, I'd rather
> not risk it...
>
> So, LVM or not?
You can make snapshots from ESXi (btw snapshot is *not* backup),
and you can resize VM-disks as well. So the right question is:
What are the LVM features I need? If I do not need any, then why
should I bother with it?
Jarry
--
_______________________________________________________________
This mailbox accepts e-mails only from selected mailing-lists!
Everything else is considered to be spam and therefore deleted.
^ permalink raw reply [flat|nested] 46+ messages in thread
* Re: [gentoo-user] LVM on VM or not? - WAS Re: Best filesystem for virtualized gentoo mail server - WAS: vmWare HowTo / best practices
2013-04-21 16:27 ` Jarry
@ 2013-04-21 16:56 ` Tanstaafl
2013-04-21 17:31 ` Tanstaafl
2013-04-22 0:55 ` Randy Barlow
0 siblings, 2 replies; 46+ messages in thread
From: Tanstaafl @ 2013-04-21 16:56 UTC (permalink / raw
To: gentoo-user
On 2013-04-21 12:27 PM, Jarry <mr.jarry@gmail.com> wrote:
> On 21-Apr-13 18:15, Tanstaafl wrote:
>>
>> Ok, googling reveals lots of conflicting opinions about using LVM in a
>> VM environment.
>>
>> I was wanting to use it mainly for its snapshot ability (to get
>> consistent backups of my mailstore and mysql DBs).
>>
>> Also it would be very nice to be able to resize things if needed (I have
>> adequate storage available).
>>
>> But I've found lots of opinions that using LVM in a virtualized
>> environment can lead to data corruption, and if this is true, I'd rather
>> not risk it...
>>
>> So, LVM or not?
>
> You can make snapshots from ESXi (btw snapshot is *not* backup),
> and you can resize VM-disks as well. So the right question is:
> What are the LVM features I need? If I do not need any, then why
> should I bother with it?
Yes I can't take snapshots with ESXi, but everything I've read says that
for these to be consistent, they need to be done when the VM is shutdown.
Also, they take a LONG time, whereas an LVM snapshot happens almost
immediately.
^ permalink raw reply [flat|nested] 46+ messages in thread
* Re: [gentoo-user] LVM on VM or not? - WAS Re: Best filesystem for virtualized gentoo mail server - WAS: vmWare HowTo / best practices
2013-04-21 16:56 ` Tanstaafl
@ 2013-04-21 17:31 ` Tanstaafl
2013-04-22 0:55 ` Randy Barlow
1 sibling, 0 replies; 46+ messages in thread
From: Tanstaafl @ 2013-04-21 17:31 UTC (permalink / raw
To: gentoo-user
On 2013-04-21 12:56 PM, Tanstaafl <tanstaafl@libertytrek.org> wrote:
> Yes I can't take snapshots with ESXi,
Sorry, should have read 'Yes I can take snapshots with ESXi...'
^ permalink raw reply [flat|nested] 46+ messages in thread
* Re: [gentoo-user] LVM on VM or not? - WAS Re: Best filesystem for virtualized gentoo mail server - WAS: vmWare HowTo / best practices
2013-04-21 16:56 ` Tanstaafl
2013-04-21 17:31 ` Tanstaafl
@ 2013-04-22 0:55 ` Randy Barlow
1 sibling, 0 replies; 46+ messages in thread
From: Randy Barlow @ 2013-04-22 0:55 UTC (permalink / raw
To: gentoo-user
On Sun, 2013-04-21 at 12:56 -0400, Tanstaafl wrote:
> Yes I can't take snapshots with ESXi, but everything I've read says
> that
> for these to be consistent, they need to be done when the VM is
> shutdown.
>
> Also, they take a LONG time, whereas an LVM snapshot happens almost
> immediately.
This reveals a significant advantage to using LVM, if you wish to use
snapshots as the target for backups. I personally prefer this, as I
mentioned in another e-mail in this thread, due to getting a "crash
consistent" view of the disk since it will not change while the backup
is in progress. The fact that LVM snapshots happen instantly is an
advantage, and also that you can perform the snapshots while the system
is running.
--
Randy Barlow
^ permalink raw reply [flat|nested] 46+ messages in thread
* Re: [gentoo-user] LVM on VM or not? - WAS Re: Best filesystem for virtualized gentoo mail server - WAS: vmWare HowTo / best practices
2013-04-21 16:15 ` [gentoo-user] LVM on VM or not? - WAS " Tanstaafl
2013-04-21 16:27 ` Jarry
@ 2013-04-21 16:32 ` Randy Barlow
2013-04-21 16:38 ` Randy Barlow
2013-04-21 17:42 ` Michael Hampicke
1 sibling, 2 replies; 46+ messages in thread
From: Randy Barlow @ 2013-04-21 16:32 UTC (permalink / raw
To: gentoo-user
On Sun, 2013-04-21 at 12:15 -0400, Tanstaafl wrote:
> But I've found lots of opinions that using LVM in a virtualized
> environment can lead to data corruption, and if this is true, I'd rather
> not risk it...
>
> So, LVM or not?
This is surprising to me, because at my former employer we used LVM for
all of our virtual machines in Xen. It performed quite well, and we
never experienced any data loss due to that setup. LVM gives a lot of
flexibility in managing virtual machines, so I'd highly recommend it.
I believe LVM devices can be passed to KVM guests as well, though I have
never personally tried that.
--
Randy Barlow
^ permalink raw reply [flat|nested] 46+ messages in thread
* Re: [gentoo-user] LVM on VM or not? - WAS Re: Best filesystem for virtualized gentoo mail server - WAS: vmWare HowTo / best practices
2013-04-21 16:32 ` Randy Barlow
@ 2013-04-21 16:38 ` Randy Barlow
2013-04-21 19:04 ` Tanstaafl
2013-04-21 17:42 ` Michael Hampicke
1 sibling, 1 reply; 46+ messages in thread
From: Randy Barlow @ 2013-04-21 16:38 UTC (permalink / raw
To: gentoo-user
On Sun, 2013-04-21 at 12:32 -0400, Randy Barlow wrote:
> LVM gives a lot of
> flexibility in managing virtual machines, so I'd highly recommend it.
I should mention one specific advantage to using LVM over file-based
images: I believe you will find that LVM performs better. This is due to
avoiding the duplicated filesystem overhead that would occur in the
file-based image approach. If the guest wants to fsync(), for example,
both filesystems need to be involved (the guest's, and the host's). With
LVM, you still have the host processing the LVM bits of that process,
but at least the host's filesystem doesn't need to be involved.
Of course, giving the guest it's own raw block device (a disk, or a
partition) would also have this advantage, so here I'm mostly just
comparing LVM to the "easy" file-backed disk image approach.
--
Randy Barlow
^ permalink raw reply [flat|nested] 46+ messages in thread
* Re: [gentoo-user] LVM on VM or not? - WAS Re: Best filesystem for virtualized gentoo mail server - WAS: vmWare HowTo / best practices
2013-04-21 16:38 ` Randy Barlow
@ 2013-04-21 19:04 ` Tanstaafl
2013-04-21 22:15 ` Pandu Poluan
2013-04-22 0:34 ` Randy Barlow
0 siblings, 2 replies; 46+ messages in thread
From: Tanstaafl @ 2013-04-21 19:04 UTC (permalink / raw
To: gentoo-user
On 2013-04-21 12:38 PM, Randy Barlow <randy@electronsweatshop.com> wrote:
> I should mention one specific advantage to using LVM over file-based
> images: I believe you will find that LVM performs better. This is due to
> avoiding the duplicated filesystem overhead that would occur in the
> file-based image approach. If the guest wants to fsync(), for example,
> both filesystems need to be involved (the guest's, and the host's). With
> LVM, you still have the host processing the LVM bits of that process,
???
This doesn't make sense to me.
Unless you're talking about using LVM on the HOST.
I'm not. I didn't specify this in this particular post, but I'm using
vmWare ESXi, and installing a gentoo VM to run on it.
So, I'm asking about using the LVM2 installation manual from Gentoo and
using LVM2 for just my gentoo VM...
So, in this case, is it still recommended/fully supported/safe?
Thanks
^ permalink raw reply [flat|nested] 46+ messages in thread
* Re: [gentoo-user] LVM on VM or not? - WAS Re: Best filesystem for virtualized gentoo mail server - WAS: vmWare HowTo / best practices
2013-04-21 19:04 ` Tanstaafl
@ 2013-04-21 22:15 ` Pandu Poluan
2013-04-23 14:19 ` Tanstaafl
2013-04-22 0:34 ` Randy Barlow
1 sibling, 1 reply; 46+ messages in thread
From: Pandu Poluan @ 2013-04-21 22:15 UTC (permalink / raw
To: gentoo-user
[-- Attachment #1: Type: text/plain, Size: 1435 bytes --]
On Apr 22, 2013 2:05 AM, "Tanstaafl" <tanstaafl@libertytrek.org> wrote:
>
> On 2013-04-21 12:38 PM, Randy Barlow <randy@electronsweatshop.com> wrote:
>>
>> I should mention one specific advantage to using LVM over file-based
>> images: I believe you will find that LVM performs better. This is due to
>> avoiding the duplicated filesystem overhead that would occur in the
>> file-based image approach. If the guest wants to fsync(), for example,
>> both filesystems need to be involved (the guest's, and the host's). With
>> LVM, you still have the host processing the LVM bits of that process,
>
>
> ???
>
> This doesn't make sense to me.
>
> Unless you're talking about using LVM on the HOST.
>
> I'm not. I didn't specify this in this particular post, but I'm using
vmWare ESXi, and installing a gentoo VM to run on it.
>
> So, I'm asking about using the LVM2 installation manual from Gentoo and
using LVM2 for just my gentoo VM...
>
> So, in this case, is it still recommended/fully supported/safe?
>
> Thanks
>
Honestly, I don't see how LVM can interact with VMware's VMDK... unless one
use VMware's thin provisioning over a SAN Storage Thin Provisioning, in
which case all hell will break loose once the actual disk size is reached...
Stick with VMware Thin Provisioning XOR SAN Storage Thin Provisioning.
Never both.
One thing you have to think about, is whether to implement
LVM/partition-less, or LVM/partitions.
Rgds,
--
[-- Attachment #2: Type: text/html, Size: 1839 bytes --]
^ permalink raw reply [flat|nested] 46+ messages in thread
* Re: [gentoo-user] LVM on VM or not? - WAS Re: Best filesystem for virtualized gentoo mail server - WAS: vmWare HowTo / best practices
2013-04-21 22:15 ` Pandu Poluan
@ 2013-04-23 14:19 ` Tanstaafl
0 siblings, 0 replies; 46+ messages in thread
From: Tanstaafl @ 2013-04-23 14:19 UTC (permalink / raw
To: gentoo-user
On 2013-04-21 6:15 PM, Pandu Poluan <pandu@poluan.info> wrote:
> One thing you have to think about, is whether to implement
> LVM/partition-less, or LVM/partitions.
Well, I was going to just allocate a new 'drive' (create one within
vmWare and then attach it to my gentoo VM), and use that entire virtual
'disk' for LVM... is that what you mean by 'partition-less'?
Hadn't really thought about the differences much... what are the
advantages or disadvantages to using partitions? Seems like it would be
more flexible without partitions?
^ permalink raw reply [flat|nested] 46+ messages in thread
* Re: [gentoo-user] LVM on VM or not? - WAS Re: Best filesystem for virtualized gentoo mail server - WAS: vmWare HowTo / best practices
2013-04-21 19:04 ` Tanstaafl
2013-04-21 22:15 ` Pandu Poluan
@ 2013-04-22 0:34 ` Randy Barlow
1 sibling, 0 replies; 46+ messages in thread
From: Randy Barlow @ 2013-04-22 0:34 UTC (permalink / raw
To: gentoo-user
On Sun, 2013-04-21 at 15:04 -0400, Tanstaafl wrote:
> ???
>
> This doesn't make sense to me.
>
> Unless you're talking about using LVM on the HOST.
Ah, apologies, I think I had misunderstood. Given that you are using
ESXi, I should have thought that LVM on the host wouldn't be possible.
Are you set on using ESXi? If not, I think there are some compelling
advantages to some of the open source solutions. If so, you are correct
that my suggestion about performance advantages wouldn't apply to you.
> I'm not. I didn't specify this in this particular post, but I'm using
> vmWare ESXi, and installing a gentoo VM to run on it.
>
> So, I'm asking about using the LVM2 installation manual from Gentoo
> and
> using LVM2 for just my gentoo VM...
>
> So, in this case, is it still recommended/fully supported/safe?
In this case, I think you may lose some of the LVM advantages. Assuming
that a volume resize in ESXi is pretty easy, you won't need the guest to
use LVM to take advantage of it.
Here's one though, and I'm not sure the answer: can you easily mount an
ESXi snapshot somewhere, and make a real backup from it? I use LVM to
snapshot my FS, mount that snapshot somewhere, and then I have my backup
software back that up instead of the live system. This has the advantage
of allowing the backup software to have a consistent view of the disk as
it does its work, and gives you sort of a "crash consistent" backup. If
you can still do that with ESXi, that's about the only other advantage.
As to whether its supported, I'm not an expert of ESXi (so don't count
me as an expert), but as far as I know, there should be nothing stopping
you from using LVM in the guests. LVM just works with block devices, and
the ESXi disks should work like block devices, so everything should be
fine. Again, I've never used VMWare very extensively, so this isn't
coming from experience.
--
Randy Barlow
^ permalink raw reply [flat|nested] 46+ messages in thread
* Re: [gentoo-user] LVM on VM or not? - WAS Re: Best filesystem for virtualized gentoo mail server - WAS: vmWare HowTo / best practices
2013-04-21 16:32 ` Randy Barlow
2013-04-21 16:38 ` Randy Barlow
@ 2013-04-21 17:42 ` Michael Hampicke
1 sibling, 0 replies; 46+ messages in thread
From: Michael Hampicke @ 2013-04-21 17:42 UTC (permalink / raw
To: gentoo-user
Am 21.04.2013 18:32, schrieb Randy Barlow:
> On Sun, 2013-04-21 at 12:15 -0400, Tanstaafl wrote:
>> But I've found lots of opinions that using LVM in a virtualized
>> environment can lead to data corruption, and if this is true, I'd
>> rather
>> not risk it...
>>
>> So, LVM or not?
>
> This is surprising to me, because at my former employer we used LVM
> for
> all of our virtual machines in Xen. It performed quite well, and we
> never experienced any data loss due to that setup. LVM gives a lot of
> flexibility in managing virtual machines, so I'd highly recommend it.
>
> I believe LVM devices can be passed to KVM guests as well, though I
> have
> never personally tried that.
Correct, in almost every case where I set up a VM Host, I use LVM to
manage the "disks" that the guests will get.
^ permalink raw reply [flat|nested] 46+ messages in thread
* [gentoo-user] Partition layout questions - WAS Re: Best filesystem for virtualized gentoo mail server - WAS: vmWare HowTo / best practices
2013-04-20 15:00 ` [gentoo-user] Best filesystem for virtualized gentoo mail server - WAS: " Tanstaafl
` (3 preceding siblings ...)
2013-04-21 16:15 ` [gentoo-user] LVM on VM or not? - WAS " Tanstaafl
@ 2013-04-21 16:53 ` Tanstaafl
4 siblings, 0 replies; 46+ messages in thread
From: Tanstaafl @ 2013-04-21 16:53 UTC (permalink / raw
To: gentoo-user
Next, as to partition layout.
I was considering this partition layout:
/boot (ext2), 100M
/swap, 2048G
/ (ext4), 40G
/tmp (ext2), 2G
/var (xfs), 600G
But doing some reading, I stumbled on some other suggestions, like:
Bind /tmp to tmpfs, ie:
tmpfs /tmp tmpfs nodev,nosuid 0 0
Then I read another suggestion to bind /var/tmp to /tmp:
/tmp /var/tmp none rw,noexec,nosuid,nodev,bind 0 0
Which means that both /tmp and /var/tmp are now bound to /tmp?
But, I also read one one of these pages that tmpfs should NOT be used
for /var/tmp, because it stores files that need to be persistent across
reboots - is this still true?
My main concerns are security (which dirs should be separate so they can
be mounted as securely as possible, is, nodev noexec and nosuid mount
options)?
1. Should I go ahead and make separate smallish (maybe 1 or 2GB) /home
so I can mount it nodev,noexec,nosuid?
2. Should I make a separate partition for /var/tmp so I can mount it as
nodev,noexec,nosuid, and bind /tmp to /tmpfs as above? Or does the
caveat about /var/tmp storing files that need to be persistent across
reboots no longer apply, and I can bind them both to tmpfs?
3. Dumb question (google didn't give me an answer) - can I mount all of
/var noexec and nosuid? Assuming not...
4. Since I'm running dovecot with a single user (vmail), and dovecot
stores sieve scripts in the users 'home' dir, does this mean I can't
mount that directory with nodev noexec and/or nosuid?
5. Webapps... can I mount the dir where these are installed with
nodev,noexec,nosuid (I still use webapp-config to manage my website
installations, and currently these are in /var/www)?
I'm thinking an alternative would be to put all data that can be stored
on a partition that is mounted nodev,noexec,nosuid, ie:
/virtual
which would contain:
/virtual/home
/virtual/mail
/virtual/www
Maybe I'm overthinking/overcomplicating this, but obviously now is the
time to make these decisions...
So, comments/criticisms welcome as always...
^ permalink raw reply [flat|nested] 46+ messages in thread