[gentoo-user] VMWare Hypervisor - SD vs CF card?

[gentoo-user] VMWare Hypervisor - SD vs CF card?

[Tanstaafl]

Hi everyone,

Ok, here's my dilemma...

I have some new Dell R515 servers (12 bay versions). These do not have 
any Dell supported internal SD (or CF) card options for the hypervisor, 
but they do have an internal 2.5" dual SATA/SAS drive cage, for running 
a bootable OS (in my case the ESXi hypervisor)...

Well, I really hate the idea of wasting money and disk space (for 2 
146GB SAS drives to be run in a mirror, what is being recommended to me) 
on something that only requires about 32MB to install (the hypervisor) 
when apparently there is a really cool option like:

for CF cards:
http://www.addonics.com/products/ad2sahdcf.php

or

for SD cards:
http://www.addonics.com/products/ad2sahdcf.php

I could get 2 of these for each server, each with a mirrored pair of CF 
(or SD) cards (mirror mode is defined by a jumper or switch on the 
adapter), then mirror those (in the BIOS), which would result in a total 
of FOUR CF (or SD) redundant cards (a mirror of 2 mirrored pairs) for 
the hypervisor... and I can do this for quite a bit less than even a 
SINGLE 146GB SAS drive...

Is there any reason NOT to do this?

What am I missing (other than the fact that Dell won't support this 
config, but I'm not using them for software support anyway)?

Appreciate any/all comments...

Charles

Re: [gentoo-user] VMWare Hypervisor - SD vs CF card?

[Michael Mol]

On Tue, Jun 19, 2012 at 7:47 AM, Tanstaafl <tanstaafl@libertytrek.org> wrote:
> Hi everyone,
>
> Ok, here's my dilemma...
>
> I have some new Dell R515 servers (12 bay versions). These do not have any
> Dell supported internal SD (or CF) card options for the hypervisor, but they
> do have an internal 2.5" dual SATA/SAS drive cage, for running a bootable OS
> (in my case the ESXi hypervisor)...
>
> Well, I really hate the idea of wasting money and disk space (for 2 146GB
> SAS drives to be run in a mirror, what is being recommended to me) on
> something that only requires about 32MB to install (the hypervisor) when
> apparently there is a really cool option like:
>
> for CF cards:
> http://www.addonics.com/products/ad2sahdcf.php
>
> or
>
> for SD cards:
> http://www.addonics.com/products/ad2sahdcf.php
>
> I could get 2 of these for each server, each with a mirrored pair of CF (or
> SD) cards (mirror mode is defined by a jumper or switch on the adapter),
> then mirror those (in the BIOS), which would result in a total of FOUR CF
> (or SD) redundant cards (a mirror of 2 mirrored pairs) for the hypervisor...
> and I can do this for quite a bit less than even a SINGLE 146GB SAS drive...
>
> Is there any reason NOT to do this?
>
> What am I missing (other than the fact that Dell won't support this config,
> but I'm not using them for software support anyway)?
>
> Appreciate any/all comments...

I'd say give it a shot on something non-critical, see if it works. The
big thing I'd be uncertain of is bootup time for the Addonics
adapter...how long before it tells the SATA controller it's ready?

Incidentally, I think that adapter is brilliant. I might snag one in a
few months and set up the Gentoo live DVD as a read-only boot install.

-- 
:wq

Re: [gentoo-user] VMWare Hypervisor - SD vs CF card?

[Alan McKinnon]

On Tue, 19 Jun 2012 07:47:10 -0400
Tanstaafl <tanstaafl@libertytrek.org> wrote:

> Hi everyone,
> 
> Ok, here's my dilemma...
> 
> I have some new Dell R515 servers (12 bay versions). These do not
> have any Dell supported internal SD (or CF) card options for the
> hypervisor, but they do have an internal 2.5" dual SATA/SAS drive
> cage, for running a bootable OS (in my case the ESXi hypervisor)...
> 
> Well, I really hate the idea of wasting money and disk space (for 2 
> 146GB SAS drives to be run in a mirror, what is being recommended to
> me) on something that only requires about 32MB to install (the
> hypervisor) when apparently there is a really cool option like:
> 
> for CF cards:
> http://www.addonics.com/products/ad2sahdcf.php
> 
> or
> 
> for SD cards:
> http://www.addonics.com/products/ad2sahdcf.php
> 
> I could get 2 of these for each server, each with a mirrored pair of
> CF (or SD) cards (mirror mode is defined by a jumper or switch on the 
> adapter), then mirror those (in the BIOS), which would result in a
> total of FOUR CF (or SD) redundant cards (a mirror of 2 mirrored
> pairs) for the hypervisor... and I can do this for quite a bit less
> than even a SINGLE 146GB SAS drive...
> 
> Is there any reason NOT to do this?
> 
> What am I missing (other than the fact that Dell won't support this 
> config, but I'm not using them for software support anyway)?
> 
> Appreciate any/all comments...
> 
> Charles
> 
> 

With my last batch of ESX hosts, someone "forgot" to order the R710 SD
internal add-on. But it has 8 x 600M SAS drives....

So what I did is configured all drives as a RAID 10 and let ESX grab
enough for the hypervisor  and leave the rest for regular storage.

I figured that ESX is an appliance anyway, there's nothing on it I
can't get back by running the installer again, I have backups of the
isos and templates. And if a drive pokes and takes out a guest, I still
equally screwed regardless of whether I gave some space away to the
hypervisor or not.




-- 
Alan McKinnnon
alan.mckinnon@gmail.com

Re: [gentoo-user] VMWare Hypervisor - SD vs CF card?

[Tanstaafl]

On 2012-06-19 9:56 AM, Alan McKinnon <alan.mckinnon@gmail.com> wrote:
> With my last batch of ESX hosts, someone "forgot" to order the R710 SD
> internal add-on. But it has 8 x 600M SAS drives....
>
> So what I did is configured all drives as a RAID 10 and let ESX grab
> enough for the hypervisor  and leave the rest for regular storage.

That was my initial plan, but I really like the idea of having the 
hypervisor separated... and I also like the idea of running it on FLASH 
media...

I'll be ordering the Addonics today, and will report back if everything 
works as expected/hoped...

Anyone have any recommendation for a high quality/high-speed (but 
smallest capacity) CF media to put in these?

Re: [gentoo-user] VMWare Hypervisor - SD vs CF card?

[Michael Mol]

On Tue, Jun 19, 2012 at 10:21 AM, Tanstaafl <tanstaafl@libertytrek.org> wrote:
> On 2012-06-19 9:56 AM, Alan McKinnon <alan.mckinnon@gmail.com> wrote:
>>
>> With my last batch of ESX hosts, someone "forgot" to order the R710 SD
>> internal add-on. But it has 8 x 600M SAS drives....
>>
>> So what I did is configured all drives as a RAID 10 and let ESX grab
>> enough for the hypervisor  and leave the rest for regular storage.
>
>
> That was my initial plan, but I really like the idea of having the
> hypervisor separated... and I also like the idea of running it on FLASH
> media...
>
> I'll be ordering the Addonics today, and will report back if everything
> works as expected/hoped...
>
> Anyone have any recommendation for a high quality/high-speed (but smallest
> capacity) CF media to put in these?

CF is really behind the times. I'd probably suggest going with one of
the SD adapters, and picking up whatever card professional
photographers swear by.

Based on recent experience, I'd also suggest keeping it small. 2GB or
less...which is more than what you need.

-- 
:wq

Re: [gentoo-user] VMWare Hypervisor - SD vs CF card?

[Tanstaafl]

On 2012-06-19 10:28 AM, Michael Mol <mikemol@gmail.com> wrote:
> CF is really behind the times.

Really? Nothing I've read indicates that - can you point me to something 
that discusses how/why Cf is 'behind the times'?

I'm serious, I just ordered the CF adapter/cards, but I'm fully prepared 
to send them back if you can show me something authoritative that backs 
up that claim...

Thanks,

Charles

Re: [gentoo-user] VMWare Hypervisor - SD vs CF card?

[Matthew Marlowe]

On Tue, Jun 19, 2012 at 4:47 AM, Tanstaafl <tanstaafl@libertytrek.org> wrote:
> I could get 2 of these for each server, each with a mirrored pair of CF (or
> SD) cards (mirror mode is defined by a jumper or switch on the adapter),
> then mirror those (in the BIOS), which would result in a total of FOUR CF
> (or SD) redundant cards (a mirror of 2 mirrored pairs) for the hypervisor...
> and I can do this for quite a bit less than even a SINGLE 146GB SAS drive...
>
> Is there any reason NOT to do this?
>

If you have a small ESX cluster, there are numerous advantages to
having some local storage on each your ESX hosts in addition to your
primary SAN storage:
- Testing major ESX version upgrades prior to rolling out to cluster
(converting VM's to new hardware format, while leaving old VM's on
SAN)
- If your setup is too small to have high performance spare SAN
devices + storage, what do you do when you have to do a major upgrade
of the SAN and/or possibly perform data destructive RAID format
changes? iSCSI storage vmotion would allow you to migrate VM's to
local storage on ESX servers while SAN is upgraded...several extra
hard drives + raid controllers are cheaper than buying another
equalogic/emc device.
- Some cluster backup software like to replicate backup data outside
of the SAN and backup server.....I felt much better when I was
performing nightly backups from the SAN to local storage on the ESX
boxes and then exporting the dedup'd backup data to backup server for
writing to tape.  But, there are many ways to resolve this.
- Local ESX storage is much cheaper than SAN...there were several
cases where I used to run production VM's via SAN, and temporary
dev/test VM's on ESX server local storage
- Lastly, I never really have been a fan of ESXi as an upgrade from
ESX.....seems that it was more driven by vmware making windows admins
feel more confident since they didn't have to learn linux for ESX
console.

But, there is nothing keeping you from getting mirrored CF/SD cards
for the hypervisor boot and also keeping a few mirrored 2TB SATA
drives on each host for local datastores (7200rpm SATA is much cheaper
than 15K rpm SAS).

Of course, for large ESX clusters, you can probably afford numerous
SAN devices which would negate most of the above.

Matt

Re: [gentoo-user] VMWare Hypervisor - SD vs CF card?

[Tanstaafl]

On 2012-06-22 12:04 AM, Matthew Marlowe <matt@professionalsysadmin.com> 
wrote:
> On Tue, Jun 19, 2012 at 4:47 AM, Tanstaafl<tanstaafl@libertytrek.org>  wrote:
>> I could get 2 of these for each server, each with a mirrored pair of CF (or
>> SD) cards (mirror mode is defined by a jumper or switch on the adapter),
>> then mirror those (in the BIOS), which would result in a total of FOUR CF
>> (or SD) redundant cards (a mirror of 2 mirrored pairs) for the hypervisor...
>> and I can do this for quite a bit less than even a SINGLE 146GB SAS drive...
>>
>> Is there any reason NOT to do this?

> If you have a small ESX cluster, there are numerous advantages to
> having some local storage on each your ESX hosts in addition to your
> primary SAN storage:

We actually will be using ONLY local storage... Dell R515's with 8 450GB 
SAS drives in RAID10 (with one hot spare assigned)...

A decent SAN wasn't in the budget (yet, but we may go  that route in a 
year or two)...

> - Lastly, I never really have been a fan of ESXi as an upgrade from
> ESX.....seems that it was more driven by vmware making windows admins
> feel more confident since they didn't have to learn linux for ESX
> console.

This is a new install, so not an 'upgrade'...

> But, there is nothing keeping you from getting mirrored CF/SD cards
> for the hypervisor boot and also keeping a few mirrored 2TB SATA
> drives on each host for local datastores (7200rpm SATA is much cheaper
> than 15K rpm SAS).

I do plan on having a couple of large SATA drives in RAID0 (for speed) 
for temporary snapshots (which I then backup using rsnapshot or my VM 
backup s/w) and for if I ever need to add some drives to my RAID10 
(probably won't, the 1.7TB I'll have is 4 times what we have now which 
is only 70% utilized)...

I get the CF cards today (already have the adapters), so we'll see how 
this goes this weekend...

Re: [gentoo-user] VMWare Hypervisor - SD vs CF card?

[Tanstaafl]

On 2012-06-22 12:04 AM, Matthew Marlowe <matt@professionalsysadmin.com> 
wrote:
> But, there is nothing keeping you from getting mirrored CF/SD cards
> for the hypervisor boot

Also, my questions was more just to which cards are considered best/most 
stable - SD or CF...

Re: [gentoo-user] VMWare Hypervisor - SD vs CF card?

[Paul Hartman]

On Fri, Jun 22, 2012 at 7:50 AM, Tanstaafl <tanstaafl@libertytrek.org> wrote:
> On 2012-06-22 12:04 AM, Matthew Marlowe <matt@professionalsysadmin.com>
> wrote:
>>
>> But, there is nothing keeping you from getting mirrored CF/SD cards
>> for the hypervisor boot
>
>
> Also, my questions was more just to which cards are considered best/most
> stable - SD or CF...

Ultimately they both probably have the same flash chips inside of them
so if your main concern is reliability, I don't think it matters.

If your concern is performance, CF seems to be used in more
"professional" applications and more high-speed CF cards are readily
available.

In either case I would suggest avoiding the cheap no-name brands.
Sandisk Extreme Pro is likely the fastest card you can buy (of either
CF or SD form factor), it is available up to 100MB/sec write speeds,
but of course your card reader/host needs to support speeds like that.
Sandisk also routinely has more than 10x the random I/O performance of
most of the other brands which is important when using it on a
computer and not in a linear recording device (photos/video).

Re: [gentoo-user] VMWare Hypervisor - SD vs CF card?

[Tanstaafl]

On 2012-06-22 11:00 AM, Paul Hartman <paul.hartman+gentoo@gmail.com> wrote:
> On Fri, Jun 22, 2012 at 7:50 AM, Tanstaafl<tanstaafl@libertytrek.org>  wrote:
>> Also, my questions was more just to which cards are considered best/most
>> stable - SD or CF...

> Ultimately they both probably have the same flash chips inside of them
> so if your main concern is reliability, I don't think it matters.
>
> If your concern is performance, CF seems to be used in more
> "professional" applications and more high-speed CF cards are readily
> available.
>
> In either case I would suggest avoiding the cheap no-name brands.
> Sandisk Extreme Pro is likely the fastest card you can buy (of either
> CF or SD form factor), it is available up to 100MB/sec write speeds,
> but of course your card reader/host needs to support speeds like that.
> Sandisk also routinely has more than 10x the random I/O performance of
> most of the other brands which is important when using it on a
> computer and not in a linear recording device (photos/video).

Thanks Paul, that's all pretty much what I'd concluded as well from my 
research...

I went with the 4GB SanDisk Ultra though (30MB/s), since these will only 
be used to boot the VMWare hypervisor (which runs fully in RAM once it 
is booted)...

Now I'm looking forward to seeing them in action this weekend... :)

Re: [gentoo-user] VMWare Hypervisor - SD vs CF card?

[Michael Mol]

On Fri, Jun 22, 2012 at 12:08 PM, Tanstaafl <tanstaafl@libertytrek.org> wrote:
> On 2012-06-22 11:00 AM, Paul Hartman <paul.hartman+gentoo@gmail.com> wrote:
>>
>> On Fri, Jun 22, 2012 at 7:50 AM, Tanstaafl<tanstaafl@libertytrek.org>
>>  wrote:
>>
>>> Also, my questions was more just to which cards are considered best/most
>>> stable - SD or CF...
>
>
>> Ultimately they both probably have the same flash chips inside of them
>> so if your main concern is reliability, I don't think it matters.
>>
>> If your concern is performance, CF seems to be used in more
>> "professional" applications and more high-speed CF cards are readily
>> available.
>>
>> In either case I would suggest avoiding the cheap no-name brands.
>> Sandisk Extreme Pro is likely the fastest card you can buy (of either
>> CF or SD form factor), it is available up to 100MB/sec write speeds,
>> but of course your card reader/host needs to support speeds like that.
>> Sandisk also routinely has more than 10x the random I/O performance of
>> most of the other brands which is important when using it on a
>> computer and not in a linear recording device (photos/video).
>
>
> Thanks Paul, that's all pretty much what I'd concluded as well from my
> research...
>
> I went with the 4GB SanDisk Ultra though (30MB/s), since these will only be
> used to boot the VMWare hypervisor (which runs fully in RAM once it is
> booted)...
>
> Now I'm looking forward to seeing them in action this weekend... :)

OK, I missed that piece. I presumed there would be writes to the hard disk.

Any reason you can't have these guys netboot?


-- 
:wq

Re: [gentoo-user] VMWare Hypervisor - SD vs CF card?

[Tanstaafl]

On 2012-06-22 12:26 PM, Michael Mol <mikemol@gmail.com> wrote:
> OK, I missed that piece. I presumed there would be writes to the hard disk.
>
> Any reason you can't have these guys netboot?

Only that I've never done that before with servers, and my only 
experience with netbooting at all was with LTSP about 10 years ago.

I think having 4 CF cards (mirrored pair of mirrored pairs) will be 
enough redundancy though... ;)

Oh... one other question...

These CF adapters only have 2 screw holes (made to go into laptops, not 
mounted in a cage), so I can't mount them *properly* in the cage... 
anyone know where I can get 2.5" 'shell' cases that I could install 
these cards in so I can mount them properly? Right now I have to shove a 
piece of anti-static material in between them and the cage (and each 
other) to prevent them from accidentally touching (yuck!)...

Re: [gentoo-user] VMWare Hypervisor - SD vs CF card?

[Tanstaafl]

On 2012-06-23 7:11 AM, Tanstaafl <tanstaafl@libertytrek.org> wrote:
> On 2012-06-22 12:26 PM, Michael Mol <mikemol@gmail.com> wrote:
>> OK, I missed that piece. I presumed there would be writes to the hard
>> disk.
>>
>> Any reason you can't have these guys netboot?
>
> Only that I've never done that before with servers, and my only
> experience with netbooting at all was with LTSP about 10 years ago.
>
> I think having 4 CF cards (mirrored pair of mirrored pairs) will be
> enough redundancy though... ;)

Well, these seem to work swimmingly well... now I just need to find some 
kind of non-flammable/heat resistant insulating material that I can use 
to keep these cards from touching themselves or the metal cage (see 
below)...

> Oh... one other question...
>
> These CF adapters only have 2 screw holes (made to go into laptops, not
> mounted in a cage), so I can't mount them *properly* in the cage...
> anyone know where I can get 2.5" 'shell' cases that I could install
> these cards in so I can mount them properly? Right now I have to shove a
> piece of anti-static material in between them and the cage (and each
> other) to prevent them from accidentally touching (yuck!)...
>