VMworld 2010 – In-N-Out Burger Meetup

Posted on August 3, 2010 by Jon Owings

Last year we had a great time going to In-n-out. For someone like me that was born and raised in Southern California, In-n-out is one of those things I must have when coming back to California. Luckily there is a location within a short trolley ride of the Moscone Center / VMworld 2010. If there is a lesson from last year if you are not used to public transportation you may need some practice.

Leave a comment or hit me up on twitter @2vcps if you are think you will make it.

Meetup
Where: at the In-n-out in San Francsico
When: Tuesday 8/31 at 5:30 pm – Gives you enough to go eat and come back before any Vendor sponsored party/events. I won’t interfere with free food and beer.
What: Eat Double Double, Fries

UPDATE

@trey_anderson was cool enough to make this map with directions from Moscone to In-N-Out

All out of HA Slots

Posted on July 1, 2010 by Jon Owings

A few weeks a go I was moving a customer from an old set of ESX servers (not HA clustered) to a new infrastructure of Clustered ESX hosts. After building, testing and verifying the hosts we started moving the VM’s. It became apparent after a little while there were some resource issues. After just a few VM’s were moved an alert appeared that we could not start any new machines. I start looking at the cluster and there is plenty of extra Memory and CPU. Still nothing will start.
I say to myself, “Self, we have read about this before.” I thought back to this HA Deep Dive article by Duncan Epping.
Lets check the HA slots! (on a side note, if you use HA and have never read the Deep Dive, go do it now!)

As you can see here the slot size is rather giant. We have the largest CPU and Memory reservation plus some overhead (for simplicity) and that blows the size of the slot way up. I didn’t set the reservation, but surely they were there. 8GB of reserved memory. 4000MHz of CPU. Ouch. Where did that come from? It followed the VM from the old host to the new one. One of the reasons I was there was to setup a new cluster since the older ones were performing so slow on the local storage. It seems like someone tried to help some critical VM’s along the way by adding the reservations. I removed the reservations and had plenty of slots as you see below.

Yeah! I was able to power on another VM!

The new cluster blew away the old one. Went from older Xeon’s to 6 core Nehalem’s, from local disks to 48 disks of Equallogic Storage. The reservation was no longer needed.

Lessons:
1. Be careful with reservations, it can impact your failover capacity.
2. Reservations set on the machine will follow it to a new host.

The Mini ESXi 4 Portable Server

Posted on June 16, 2010 by Jon Owings

Special thanks to Kevin Miller (@kevin_miller), for making sure I didn’t burn up anything and running out to Fry’s to get a new CPU when the orginal we ordered turned out to be not compatible.

List of the material used in final Version:

Intel DH57JG – Motherboard
Intel i5-661 Core Duo – Processor (special note: DO NOT get the i5-655k unlocked CPU in the picture, it was not compatible with any intel Motherboard)
Lian Li MINI ITX Case (6x9x12 in dimensions) supports Full size PSU
Rosewill silent PSU
Intel dual port PCIe 16x Gbe NIC
Stock intel heatsink and fan (Nexus fan in the picture was very cool, just too big to fit anything into the PCIe port.
2x Intel 80GB SSD
2x4GB OCZ Memory Kit

The server built amazingly fast using PXE boot and installing an ESXi image. Only thing that didn’t work that I wish it did was the on board NIC, 2 GigE ports will be more than enough for our purpose. This is not a lab machine but will run all the components I need to deploy a solution in an automated fashion. I really like VMware Fusion but if I try to run Windows 7, a Linux Server and a Windows 2003 server at the same time my awesome Mac Book Pro becomes useless. This is a lightweight and easy way to bring all of these components together for relatively low cost and if it can save half a day on an install it will pay for itself pretty fast.
I am running:
Windows 2003 ( a vCenter Template)
UDA20 – Ultimate Deployment Appliance that I fought with for a while and finally have it running reliably.
vSphere Mangement Assistant

As of this post working on building:
Windows 7 VM (for Powershell and other tools)
Maybe a free NFS/CIFS server for some easy file shares.
Linux Server (just in case I need it)

Now for some pics:

Before:

SSDs Mounted

Two 80 GB SSD’s are mounted in this space.

Everything going into the Case

Size relative to my hand

I do not have NBA player sized hands, hopefully this shows just how small the case is. This solution is a little more pricey for just a lab machine but if it travels I do not need a SATA disk failing because the server had a bumpy ride.

Update Manager and Isolated ESX Service Console Networks

Posted on June 1, 2010 by Jon Owings

Sometimes you may be required to run your vCenter server that has two network interfaces. One in the network it can be reached for remote desktop access and the other where it has access to the ESX servers in order to manage the VMware hosts. This is sort of a hybrid model of an isolated management network. Where only one host can reach the management ports. One thing to think about in this model is Update Manager by default will not like it. Everything may look ok, but trying to scan a host will fail. Luckily though it is an easy fix.

In the update manager configuration tab change the ip in the picture to the IP accessible by the ESX servers. Then remember to restart the Update Manager services. Now go back and run the ESX scan/stage/remediation.

B.Y.O.P – The Alternative Vblock

Posted on May 28, 2010 by Jon Owings

In college I often would be invited to a get together that could often include the letters BYOB, Bring Your Own Beer. Sometimes a cookout would be BYOM, Bring Your Own Meat (or meat alternative for the vegetarians). So today I want to leverage this to push my new acronym B.Y.O.P. Bring Your Own Pod. Lately I have been seeing people talk about Vblocks. If I can venture a succinct definition a Vblock is a pre-configured set of Cisco, EMC and VMware products tested by super smart people, approved by these people to work together, then supported by these organizations as a single entity. Your reseller/solutions provider really should already be doing this very thing for you. You may choose to buy just the network piece, or the hypervisor but your partner should be able to verify a solution to work from end to end and provide unified support.

So You can’t call it BYOPCVCEP

Why not Vblock? This might get me blacklisted by the Elders of the vDiva council, but VCE doesn’t exist to make your life in the datacenter easier, they exist to sell you more VMware, Cisco and EMC. Vblock for sure simplifies your buying experience. I believe they are all great products and may very well do just what you need. Without competition though the only winner is VCE. Do not by forced into a box by the giant vendors. Find someone that can help determine your end goal, provide you vendor neutral analysis of the building blocks needed to achieve your end goal. Then provide the correct vendors and unified support to Build Your Own Pod.

So What is the Alternative Vblock

Originally I was going to draw up a sweet solution of 3par, Xsigo and Dell R610′s and say, “Hey everyone! This is some cool stuff. Try to quiet the overwhelmingly loud voice calling from VCE and give this Alternative Vblock a try.” As I thought more and more about it I think doing that is contrary to my main point. I would like more to provide the discussion points or some possible products among others that can be used to Build Your Own Pod. I am a firm believer in getting what is right for your datacenter needs. So here is a few links to help begin the discussion.

Xsigo and Pod – Jon Toor
3par and iBlocks – Marc Farley

You might be a vDiva if…

Posted on May 25, 2010 by Jon Owings

I am avoiding a post where I have to think really hard about a topic. That makes me procrastinate and come up with even crazier ideas. I am writing this one down now. Most of these apply to me so if you are offended by any of them you are probably a vDiva.

You might be a vDiva if…

… you roll your eyes when someone talks about installing a PHYSICAL server.

… you are on twitter to see how many people you can get to look at your blog, but you never stoop so low to interact with the common folk.

… you are surprised when the guy at the table at the VMUG doesn’t know who you are.

… you constantly check your Google Analytics account to see how many views you have. (guilty)

… you refer to yourself as @… (your twitter account)

… you hunt down @jtroyer if you latest post takes too long to get on the v12n board.

… your require a signed rider agreement with your speaking topic for VMworld, saying you need 800 green M&M’s, a copy of Lord of the Rings in your hotel room, and direct phone access to Steve Herrod’s iPhone.

I probably ticked a bunch of people off. I am just having fun. Have a great day! Go ahead and add your own in the comments.

Operational Readiness

Posted on May 3, 2010 by Jon Owings

One thing I am thinking about due to the VCDX application is operational readiness. What does it mean to pronounce this project or solution good-to-go? In my world it would be to test that each feature does exactly what it should be doing. Most commonly this will be failover testing, but could reach into any feature or be as big as DR plan that involves much more than the technical parts doing what they should. Some things I think need to be checked:

Resources

Are the CPU, Memory, Network and Storage doing what they should be? Some load generating programs like IOmeter can be fine to test network and storage performance. CPU busy programs can verify Resource Pools and DRS are behaving the way they should.

Failover

You have redundant links right? Start pulling cables. Do the links failover for Virtual Machines, Service Console, and iSCSI? How about the redundancy of the physical network, even more cable to pull! Also test that the storage controllers failover correctly. Also, I will make sure HA does what it is supposed to, instantly power off a host and make sure some test virtual machines start up somewhere else on the cluster.

Virtual Center Operations

Deploy new virtual machines, host and storage VMotion, deploy from a template, and clone a vm are all things we need to make sure are working. If this is a big enough deployment make sure the customer can use the deployment appliance if you are making use of one. Make sure the alarms send traps and emails too.

Storage Operations

Create new luns, test replication, test storage alarms and make sure the customer understands thin provisioning if it is in use. Make sure you are getting IO as designed from the Storage side. Making use of the SAN tools to be sure the storage is doing what it should.

Applications

You can verify that each application is working as intended within the virtual environment.

There must be something I am missing but the point is trying to test out everything so you can tell that this virtualization solution is ready to be used.

Using Network Load Balancing with View

Posted on April 26, 2010 by Jon Owings

If you have a smaller View deployment but still want to have redundant connection servers look no further than Microsoft NLB. Solve this problem without the need for an expensive hardware loadbalancer. Will it have all of the bells and whistles? No. If you have less than a 1000 users you probably would not see the benefit of the advanced features in a hardware load balancer. Make sure to read the whitepaper from VMware about NLB in Virtual Machines.

I am making the assumption you are like me and want everything to be as virtual as possible. So the View Connection Manager servers will be VM’s

Setup the primary and replica View Servers

I won’t go over installing View. Just be sure to setup the initial manager server. Then go ahead and setup the replica VM.

Configure NLB

Go the the Administrative tools and open the Network Load Balancing Manager. Right click the top node in the tree menu on the left and select New Cluster.
Set the IP and other information you will used for the Load Balanced cluster. This is a new IP not used by your View Manager servers.
In the VMware document referenced above VMware recommends setting the Cluster operation mode to Multicast.
Click Next then next again. When asked to configure port rules I leave it on the default and click next. You can chose limit this to certain ports.

Click Next again and enter localhost in the wizard to configure the local interfaces for NLB. Click next and make sure to note the priority. When setting up the replica server this number needs to be different. Finally click finish and wait for the configuration to finish. You should now be able to ping your new cluster IP address.

Setup the Replica Server in the Load Balancer

Righ Click the node in the tree menu for the NLB Cluster you just created and select Add new host to cluster. Enter the IP for the Replica Server and click connect. Select the interface that will be used for the Load Balancing and click next. Make sure the Priority is unique from the first server. If it gives you any grief after this point close and re-open the Network Load balancing Manager. The working cluster should look like this:

Test the Failover

Start a continual ping to the cluster IP. Now use the vSphere Client to disconnect the network from one of the servers. Watch the pings continue to come back.

Finally, create a DNS A record (something like desktop.yourdomain.com) and point it to the cluster IP. You now have some decent failover in case of a VM failure and even a host failure (suggestion would be to use seperate hosts for the VM’s).

Note – You may need to add static ARP entries into your switching depending on your network topology. Be sure to test this fully and consult your network manufacturer’s documention for help with static ARP.

Adaptive Queuing in ESX

Posted on April 25, 2010 by Jon Owings

While troubleshooting another issue a week or two ago I came across this VMware knowledge base article. Having spent most of the time with other brand arrays in the past, I thought this was a pretty cool solution verses just increasing the queue length of the HBA. I would recommend setting this on your 3par BEFORE you get QFULL problems. Additionally, Netapp has an implementation of this as well.

Be sure to read the note at the bottom especially:

If hosts running operating systems other than ESX are connected to array ports that are being accessed by ESX hosts, while the latter are configured to use the adaptive algorithm, make sure those operating systems use an adaptive queue depth algorithm as well or isolate them on different ports on the storage array.

I do need to dig deeper how this affects performance as the queue begins to fill, not sure if one method is better than another. Is this the new direction that many Storage Vendors will follow?

Until then, the best advice is to do what your storage vendor recommends, especially if they say it is critical.

Here is a quick run through for you.

In the vSphere Client

Select the ESX host and go to the configuration tab and click on the Advanced Settings under Software.

In the Advanced Settings

Select the option for Disk and scroll down to the QFullSampleSize and QFullThreshold.
Change the values to the 3par recommended values:
QFullSampleSize = 32
QFullThreshold = 4

2vcps and a Truck

Tag Archives: vmware

VMworld 2010 – In-N-Out Burger Meetup

All out of HA Slots

The Mini ESXi 4 Portable Server

Before:

SSDs Mounted

Everything going into the Case

Size relative to my hand

Update Manager and Isolated ESX Service Console Networks

B.Y.O.P – The Alternative Vblock

So You can’t call it BYOPCVCEP

So What is the Alternative Vblock

You might be a vDiva if…

Operational Readiness

Using Network Load Balancing with View

Setup the primary and replica View Servers

Configure NLB

Setup the Replica Server in the Load Balancer

Test the Failover

Adaptive Queuing in ESX

In the vSphere Client

In the Advanced Settings