So I often have epiphany teasers while driving long distances or stuck in traffic. I call them teasers because they are never fully developed ideas and often disappear into thoughts about passing cars, or yelling at the person on their cell phone going 15 MPH taking up 2 lanes.
Here is some I was able to save today (VMware related):
1. What if I DID want an HA cluster to be split in two different locations, Why?
2. Why must we over-subscribe iSCSI vmkernel ports to make the best use of the 1gbe phyical nics. Is it a just the software iSCSI in vSphere? Is just something that happens with IP storage? I should test that sometime…
3. If I had 10 GB nics I wouldn’t use them on Service Console or Vmotion that would be a waste. No wait, VMotion ports could use it to speed up your VMotions.
4. Why do people use VLAN 1 for their production servers? Didnt’ their Momma teach em?
5. People shouldn’t fear using extents, they are not that bad. No, maybe they are. Nah, I bet they are fine, how often does just 1 lun go down. What are the chances of it being the first lun in your extent? Ok maybe it happens a bunch. I am too scared to try it today.
Thank you to everyone out there that gave any attention to this little blog. I tried to supply new information where I could and also document my journey trying to improve at the esxcfg- commands. In 2009 the traffic increased from less than 100 visits in a week to approaching 500 in a week. That is a slow hour for some of the best vm bloggers (Yellow-Bricks, Scott Lowe) but I am not trying to replace them.
In March I was layed off. I spent 2 months full time looking for work. The VMware community really stepped up. Special thanks go out to a few people that tried to help out or checked in with how the search was going: @jasemccarty @maishk @jasonboche and many others. By the end of April I started a new position as a Virtualization Infrastructure Engineer at Veristor Systems. This job required moving the family from Memphis, Tennessee to Atlanta, Georgia. All the fun of a new job, house hunting and moving to a completely new city. We closed on our new house just a few days after VMworld 2009. My first trip to VMworld was amazing but I would suggest not buying a house while you are in and out of conferences, labs and seminars. Lots of details to fix, I will be happy if this is not the case ever again.
Getting into the VMworld stuff.
More VMworld Stuff
After such a busy year it was only the beginning of September. The rest of the year was filled with VMware Installation projects and VMware health checks. I was all over the Southeast US. Maybe I can come visit some of you in 2010?
Technical/VM/Blogging Goals for 2010:
- Continually improve the quality of content going to this blog.
- Schedule, take and pass the VCDX exams.
- Improve my overall storage knowledge.
- Renew my expired CCNA (bad move letting this lapse in 2009)
2009 was such a big year personally. I hope and pray for a little more stability in 2010 but I also love the excitement of new challenges, but the VCDX should be enough. 
Our Backyard in December
Also, I scheduled a Merry Christmas post, but for some reason I didn’t do what I thought I did. So a late Merry Christmas to all of you out there.
Recently I have spent time re-thinking certain configuration scenarios and asking myself, “Why?” If there is something I do day to day during installs is this still true when it comes to vSphere? or will it still be true when it comes to future versions.
Lately I have questioned how I deploy LUNs/volumes/datastores. I usually deploy multiple moderate size datastores. In my opinion this was always the best way to fit in MOST situations. I also will create datastores based on need afterward. So will create some general use datastores then add a bigger or smaller store based on performance/storage needs. After all the research I have done and asking questions on twitter* I still think this is a good plan in most situations.
I went over a VMworld.com session TA3220 – VMware vStorage VMFS-3 Architectural Advances since ESX 3.0 and read this paper:
http://www.vmware.com/resources/techresources/1059
I also went over some blog posts at Yellow-Bricks.com and Virtualgeek.
An idea occurred to me when it comes to using extents in VMFS, SCSI Reservations/Locks, and VDI “Boot Storms”. First some things a picked up.
1. Extents are not “spill and fill” VMFS places VM files across all the LUNs. Not quite what I would call load balancing, since it does not take IO load into account when placing files. So in situations where all the VM’s have similar loads this won’t be a problem.
2. Only the first LUN in a VMFS span gets locked by “storage and VMFS Administrative tasks” (Scalable Storage Performance pg 9). Not sure if this implies all locks.
Booting 100’s of VM’s for VMware View will cause locking and even though vSphere is much better when it comes to how quickly this process takes. There is still an impact. So I am beginning to think of a disk layout to ease administration for VDI, and possibly lay the groundwork for improved performance. Here is my theory:
Create four LUNs with 200GB each. Use VMFS to extents to group them together. Resulting in an 800 GB datastore with 4 disk queues and only 1 LUN that locks during administrative tasks.
Give this datastore to VMware View and let it have at it. Since the IO load for each VM is mostly the same, and really at the highest during boot other tasks performed on the LUN after the initial boot storm will have even less impact. So we can let desktops get destroyed and rebuilt/cloned all day with only locking that first LUN. This part I still need to confirm in the LAB.
What I have seen in the lab is with same sized clones the data on disk was spread pretty evenly across the LUNs.
Any other ideas? Please leave a comment. Maybe I am way off base.
*(thanks to @lamw @jasonboche and @sakacc for discussing or answering my tweets)
I was looking for last couple weeks for a good way to re-purpose PC’s as thin clients to ease the investment in VDI. I stumbled across this PDF from VMware and I thought it was great. I would tend towards using group policy to deploy the new shell described on pages 3 and 4. It can always be undone if the PC is needed as a PC again.
Check it out.
You pretty much replace the default shell (explorer.exe) with the VMware View Client. I would suggest using some group policy to keep people from using the task manager to start new processes. This should be a temporary solution until you have budget to buy some real thin clients or net books even.
There are of course lots of options out there for thin clients, and software to provision a “thin OS” to machines. This is free and easy though. I thought it was cool so I decided to share.
I have been writing on ESX command line stuff for far too long now. I just need to finish it before ESX 5 comes out with no command line support and my year long series is rendered obsolete before it is finished.
Here is a few more commands from ESX 3.5u4. They are pretty short and sweet. So enjoy.
esxcfg-pciid
Used to read the xml files under /etc/vmware/pciid/ – from what I can tell it is used in the process of getting unsupported hardware to work.
esxcfg-rescan
Supply this command with a vm hba and it will rescan for new LUNs.
esxcfg-resgrp
Do a esxcfg-resgrp -l and get a ton of information try following it with a “| more”: esxcfg-resgrp -l | more
It does include options to add remove and restore resource groups. Here is a place on the interwebs with information on how to use it.
esxcfg-swiscsi
Used to enable and disable the Software iSCSI Initiator. esxcfg-swiscsi -q will give you the list of the status.
Make sure to create a vmkernel port first.
esxcfg-upgrade
According to the ESX Configuration Guide this command is not for general use but is used to upgrade VMFS2 to VMFS3. Would be very smart to have moved all VM’s off of the VMFS2 before messing with this command.
esxcfg-vmhbadevs
Use the -m tag to list all vmfs volumes. Typical output will look like this:
vmhba0:0:0:3 /dev/sda3 4afc26df-ddbe3ee5-2037-000c29acbc0a
esxcfg-nas
Standard use of this command is to add or list your NFS mounts.
List: esxcfg-nas –l
Add: esxcfg-nas –a –o <host> -s <share> <name>
Not much more I can say. A little more detail here:
http://b2v.co.uk/b2vguide2vmware3.htm
A thread in the communities about a problem someone had where the nfsclient wasn’t loaded:
http://communities.vmware.com/message/864559
So go out and add some NFS datastores!
It has been almost 1 year since I started looking at the esxcfg-* commands. It initially came as a look at the first part of the Enterprise Administration Exam’s Blueprint very first bullet point. In that post I talked about using esxcfg-mpath to identify which luns are fiber, iSCSI, NFS or local.
Today lets look a little bit deeper at the command and how it can be used to help you day to day.
We can always use esxcfg-mpath -l to list all of the luns and their paths. A good thing to check here is that you have the same number of paths to each datastore that comes from the SAN. You may have a zoning issue if a certain lun can only be seen from 1 path rather than all of them. In general each hba will see the lun through each controller in an active active type fiber channel SAN. So hba A should see the lun from Controller A and B. Likewise, hba B should see Controller A and B for a total of 4 paths. If you are using fixed or MRU as the pathing policy only one will be active but esxcfg-mpath -l will show four paths.
Of course if you have more hbas and controllers you will have more paths.
You can follow the examples given by the esxcfg-mpath -h to get help. One useful tool is to create a crude script using esxcfg-mpath –policy with the –lun tag to change the policy from say MRU to Fixed. I am not a perl scripter and I sure someone already has a real shell or perl script to set the policy but I do like to prepare multiple command in notepad then paste them into the cli.
esxcfg-mpath –policy=fixed –lun=vmhba0:0:1
esxcfg-mpath –policy=fixed –lun=vmhba0:0:2
and so on…
Then try to feel really smart by alternating the paths so every fourth lun will use different paths.
esxcfg-mpath –path=vmhba1:0:1 –lun=vmhba0:0:1 –state=on
esxcfg-mpath –path=vmhba2:0:1 –lun=vmhba0:0:2 –state=on
esxcfg-mpath –preferred –path=vmhba1:0:1 –lun=vmhba0:0:1
esxcfg-mpath –preferred –path=vmhba2:0:1 –lun=vmhba0:0:2
The first two lines set the path for the respective luns to different hbas. The last two lines set the preferred path to that same port on the hba. So when there is failover the path will fail back to your set config when all is well.
So I was updating some of my blog posts on the esxcfg-* commands with any changes in ESX 4. I wrote earlier I did not know much about the esxcfg-advcfg command. Since writing that post at the end of 2008, I found Duncan Epping used esxcfg-advcfg in 3.5 to set the option rescan all the Hba’s. I thought this was a great shortcut and decided to try it out in vSphere but:
[root@esx4 ~]# esxcfg-advcfg -s 1 /Scsi/ScsiRescanAllHbas
Exception occured: Unable to find option ScsiRescanAllHbas
So I looked through vCenter 4 and did not find the option under Scsi I looked around some in the other Advanced Options and it is no where to be found.
Has this been removed or moved somewhere else? If you know hit me up on twitter @2vcps
After the last two entries being somewhat boring and not useful at least in my opinion. I am glad this command has a little more to it.
[root@esx1 root]# esxcfg-module -h
Usage: esxcfg-module
-g–get-options Get the options for a given module and whether it is loaded on boot.
-s–set-options Set the options for a given module. WARNING this may be overwritten by per device options.
-e–enable Enable a given module, indicating it should be loaded on boot.
-d–disable Disable a given module, indicating it should not be loaded on boot.
-q–query Query enabled modules options.
-l–list List all modules and whether they are enabled.
-h–help Show this message.
[root@esx1 root]# esxcfg-module -l
Device Driver Modules
Module Enabled Loaded
vmklinux true true
cciss true true
tg3 true true
qla4022 true true
e1000 true true
qla2300_707_vmwtrue truel
vmdriver true true
vmfs3 true true
etherswitch true true
shaper true true
tcpip true true
cosShadow true true
migration true true
nfsclient true true
deltadisk true true
vmfs2 true true
iscsi_mod true true
[root@esx1 root]#
So a common use I have seen with this command is to turn off unused modules. Go ahead and free up some resources turn off vmfs2! You don’t use it.
Another use is to change HBA options.
/usr/sbin/esxcfg-module -s ql2xmaxqdepth=64 qla2300_707_vmw
Remember to follow it with:
/usr/sbin/esxcfg-boot -b
Check out VCDX Master Ninja/Jedi Duncan Epping using this command combo in a scripted install.
Why haven’t you looked into all of these free VMworld Sessions?
So greatly collected all in one place at the best one stop shop for all VMware related material on the web Eric Siebert’s vSphere-land.
This will be great distraction from me renewing my CCNA and getting MCITP in Exchange 2007. My best excuse for not studying is there is so many interesting topics out there!
Recent Comments