I am no network super-genius but I do enough with networking to be able to get by. Two common mistakes I find many times are flat networks and firewalls as the default gateway. A flat network is when generally switches are connected to one another without any configuration. There is one broadcast domain which means every packet that the switch does not have an entry in the MAC address table, is sent out all the ports but the originating port. This repeats across all of the switches until the layer 2 destination is found. Now, this means your expensive Cisco switches are barely better than hubs. You don’t have collisions like you would on a hub and once the switch learns where the MAC address lives it keeps that information for a certain amount of time. Then again in this network setup the logs are most likely not monitored so if there where collisions and other errors it goes unnoticed.
That is not the title of this post though. Although related to a flat network using the firewall is a different issue. Using the firewall as the router works just fine when you have a flat network. You may never notice the problem in a small network, but as your network grew you noticed how problems can come up when there is just one big network. So someone smart said use vlans to segment the network, create smaller broadcast domains. Then when you try to fix or change the flat network with subnets and vlans can you find out the new vlans can not reach the rest of the original network.

The current flat network with switches and the firewall used as the default gateway or router.

The problem comes when you add subnets that are different than the interface ip of the firewall. Firewalls in general have issue with redirecting traffic bound for other networks back out of the same interface. So in the picture above traffic from vlan 1 that is using the firewall as the default gateway trying to reach the subnet on vlan 10. Since the host on vlan 1 does not know where that network lives it sends the traffic to the default gateway. Even if you added a static route to the firewall the traffic will often fail. That is because firewalls are not meant to route but rather send traffic between trusted and untrusted networks and vice-versa. So the question becomes how do you actually fix your flat network that has the firewall as the router. There is of course more complicated solutions to provide high availability using VRRP or HSRP.
First get a real layer 3 device. That is a router or a switch capable of routing between multiple vlans. The good news is many of your newer switches are capable of layer 3, it is included in many Dell and HP switches, it may still be an add-on with Cisco. I haven’t used a new switch in the last year that did not have layer 3.
Next important step is use the layer 3 device (switch or router) to route everything. Set a default route in the layer 3 device to send only outbound traffic to the firewall and bam everything works. Why is this so hard. Many times there is hundreds of servers and desktops already configured to use the firewall as their router. We will do a lot of work to avoid having to do a bunch of manual work.

Now you are using a router to route and the firewall to block bad things and maybe even do NAT. (note: If you are doing NAT be sure to add your new VLANs to your NAT rules so the new networks can reach the outside of your firewall.)

Related posts:

1. ESX Commands: esxcfg-vswitch
2. ESX Commands – esxcfg-firewall
3. Secure to the Hosted VM

This happened a long time ago. I arrived at a customer site to install View Desktop Manager (may have been version 2). This was before any cool VDI sizing tools like Liquidware Labs. I am installing ESX and VDM I casually ask, “What apps will you be running on this install?” The answer was, “Oh, web apps like youtube, flash and some shockwave stuff.” I thought “ah dang” in my best Mater voice. This was a case of two different organizations thinking someone else had gathered the proper information. Important details sometimes fall through the cracks. Since that day, I try to at least uncover most of this stuff before I show up on site.

Even though we have great assessment tools now, remember to ask some questions and get to know what is your customers end goal.

Things I learned that day. As related to VDI.

1. Know what your client is doing, “What apps are you going to use?”

2. Know where your client wants to do that thing from, “So, what kind of connection do you have to that remote office with 100+ users?”

This is not the full list of questions I would ask, just some I learned along the way.

Related posts:

1. ESX Commands – esxcfg-firewall
2. VMware View – Repurpose your Existing PC’s as Thin Clients
3. View Open Client 4.5 Beta 1.1 Now Out

*Disclaimer – I work for a Xsigo and VMware partner.

I was in the VMware View Design and Best practices class a couple weeks ago. Much of the class is built on the VMware View Reference Architecture. The picture below is from that PDF.

It really struck me how many IO connections (Network or Storage) it would take to run this POD. Minimum (in my opinion) would be 6 cables per host with ten 8 host clusters that is 480 cables! Let’s say that 160 of those are 4 gb Fiberchannel and the other 320 are 1 gb ethernet. The is 640 gb for storage and 320 for network.

Xsigo currently uses 20 gb infiniband and best practice would be to use 2 cards per server. The same 80 servers in the above cluster would have 3200 gb of bandwidth available. Add in the flexibility and ease of management you get using virtual IO. The cost savings in the number director class fiber switches and datacenter switches you no longer need and the ROI I would think the pays for the Xsigo Directors. I don’t deal with pricing so this is pure contemplation. So I will stick with the technical benefits. Being in the datacenter I like any solution that makes provisioning servers easier, takes less cabling, and gives me unbelievable bandwidth.

So just in the way VMware changed the way we think about the datacenter. Virtual IO will once again change how we deal with our deployments.

Related posts:

1. Using Network Load Balancing with View
2. Operational Readiness
3. Tale of Two Datacenters

The esxcfg-vswif command allows you to create and modify Service Console ports and their IP information. Many times I have to change stuff after the install process is complete and the only place is via the direct service console because network communication is not possible. This usually happens when the network team changes a vlan in the middle of the install or they change a subnet. Not to disparage network teams many times I am the network team and the virtualization team.
Create a new vswif:
#first add a port group with esxcfg-vswitch
esxcfg-vswitch -A "Service Console Test" vSwitch-Test
#then use esxcfg-vswif to create a new vswif
esxcfg-vswif -a -i 172.16.50.40 -n 255.255.255.0 -p "Service Console Test" vswif1
#List your vswifs
esxcfg-vswif - l
#Example:
[root@esx3 root]# esxcfg-vswif -l
Name Port Group IP Address Netmask Broadcast Enabled DHCP
vswif0 Service Console 172.16.50.50 255.255.255.0 172.16.50.255 true false
vswif1 Service Console Test172.16.50.40 255.255.255.0 172.16.50.255 true false

Modify your Service Console network information:
esxcfg-vswif -i 172.16.50.41 -n 255.255.255.0 vswif1
#example
[root@esx3 root]# esxcfg-vswif -i 172.16.50.41 -n 255.255.255.0 vswif1
Setting IP config
Nothing to flush.
[root@esx3 root]# esxcfg-vswif -l
Name Port Group IP Address Netmask Broadcast Enabled DHCP
vswif0 Service Console 172.16.50.50 255.255.255.0 172.16.50.255 true false
vswif1 Service Console Test172.16.50.41 255.255.255.0 172.16.50.255 true false


Related posts:

1. ESX Commands – esxcfg-module
2. ESX Commands – esxcfg-firewall
3. ESX Commands – esxcfg-linuxnet

I have really forgot to keep up on my VCDX study path. So today a quick tidbit on the esxcfg-firewall command.
Many of us today will use the vCenter Client to change firewall ports on the ESX. One instance where I exclusively mess with the firewall from the command line using esxcfg-firewall is when I install Dell OpenManage. I am already in the console to install the agents so I might as well open the firewall from the console too.
This really applies to any kind of agent or software you add to your ESX installation. So if you find yourself already in the console why not save a step and do it from the cli?

Lets look at the command

# esxcfg-firewall -o 1311,tcp,in,OpenManageRequest

First is the command, esxcfg-firewall, -o is for openport, the 1311 is the port number, tcp is protocol, in is the direction and the final part is the name of the service.

Now if you want to see all of your esxcfg-firewall settings try:
esxcfg-firewall -q

Show if specifig service is enabled.
esxcfg-firewall -q [service name]

Of course typing esxcfg-firewall -h gives lots of good help.

Some links: (You can google and find a ton more)

ESX Guide
VMware Land
Yellow Bricks
Vritualization Admin

Related posts:

1. ESX Commands: esxcfg-vswif
2. ESX Commands – esxcfg-hwiscsi
3. ESX Commands – esxcfg-linuxnet

When I first started out in College I needed a work study job. Since I liked to help people with their computer problems I applied and was hired for a position doing phone and in person support for the University. One of the best things about starting out at a school they don’t mind teaching. Our trainer said that in previous years new employees would be slotted into being Windows or Mac or UNIX support. He said we would be Wunder-Cons (our title was consultant instead of help desk dude). We had the privilege of having to support all of it. This thrust me into the world IT no matter what the piece of paper from USC said I was a Bachelor of.

I believe a new kind of Wunder-Consultant/Engineer is being made. With the announcement of the Nexus 1000v last fall the line between Network Engineer and Datacenter/Server Engineer is getting blurred. The SAN and Server Engineers have had this tension for a while now. Virtualization is a fun technology to learn but who gets the responsibility? I have seen where the SAN team owns the ESX’s and the Server team operates the VM’s like they are physical. The Network team not trusting or understanding why they want a bunch of 1GigE trunk ports. Across larger organizations it would look different but the struggle may be just the same. Who is in control of the VM’s? Are they secure? Who gets called at 1am when something dies? This is internal to the IT department and does not consider that Sales doesn’t want to share memory with accounting.

I can see these technologies pushing engineers into being jacks of all trades. To be a truly Architect level in VMware today you must be awesome with storage and servers. You have to be able to SSH into an ESX, choose the right storage for an application, and setup templates of Windows 2003. That is an easy day. You already will have to troubleshoot IO (because all problems get blamed on the virtualization first).

With the Nexus 1000v I picture the Virtualization Admins learning the skills to configure and troubleshoot route/switch inside and outside the Virtual Infrastructure. Add to that Cisco’s push this year with 10GigE and FCoE and their own embedded virtualization products. The lines between job duties are getting blown away.

Who is poised to become the experts in this realm? The network, server or storage admins? In this economy it may be good to know how to do all three jobs. I am sure corporations would love to pay just one salary to perform these tasks.

Randomly I though how would this relate to SOX? Could it pose any problems with compliance? I will save that for next time.

Related posts:

1. Review the Year – 2008
2. Storage KB entries from VMware
3. From Professional VMware – Virtual Machine Disk Sizing Tool

GNS3 is a excellent tool that uses dynamips to simulate routers running real Cisco IOS. You must have rights on your CCO account to download the IOS. It also includes the PIX emulator so you can check out your PIX/ASA configs.

Related posts:

1. The Forging of the new Network/VMware/Storage Professional
2. Education and Virtualization – Oh, the Possibilities
3. The Mini ESXi 4 Portable Server