Rescuing A Poor Performing VMware Virtual Infrastructure

Rescuing Slow VM Performance

Does your VMware vSphere Environment require First Aid?

Rescuing A Poor Performing VMware vSphere Virtual Infrastructure

About 4 years ago I was hired by a firm to help rescue their VMware infrastructure. Though they had everything any top performing VMware environment needed for success, including a VMware TAM, it just wasn’t meeting customer requirements for performance.

The job began with me interviewing frustrated developers who demanded I move them off slow VMs back on to physical servers ASAP, “Get me off this VMware crap, now!” It was my job to calm them down and reassure them we were going to fix the slow VM problems.

Helps On The Way!

After about a week the problems were getting clear to me: mixed matched combinations of server hardware, poorly configured storage, oversubscribed network with backup running on the same switches (VLANs) and poor VM configuration and lack of knowledge and expertise.

Basically it was the typical story I’ve seen many times when the free version of ESXi turns deadly.

VMware is really an awesome technology if it’s set up right.

Unfortunately most companies for years have been deploying into an already badly designed infrastructure then expect VMware to somehow improve server performance while at the same time saving money. Adding VMware or any other hyper-visor product will only highlight the issues that already exist: bad storage, network, design and build practices. An unwillingness to change the way things are done in storage and network management will doom your virtual infrastructure real fast.

  1. First start by getting help from someone with expertise in the virtual infrastructure space. Normally this will be a 3rd party VMware Engineer that is not defending their own bad practices.
  2. Do an inventory and technical analysis on everything that VMware leverages: how networks are setup, VLAN configuration, firewalls, backup traffic, storage types, configuration, spindle count, drive sizes, RAID configurations, LUN sizes, Brocades, what shares the same storage and network equipment. Then start on server hardware: model, type, brand, CPUs, memory, blade or not blade, firmware, HBA, NICs. Finally – what VMware ESX/I versions are being used: patch level, VMtools and VM configuration, etc.
  3. Now decide what the long-term expectations are for this VI environment. For example, do you plan to virtualize everything, possible? If so, then it’s time to think about dedicated resources that are only used for virtualization. No more shared networks and storage environments.
  4. Redesign for the future and start migrating to a new tested, validated and standard virtual infrastructure that uses best practices for virtualization. Note: Ignorance is the enemy and allowing system, network or storage administrators to do foolish things will cause poor performance, outages and down-time.

A well performing virtual infrastructure is standardized, and in my experience, is scalable by adding more of the same blocks of capacity (example only: cluster of 4 hosts, using 4 terabytes, with networks the same and VM configurations following a documented and test design). Note: Try to stay away from one-off requests, if possible.

An unwillingness to change the way things are done in storage and network management will doom your virtual infrastructure real fast.

Diagnosing Slow VMware VMs

The worst environments are normally a result of leaders and teams doing dumb things requested by ignorant managers or executives. Here’s my suggestion on how to solve this.

  1. Get your vendor partners to come in and do lunch-and-learns or training sessions to help educate staff on products.
  2. Make sure you ask them  plenty of good thought-out questions about what the risks, or disadvantages are, of doing things that are questionable.

Hopefully your vendors will be honest when answering your questions (some may hold back because they sense what you are doing).

SOW for a Health Check

If worst comes to worst, call VMware and get a SOW worked out to have someone do a health check on your vSphere environment. I never understood this but executes find bad news more believable when it comes from a 3rd party.

Your turn to share tips and comments on how to tune and tweak a poor performing VMware and get rid of slow VMs?


WAIT! Before You Go... Join Other Go-Getters

Enter your name & email address below to get started.

We're listening. Go ahead and share your ideas?

vSphere 6
8 Updates That Make vSphere 6 Better – Keith Barker
25 VMware Interview Questions: Tough & Technical (Download)
VMware jobs in Phoenix
VMware Jobs in Phoenix Arizona (Comparing Opportunities)
DevOps training
10 Reasons VMware Admins Should Boost OpenStack Skills
DevOps Tools
DevOps Toolchain: Download 3 Free Tools Used By Pros (Ansible, Git, Jenkins)
MSDevOps Guide
Microsoft DevOps: 25+ Power-Packed Windows SysAdmin Skills (NEW Guide!)
continuous integration tools
DevOps Training Courses That Will Give You The Edge (Perfect For Beginners Who Want Skills That Are Noticed By Google & Facebook…)
Why DevOps Fails: How To Get Back On Track
Why DevOps Fails: How To Get Back On Track
Click here to learn how to Optimize WordPress Speed
WordPress Speed Optimization: Part 4 – VPS Case Study To Make Rank Higher In Google Through Faster Hosting & High Availability @ Lower Cost
Best Hosting for WordPress
Case Study: Best VPS For Easy WordPress Migration
cloudways review updated
Cloudways Review: 12 Pros & Cons From My Deep Dive Testing (VPS Series: Part 2)
ServerPilot Review: Installing WordPress On DigitalOcean (VPS Series: Part 1)
Storage Engineer Skills
Why Storage Engineer Skills Are HOT! (Can You Say Big Data?)
Rise of the Robots
I’m Amazed After Reading Rise of the Robots – Best Jobs In 2017 (You won’t Guess what’s going on!)
top virtual reality jobs
30+ Awesome Virtual Reality Jobs That Take You From Boring To Amazing! (Must Read…)
Thinking Like Seth Godin
How To STOP A 300% Salary Loss Because YOU Think Like A SysAdmin: Start Thinking Like Seth Godin (Read Now!)
3 Open Source Storage Software Alternatives For Beginners
shades of devops
Free DevOps eBook Project (Read It Now!)
How to install Genymotion
How To Install Genymotion for Beginners (Includes Google Play)
20 Questions VMware Newbies Want Answered (Free Career Guide)