You, Your Health and the Datacenter

Just yesterday I finally completed a marathon few of months in downsizing a live environment in one of our datacenter. This was a huge project with a very ambitious deadline that still required time spent in the office each day doing BAU. To put my workload into perspective, BAU contains customer support tickets that usually role into the next day and so on. In itself, just to keep on top of that is full time, now adding in a large datacenter consolidation with multiple parties involved more than doubles that workload.

Like any project, there are lessons that are learnt and usually incorporated into the next project. Some of the mistakes that we make are ones that are obvious and just plain common sense, but due to our own determination (or should we say, stubbiness!) we tend to make them without realising.

One of the biggest mistakes I made during this project was looking after my health. I pride myself for the fact that I don’t get sick (aside from the minor runny nose or cough) but the reality is, is that we are not invincible. A weeks back, we had a nasty virus go around the office,  I was just coming off a large stint of after hours work and being physically drained and surprise surprise, I got sick. I was so sick that I ended up taking days off which is a big deal for me. I then had a week where I took it steady and paced myself, then came the last couple of weeks and I went in guns blazing feeling on top of the world to meet the deadline. Unfortunately, I pushed myself hard, I did 60+ hours in 4 days, I started Sunday and finished Thursday morning. I would go to the datacenter, get a large amount of work done, then go home for an hours sleep, get up, get my daughter ready for the day and go to work. I would then go home and have dinner, put my daughter to bed and go back out to the datacenter, I started to make simple mistakes, but pushed on. Come Friday, I got sick again and over the next 4 days, I lost 4kgs and I only weighed 70kg to start with.

This week I spent several days in the Datacenter to complete the project, this time I put my BAU on hold so I could pace my days and get a good night sleep to limit the mistakes to almost none.

The project is now completed and the biggest lesson I have taken away from this is to look after yourself and know your limits. We all strive to be the best we can, we want to show our peers that we can do almost anything to get the job done, but the risk we take is dangerous. The percentage of mistakes we will make are greater the more tired we become, ranging from possible customer outages through to  causing physical injury to yourself or others.

So, from my recent experience, I have compiled a list of things that I think are vital to try and keep yourself happy, healthy and on top of your game.

  • Take regular breaks and keep water intake up:
    When working in a datacenter, you are in a dry environment where you are constantly moving between cold and hot aisles. Ensure you keep your fluids up, you don’t want to suddenly collapse from lack of hydration in the middle of the datacenter.
  • Ensure you get plenty of sleep and pace yourself:
    No project or job is ever worth your life. When you are tired you will make mistakes that can either impact the company or may cause an accident where yourself or someone may get injured.
  • Don’t be afraid to ask for help:
    If you find yourself running out of time, or being unable to complete all the tasks or just need a moment to take a breathe, Ask for help. There is no shame in needing assistance. We are all human.
  • Plan to spend time with the family:
    I cannot stress enough that spending time with family was a necessity to stay happy and to stop the mind focusing on the work that was ahead. Clearing the mind is essential for when you are back at the task and needing to focus.

If you can stick to these guidelines, you will not only succeed at your project, but you will be happier and healthier at the end of it. If you have a peer you are working with, take the time to remind them every few hours to take a quick 5 minutes break, it could be the difference between working on another project with them again or not.

Advertisements

PowerCLI migrate vSwitch port groups to vDS in a different vCenter

Over the 6 months I have been working hard on designing and implementing our latest infrastructure refresh and migration to another datacenter. This was a big task, especially when we had to migrate customer servers with minimal downtime. However, there were many more challenges we faced, however with the right planning in the design, these were fairly well handled.

One of the challenges was that we were using Standard vSwitches in the old 5.5/5.1 environment due to some 3rd party applications back when the environment was 4.1 which caused issues when using a vDS.
As we were building a new vCenter we decided the best method was to automate adding all the VM port groups along with their VLANs and LAG into the DvSwitch.
One thing I’ve learnt from Alan Renouf is “The best script you will ever write is one that you stole from somebody’s website” which doesn’t mean  steal it and claim it as your own, but if someone has a script that does exactly what you need, then use it, just make sure you give credit where credit is due.
It just so happened by luck that right around the time I was starting to think about the process, I saw a twitter post from Ben Liebowitz “PowerCLI Script to create a new vDS Portgroups” – Beauty, this was exactly what I was after (albeit some small changes to suit).

The next step was to get a script to match to do the initial export to CSV, after a quick google I came across a Luc Dekens script that he had written for someones request on the VMware Community Forums.  It was pretty straight forward and only require some lines removed so that the CSV only had the columns required for the import.  So once I had the scripts, it was down to testing the process on how to use them prior to prod.

  1. Edit the Export vSwitch Configuration script from Luc Dekens
  2. Run the script
  3. Open the CSV (Make sure the columns names line up with the import script)
    **As we are exporting from a vSwitch and importing to vDS, we will need to manually add a new column to the exported CSV called numports and place the correct number of ports in each row (by default 8) .
    **Also remove any multiples of portgroups (e.g. if you have multiple hosts with the same Portgroups as these will be also in the csv)
  4. Edit the Import script from Ben Liebowitz
    – Change the vDS name and LAG name to match your environment.
    – Update to the CSV path
  5. Run the import script.
  6. Confirm the ports have imported by looking at the vDS.

The process is simple, so let’s break this down into some of the areas you can edit .

In the export script, all you need to edit is the lines that control what information is exported to the CSV. Just remove the lines you do not require. for example I do not need the IP address, so I would remove the below line.

@{N="IP";E={if($vNicTab.ContainsKey($pg.Name)){$vNicTab[$pg.Name].Spec.Ip.IpAddress}}}

The csv will export to the directory you have set in PowerCLI when running the script. Below is what the csv will turn out like, however note that I have also added the numports in as well.

ESX,pgName,vlanID,numports
HyperVisor-Hostname,PortGroup_1,3005,8
HyperVisor-Hostname,Portgroup_2,3005,8
HyperVisor-Hostname,Portgroup_13,3007,8
HyperVisor-Hostname,Portgroup_34,3007,8

etc.

Now for the Import.
In the import script, make sure that you change the name of the vDS, ActiveUplink and the location of the CSV – So he following lines.

# Set the VDS Name to variable
$vds = "dvSwitch"
# Import the CSV of VLAN IDs, Portgroups, and # of ports
$vdsPortgroup = Import-Csv \path\to\New_Portgroups.csv
get-vdswitch $vdsname | Get-VDPortgroup $portgroup.pgName | Get-VDUplinkTeamingPolicy | Set-VDUplinkTeamingPolicy -UnusedUplinkPort dvUplink1, dvUplink2, dvUplink3, dvUplink4

get-vdswitch $vdsname | Get-VDPortgroup $portgroup.pgName | Get-VDUplinkTeamingPolicy | Set-VDUplinkTeamingPolicy -ActiveUplinkPort LAG

That’s it. Very straight forward set of scripts to run. I prefer to run these individually as there is the step in the middle with the csv file. Aside from that I would like to thank both Ben Liebowitz and Luc Dekens for their community support for sharing their scripts.

Install and Configure NSX Manager

I’m starting to become a bit of a fan of VMware NSX and getting excited with all the new features that came out of VMworld 2017. I recently rebuilt my lab and one of the parts I need to install is, you guessed it, NSX. So I figured I would write a series of basic “Getting Started” guides.  To start with, we will go through installing the NSX Manager, the brain of the solution. Now for the install, I just attached the NSX Manager Appliance to a vSwitch for the moment, but you will need to ensure that you have a Distributed Virtual Switch configured to utilise NSX as it is a required component. I will be installing NSX Manager 6.3.3 (The latest as of this post)

NSX is more than just networking, it is also part of the endpoint services that was previously vShield in the vCloud Networking and Security Suite (vCNS). Implementing NSX allows you to extend the feature set taking you to the next level of virtual networking.

I will be going over the install and configuration of some of the other components in the next few posts following this one.

Installing NSX Manager:

The NSX Manager OVA can be downloaded from my.vmware.com
Once downloaded, ensure the Client Integration tools are installed and then open up the VMware Webclient. (For Client Integration Tools SSL FireFox issue  see this post)

Right click cluster and select “Deploy OVF Template” à Navigate to the NSX Manager OVF file à Accept the configuration

Accept the EULA

Input details for configuration

-Password
– IPv4/IPv6 details
– DNS
– NTP
– SSH
– VMware Customer Experience Improvement Program

 

 

 

 

 

 

 

Accept all and deploy

 

 

 

Once deployed. Open up a web browser and navigate to the IP/hostname that you set for your NSX Manager.

Login with admin and the password you set for deployment.

Navigate to “General” and edit the time settings à set your timezone à Save and log off and back on again for settings to take affect

Navigate down to “NSX Management Service” àSelect Edit on “Lookup Service URL:” and enter your PSC FQDN. (Enter vCenter if using embedded PSC) Enter SSO username and password and click OK

Select edit on “vCenter Server” and enter your vCenter server address followed by vCenter Service account or SSO.

Once all lights are green, log off and log into vCenter with the account used to attached NSX to vCenter.

If you log in as another accout, you will not be able to see the Network and Security tab as you will not have been granted permission to it. (Note that my SSO is vSphere.local for this lab)

Log in as the account that You will see the Networking and Security section available in the Action menu, Home screen and the left hand Navigation menu. Select Network and Security -> click on NSX Managers -> select NSX Manager you wish to adjust -> Manage -> Users.

Click the green Plus sign -> Choose either to add a group or individual user (Suggest making an NSX Group to make control easier) -> Select the Level of Access and click OK.

Log off and log in as the user you jut granted permission to.  (Note that I am using readysetvirtual.local for my lab domain and standard user)

VMware Client Integration – FireFox SSL Popup

I’ve been having this issue for a little while now, I hadn’t found any successful posts on how to allow the VMware Client integration plug-in to run on FireFox. This occur in my last lab environment and my current.  Unfortunately, without this integration tool, OVF deployments and various other functions are unavailable in the WebClient.

Error Msg: “The VMware Client Integration Plugin has updated its SSL Certificate in FireFox.”

Since Firefox ver. 52, plugins have been disabled by default and started to behave differently. We found this out the hardware when some of our customers were starting to open their SaaS Citrix environment with HTML5 instead of the thick client.

To fix this, I found a KB article that outlines the solution. (KBA 2112076)

As you can see, the integration tool is currently installed and in Firefox 56 the message is displaying after log on to the WebClient.

Screen Shot 2017-09-06 at 10.28.31 amScreen Shot 2017-09-06 at 10.27.43 am

Go ahead and uninstall the Integration Tool as you will need to reinstall it again.

Screen Shot 2017-09-06 at 10.31.15 am

 

Uninstall your current version of FireFox and download and install FireFox Extended Support Release
Screen Shot 2017-09-06 at 10.30.17 am

Once installed, reinstall the VMware Client Integration Plug-in and launch FireFox. The Plugin should then popup once you access your venter WebClient login page. -> Select “Remember my choice for vmware-csd links” and click “Open Link”

Screen Shot 2017-09-06 at 10.40.55 am

Head over to a host and attempt an OVF deployment.  A second pop will request for Access Control -> select “Allow” and untick “Always ask before allowing this site” (Unless security reasons)

Screen Shot 2017-09-06 at 10.42.08 am

You should now not see any error messages when you attempt an OVF deployment.

Screen Shot 2017-09-06 at 10.42.21 am

Free NSX books at VMworld and Digital Download

If you are VMworld, go pick yourself up the most recent series of NSX books.

If you are NOT at VMworld, you can also download the books from the below links.

VMware NSX Micro-segmentation Day 1  – Wade Homes

VMware NSX Micro-segmentation Day 2 – Geoff Wilmington

Operationalizing VMware NSX – Kevin Lees

Automating NSX for vSphere with PowerNSX – Anthony Burke

 

More information here. 

So long vCenter on Windows and Flash WebClient

Two big announcements were made over the weekend just before VMworld. They were both inevitable, but it was just a question of “When?”.  So that day has been set, not by a date, but by a timeline.

The announcements by VMware are that vCenter for Windows and the vSphere flash WebClient have now been dropped in the next version of vSphere.

Let’s start off with vCenter for Windows. If you couldn’t see this coming, then you may have been living under a rock for the last few vSphere releases. With the vCenter Server Appliance (VCSA) being first released as part of vSphere 5.5 and then being improved significantly by vSphere 6 and then on par and exceeding capabilities in vSphere 6.5, there was no way that VMware would over time continue working on vCenter on Windows. If you follow VMware’s trends on new product releases then you will note this is similar to the deprecation of the vSphere C# client that has not been updated since 5.5 as VMware pushed the WebClient. At the time, the WebClient did not boast all the features of the C# client and thus VMware were recommending to still use the C# client for Update Manger. This was then brought into the WebClient by  vcenter 6.0 but update manager still required to be installed on a Windows server. Fast forward to the release of vSphere 6.5 and Update Manager has finally been implemented into VCSA. This was the final nail in the Windows vCenter coffin.

What are the benefits of using VCSA instead of windows? The biggest benefit is the licensing costs, eliminating the requirement for Windows OS to be installed. VCSA 5.5/6.0 previously ran on SLES11 and from 6.5 VCSA now runs on PhotonOS, a somewhat new OS platform designed and created by VMware, this drives down the costs as well as bring together tasks like updating and upgrades into a simple 1 step process (2 if you run external PSC). VMware is pushing PhotonOS out to their product suites with the latest having been the NSX Controllers in 6.3.3 (which according to VMware versioning is a minor update)

The final piece that has got my full support for VCSA is the deployment process, as a Mac user, to install the VCSA in 5.5 or 6.0 required a windows machine to run (or at least I was never able to find a way for it to from a Mac) – therefore this meant I need to stand up a windows VM first to run the installer from. Since the release of 6.5, VMware has simplified the process and has included a Mac and Linux deployment file. Aside from building a DC, there is no longer any requirements for a windows server in my environment.

So long Windows vCenter, you were good to us

Learn More Here 

 

The second announcement which I thought was an earlier than expected surprise was the vSphere flash WebClient being dropped from the next release bringing the new(ish) HTML5 client into the light as the one and only client for managing vSphere.

In vSphere 6.5, the HTML5 client is still only partially functional for vCenter however according to the article, VMware is aiming for 100% complete by the next version release. I suspect there will be further functionality added as new updates are released for vCenter.  Personally think this is a little early as the HTML5 client hasn’t had a full functioning release alongside the flash client as it was when the webclient took over from the C# client. However, VMware are on a somewhat short timeline to remove the flash content as Adobe Flash ceases in 2020. I’ve been using the flash vSphere client religiously at home (as mentioned I’m using a Mac) and I have finally accepted it, however at work I still manage to use the C# client on my 5.5 environment.

Looking at the time between vSphere releases which is generally around 18months, this leaves VMware approx. 10 months to have HTML5 up to their 100% full functionality. It will be interesting to see.

Find out more here

Guest Introspection needs to be fully uninstalled first – Error message

Being half way through a large infrastructure migration, there have been some interesting issues turn up along the way which have then required updating to the design and deployment documentation. One of the steps that I need to perform is upgrade VM hardware compatibility and VMTools. This all seemed an easy enough task to complete, especially with Update Manager, that was until it came to doing the actual VMTools upgrade. Update Manager continued to fail on a large number of VMs. When the installation was manually ran, an error message “VMware Guest Introspection must be fully uninstalled before the installation can proceed.” would

What was interesting about this was that I had migrated away from a system that did not have NSX or Guest Introspection per se. The old environment was running traditional vShield and had moved to the new environment which was now running NSX Manager and Guest Introspection only.

After going through a couple of knowledge base articles around the subject, I went ahead and tried the removal of regkeys in VMware, this did not fix the problem; in fact it was a very simple fix.

In control panel –> Programs and Features -> select VMware vShield Endpoint.

Uninstall VMware vShield Endpoint.

Once complete you can successfully go ahead and upgrade VMTools.

vSphere 6.5 Host Resources Deep Dive – First thoughts

Today I was fortunate to receive my copy of VMware vSphere 6.5 Host resources. I was quite excited to be ordering this book once it became available. I had been reading all the snippets that were popping up on twitter from @hostdeepdive and the information and whiteboard photos were really building up the excitement. This book is a collaboration between @FrankDenneman and @NHagoort – the combined intelligence and knowledge that these two have is far beyond where any book will take you, but in this particular book, they cram so much detail in that you will not want to put it down.  If you want to get the best out of your infrastructure, and become a great architect and get the best for your customers/users, then this is the book you need to read.

Not the only excited it has arrived!

I decided that I would read from the very first page and go through the book, just in the foreword by @KitColbert, VP & CTO of Cloud Platform Business Unit @ VMware, my mind was going places I hadn’t placed it before. Kit talks about how something as simple as typing a linux command and what occurs before the result can be a discussion that can last for hours. In the 4 and a half pages for the foreword, Kit breaks this down and for me, that got me thinking in a whole new light of detail that I would not usually have gone to. This has set me up to accept the challenge of the level of detail that has gone into this book.

I am only a few pages in, but I know for sure that this is going to be a ride worth taking and opening up for the level of detail that will be exposed.

I look forward to reading the rest of this great book and hope you may get the chance too!

You can purchase the book here.
https://www.amazon.com/VMware-vSphere-Host-Resources-Deep/dp/1540873064/ref=sr_1_1?ie=UTF8&qid=1499174386&sr=8-1&keywords=host+deep+dive

Guest Introspection Service – NSX

Continuing on from my last post, I thought I would get in and talk about the Guest Introspection service before I roll back and redeploy my NSX lab.

In prior versions to vSphere 6.x, part of the VMware vCloud Networking and Security (vCNS) was vShield Endpoint that was installed onto each host to allow for agentless security products to interact with virtual machines through VMTools. This was a two component setup, you would first have the vShield Manager that was connected to your vCenter which then added an installation option on each host for vShield Endpoint. Once vShield Endpoint was installed and vShield Driver (Part of VMTools install), your antivirus/anti-malware software could then protect inside your virtual machines that have been set up.

Fast forward to vSphere 6.x and the release of NSX taking over the networking and security side of things for vSphere environments. vShield was partially removed in vSphere 6.0, but completely removed by vSphere 6.5. Replacing vShield is now the NSX Guest Introspection Service (GIS) that still gets deployed to each host, but the difference is instead of having a separate vShield manager, it is included with the NSX Manager.
The GIS is free (Depending on vCloud licensing you may need to double check with your reseller) with the default licensing that comes with NSX Manager. There is a default key that is automatically deployed with NSX Manager giving you this access.

To set up your Guest Introspection Services, follow the below steps:

Pre-requisites:
IP Pool (If you do not have one configured, then you can set up during GIS deployment)

  1. Open up your Network and Security Tab –> Click on Installation –> Select Service Deployments.
  2. Click on the + sign –> Select Guest Introspection –> Choose when you want to deploy Now or Schedule –> click next.
  3. Select your Datacenter and cluster you want to install your Guest Introspection to –>  click next
  4. Choose your storage device and network you want to –> Decide to use DHCP or IP Pool, click Change –> Select IP Pool and Click the + sign to create the Pool.


  5. Confirm and click Finish.

The process will run through and migrate VMs between hosts if required. Once installed, your security software should detect the hosts and their current state and either require install a filter driver to the hosts and then the appliance (Third party components may vary between vendors).

This is a very straight forward service setup, but very powerful for the service it provides to your environment.

Thank you for reading. Please let a comment if you would like to. 

Upgrade NSX Manager Version

I’ve been getting into a bit of NSX lately and have a new fondness for virtual networking of which previously I knew a little about NSX and its use cases, but I had not spent time with deploying it and making some use of it. That being said, I am not a networking guy, but after watching the latest vBrownBag 3 part NSX series with Tim Davis (@ALDTD), I think it’s becoming a new passion for me. See the Series here. Part 1Part 2 – Part 3 (TBU)

With all that, here’s my first NSX post on How to upgrade your NSX Manager.

I am currently running version 6.2.7, however last week NSX Ver. 6.3.2 was released and I thought I would take the opportunity to upgrade now before I rebuild my environment and deploy 6.3.2 direct.

  1. Open up vSphere web client and select Networking and Security -> Select NSX Managers -> Then your NSX Manger -> Summary – to check the versionScreen Shot 2017-06-10 at 10.18.06 pm
  2. Download the latest upgrade bundle from my.vmware.comScreen Shot 2017-06-09 at 9.59.34 pm
  3. Log on to your NSX manager via it’s management IP using admin userScreen Shot 2017-06-10 at 10.24.31 pm
  4. Select upgrade from the home page (You will notice in the upgrade screen the version number currently running) -> Click the upgrade button -> Click Browse and search for your upgrade bundle -> Click Continue (This will upload the file)Screen Shot 2017-06-10 at 10.26.05 pm
    Screen Shot 2017-06-10 at 10.27.01 pm
    Screen Shot 2017-06-10 at 10.27.21 pm
  5. Once the upload has complete, you will be present with a warning to create a backup of your NSX Manager before proceeding with the upgrade. You also receive the option to enable SSH and to join the VMware Customer Experience Improvement Program. -> Click Upgrade when readyScreen Shot 2017-06-10 at 10.32.51 pm
    Screen Shot 2017-06-10 at 10.35.37 pm
  6. Once completed, click close and wait for your NSX Manger to restart.
  7. Next log into vSphere webclient and select Network and Security -> installation. here you will see the NSX manager upgraded and the Controller Cluster saying “Upgrade Available” Select Upgrade available and let NSX do its thing.Screen Shot 2017-06-10 at 10.50.33 pm
  8. Your controller node will go off and may say “Disconnected” – Just refresh the webclient.Screen Shot 2017-06-10 at 11.02.40 pm

Out of habit, I do a host force sync of services. I don’t have any other components set up at this stage as I had only got to setting up my transport zones, but make sure you run through and upgrade any other components you have deployed such as the Guest Introspection Service (New vShield) as well as any Edge services you have deployed.

Tip: You can go to the Network and Security Dashboard to see if there are any components that are out of date and require an upgrade. Click on the number to bring up more information.

Screen Shot 2017-06-10 at 11.10.37 pm.png

 

Thank you for reading. Please leave a comment if you have anything to say, be it more information/Corrections/requests.