NSX Labs – Nested ESXi for NSX Lab

Nested ESXi for NSX Lab

In my last lab build for NSX I came across some issues that were very time consuming to diagnose as they only show up when you start working with nsx

I built the nested esx lab in esxi5.1 with vsphere 6 esxi and vsca. See the diagrams in other post for details

 

 

1. Create first host, without configuring networking, power off and clone additional hosts

Check that hardware virtualization has been enabled in the vm

How to Enable Nested ESXi & Other Hypervisors in vSphere 5.1

 

2. Install vmware tools for nested ESXi

For esxi 6.0 the tools for nested  hosts is integrated into ESXi

For 5.5 hosts download the tools,

https://labs.vmware.com/flings/vmware-tools-for-nested-esxi

Upload the vib using scp to your host and run

esxcli software vib install -v /<path>/esx-tools-for-esxi-9.7.2-0.0.5911061.i386.vib

 

Avoid mac and duplication 

vCenter doesn’t mind the duplicate esxi mac addresses, but nsx controller deployments will fail.

Run the following on the nested host
# esxcli system settings advanced set -o /Net/FollowHardwareMac -i 1

 

Avoid uuid duplication (important on cloned esxi hosts)

Delete the line entry /system/uuid in /etc/vmware/esx.conf and run /sbin/auto-backup.sh to save it persistently. This ensure a new system uuid gets generated at boot time

 

Set nested host nic to e10003 – avoid packet loss dup! 

Packet loss dup! packets and failing controllers in nsx when using jumbo frames
Issues with controller deploys, pings to and from the controllers show intermittent timeout and packets loss,

You should change the nic for the nested vms to e1000e, can’t remember where I discovered this…

Power off a the virtual machine
Log in as root to the terminal of the VMware ESX host.
Locate and open the virtual machine configuration file (.vmx)
For each network interface you wish to change to e1000e, modify the virtualDev line:

ethernetN.virtualDev = “e1000” to ethernetN.virtualDev = “e1000e”

Reload the vm configuration

# vim-cmd vmsvc/getallvms

In this example, the Vmid is 2848.
Reload the .vmx file using this command:

# vim-cmd vmsvc/reload Vmid
# vim-cmd vmsvc/reload 2848

 

Set your domain controller as an ntp server

VCSA, NSX, Controller and Edge devices will all appreciate using NTP rather than host time, point everything to the NTP server using utc
Download the easyfix from https://support.microsoft.com/en-us/kb/816042 and run  in on the dc.

 

ref:

http://blog.vconsult.nl/2013/11/25/issue-with-jumbo-frames-after-upgrading-nested-esxi-servers-in-the-lab-to-5-5-and-fix/

http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1003020

http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1026043

Leave a comment

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.