VMware ESXi 5.5 zero touch builds – part 3 – storage

A note on storage

I won’t spend any time on NFS storage – it’s relatively straight forward, using a VMK with appropriate uplinks and failover. 

As hinted at in the previous post (interface vmmic override) ISCSI storage does however require additional configuration as per the ESXi storage guide http://pubs.vmware.com/vsphere-55/topic/com.vmware.ICbase/PDF/vsphere-esxi-vcenter-server-55-storage-guide.pdf (page 84). In short this boils down to a vmkernel interface having to be attached to a single vmnic  in a one-to-one configuration to be compatible with iSCSI port binding.

This can be achieved in two different ways:

• Two separate vSwitches with a single VMkernel interface each and a single vmnic each. In this scenario the two VMkernel interfaces need to be configured on separate subnets to ensure no connectivity issues. As most storage platforms are presented on the same subnet this is not normally a viable option.
  

• A single vSwitch uplinked with two vmnics and configured with two VMkernel interfaces. The vmnic adapter binding is overridden for each VMkernel interface with one active and one unused vmnic. This is more easily achieved and is in most situations the natural choice. 
  

(Both images courtesy of VMware, please refer to storage configuration document linked above.)

So, here goes:

# Storage
echo "INFO ${DELIMITER1}" >> ${strLogfile};
if [ "${CFG_ENABLEISCSI}" == "true" ];
then
 # Configure iSCSI software adapter, add two vmkernel interfaces with separate uplinks
 echo "INFO Enabling software iSCSI adapter vmhba33." >> ${strLogfile};
 esxcli iscsi software set --enabled=true >> ${strLogfile} 2>&1;
fi

# Port binding
for strBindVmk in $(echo ${CFG_ISCSIVMKS} | sed 's/,/\ /g');
do
 # Configure iSCSI port binding
 echo "INFO Configuring iSCSI port binding for ${strBindVmk} on software iSCSI storage adapter vmhba33." >> ${strLogfile};
 esxcli iscsi networkportal add --adapter vmhba33 --nic ${strBindVmk} >> ${strLogfile} 2>&1;
done

General networking

• Default gateway
• DNS
• NTP
• IPv6

########################################################################################################################
# Change general system networking settings

# Change system default gateway
echo "INFO  ${DELIMITER1}" >> ${strLogfile};
echo "INFO  Configuring default gateway." >> ${strLogfile};
esxcli network ip route ipv4 add --gateway ${CFG_DG} --network default >> ${strLogfile} 2>&1;
echo -e "INFO  Default gateway added ${CFG_DG}:\n" >> ${strLogfile};
esxcli network ip route ipv4 list >> ${strLogfile} >> ${strLogfile} 2>&1;

# DNS
echo "INFO  ${DELIMITER1}" >> ${strLogfile};
echo "INFO  Configuring DNS: ${CFG_SEARCHDOMAIN} / ${CFG_DNS1} / ${CFG_DNS2}" >> ${strLogfile};
esxcli network ip dns server remove --all --server=
esxcli network ip dns search add --domain=${CFG_SEARCHDOMAIN} >> ${strLogfile} 2>&1;
esxcli network ip dns server add --server=${CFG_DNS1} >> ${strLogfile} 2>&1;
esxcli network ip dns server add --server=${CFG_DNS2} >> ${strLogfile} 2>&1;

# Ntp
echo "INFO  ${DELIMITER1}" >> ${strLogfile};
echo "INFO  Configuring NTP: ${CFG_NTP1}" >> ${strLogfile};
echo "server ${CFG_NTP1}" >> /etc/ntp.conf;
/sbin/chkconfig ntpd on;

# Disable IPv6
echo "INFO  ${DELIMITER1}" >> ${strLogfile};
echo "INFO  Disabling IPv6." >> ${strLogfile};
esxcli network ip set --ipv6-enabled false >> ${strLogfile} 2>&1;

# Enable ESX shell (TSM) and SSH.
echo "INFO  ${DELIMITER1}" >> ${strLogfile};
echo "INFO  Swithcing on ESXi shell (TSM) and SSH." >> ${strLogfile};
vim-cmd hostsvc/enable_esx_shell;
vim-cmd hostsvc/start_esx_shell;
vim-cmd hostsvc/enable_ssh;
vim-cmd hostsvc/start_ssh;

# Change root passwd
echo "INFO  Changing root password." >> ${strLogfile};
echo ${CFG_PASSWORD} | passwd root --stdin;

Tidyup

• Copy log files to persistent storage (/scratch/build)
• Put host into maintenance mode
• Reboot

########################################################################################################################
# Housekeeping and reboot

# Copy build log to persistent scratch storage
echo "INFO ${DELIMITER1}" >> ${strLogfile};
echo "INFO Copying build log files to persistent storage in ${Buildfolder}." >> ${strLogfile};
cp /var/log/esxi_install.log ${strBuildfolder}/;
cp /var/log/hostd.log ${strBuildfolder}/hostd_install.log;
cp /var/log/vmkernel.log ${strBuildfolder}/vmkernel_install.log;

# Put host in MM and reboot
echo "INFO BUILD COMPLETE: ${BUILDVERSION}" >> ${strLogfile};
echo "INFO HOST REBOOTING." >> ${strLogfile};
esxcli system maintenanceMode set --enable=true;

reboot;

What’s missing

• Patching: whilst this is relatively straight forward to code it’s easier to do with update manager.
• Adding to VC: this isn’t straight forward, but doable with a bit of python code. Check http://www.virtuallyghetto.com/2011/03/how-to-automatically-add-esxi-host-to.html
• Other drivers / VIBs – same as with patching relatively straight forward, but might need the support level to be altered, depending on the VIB.