Basics
Scripted builds for Xenserver 6.2 is outlined in http://support.citrix.com/servlet/KbServlet/download/34970-102-706044/installation.pdf. Xenserver build scripts run in the bash shell so are a bit more flexible than in ESXi. The following scripts allow for host specific dynamic zero touch builds – mainly for larger environments but can be used for any number of hosts.
All files discussed in this post can be found on https://github.com/dagsonstebo/Citrix-Xenserver-6.2-zero-touch-build-scripts.
PXE boot process
The PXE boot process for Xenserver builds is as follows:
- Host is PXE booted from pxelinux.cfg menu, each host specific menu entry specifies host specific XML answer file.
- Host specific XML answer file specifies hostname and post build script.
- Post build script preloads actual build script, patches and drivers as well as host specific configuration file.
- Upon reboot build script configures host.
I.e. for each host we require a XML answer file and a host configuration file.
The menu config for Xenserver 6.2 is as follows:
LABEL XS62CN1 Xen 6.2 kernel mboot.c32 append xenserver62/xen.gz dom0_max_vcpus=1-2 dom0_mem=752M,max:752M \ com1=115200,8n1 console=com1,vga --- xenserver62/vmlinuz xencons=hvc \ console=hvc0 console=tty0 answerfile=http://192.168.0.100/xs62cn1.xml \ -answerfile install --- xenserver62/install.img
Answer file
The answer file specifies (see install guide for all options):
- Install location
- Keyboard mapping
- Install file location
- Root password
- Post install script
- NIC used during installation
- Timezone
- Hostname
As most things are configured by the actual build script later on the main things required here is the hostname – which is required to download the host specific configuration file during the post install phase, as well as the path / location to the post install script itself. The format of the answer file is as follows:
<?xml version="1.0"?> <installation srtype="ext"> <primary-disk>sda</primary-disk> <keymap>uk</keymap> <root-password>Password21</root-password> <source type="url">http://192.168.0.100/xs62-install/</source> <script stage="filesystem-populated" type="url">http://192.168.0.100/xs62-scripts/xs62ps.sh</script> <admin-interface name="eth0" proto="dhcp" /> <timezone>Etc/UTC</timezone> <hostname>xs62cn1.mylab.local</hostname> </installation>
Post install script
The post install script runs with the newly installed root filesystem mounted as /tmp/root/root/. The script prepopulates the filesystem as follows:
- Downloads host specific answer file based on hostname.
- Downloads (but does not yet install) patches. These are simply packaged up in a .tgz file containing all *.xsupdate files required installed.
- Downloads hardware specific drivers based on dmidecode return.
- Tweaks the boot splash screen during install to highlight the build process is still ongoing.
- Downloads the build script to /etc/init.d and configures this to run as a service on reboot.
- Reboots the host.
#!/bin/bash
#
# Copyright 2014 Dag Sonstebo
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
BUILDSERVER="http://192.168.0.100";
# Download host config file
mkdir /tmp/root/root/build
cd /tmp/root/root/build
strConfigfile=`grep HOSTNAME /tmp/root/etc/sysconfig/network | cut -d"=" -f2 | cut -d"." -f1`".cfg"
wget ${BUILDSERVER}/ks/xs62/${strConfigfile}
# Download model specific drivers
mkdir /tmp/root/root/build/drivers
cd /tmp/root/root/build/drivers
strManufacturer=`dmidecode --string system-manufacturer | sed 's/\ /_/g' | sed 's/\.//g' | sed 's/\,//g'`
strModel=`dmidecode --string system-product-name | sed 's/\ /_/g' | sed 's/\.//g' | sed 's/\,//g'`
strDriverfile="XS62_"${strManufacturer}"_"${strModel}".tgz"
wget ${BUILDSERVER}/xs62-drivers/${strDriverfile}
# Download patches
mkdir /tmp/root/root/build/patches
cd /tmp/root/root/build/patches
wget ${BUILDSERVER}/xs62-patches/xs62patches.tgz
# Tweak splash screen
cd /tmp/root/usr/share/splashy/themes/citrix-theme
wget ${BUILDSERVER}/xs62-scripts/xsbuildbackground.png
mv background.png ctxbackground.orig
mv xsbuildbackground.png background.png
# Download and start build script
cd /tmp/root/etc/init.d
wget ${BUILDSERVER}/xs62-scripts/xs62buildsvc
chmod 755 xs62buildsvc
chroot /tmp/root /sbin/chkconfig xs62buildsvc on
reboot
Host config files
The host configuration file can take any format – ideally something like XML, JSON, YAML, etc – as long as this can be parsed. Using any of these would allow for easier integration with CMDB backends.
In this case I’ve just used standard shell script variable assignment – the advantage being the variables can easily be read with a “source” or “.” include statement. The host configuration files can also be easily knocked up in large quantities with some simple shell scripting or an Excel macro.
The format for the template file is as follows – all relatively self explanatory:
####################################### # Xenserver 6.2 build config template # General settings CFG_HOSTNAME="hostname.fqdn.com"; CFG_IP="IP_address"; CFG_NETMASK="Netmask"; CFG_DG="Default_gateway"; CFG_DNS1="DNS_server_1"; CFG_DNS2="DNS_server_2"; CFG_SEARCHDOMAIN="Search_domain"; CFG_NTP1="NTP_IP_or_hostname"; CFG_PASSWORD="Root_password_here"; CFG_SERVERROLE="POOLMASTER | SLAVE"; CFG_POOLNAME="Pool_name"; CFG_POOLMASTER="Poolmaster_IP"; CFG_POOLMASTERPWD="Poolmaster_password"; CFG_DOM0MEM="Dom0_memory_limit | BLANK"; CFG_INITIALNIC="Initial_network_ethX_used_for_config_and_pooljoin"; CFG_EDITION="free | per-socket | xendesktop"; CFG_LICENSESRV="IP | hostname"; CFG_LICENSEPORT="27000 | other"; ###################################### # Networks and bonds # CFG_NW1_NAME="Network_name"; # CFG_NW1_DESC="Network_description"; # CFG_NW1_TYPE="bond | vlan"; # CFG_NW1_NICA="ethX"; # CFG_NW1_NICB="ethY"; # CFG_NW1_BONDMODE="active-backup | balance-slb | lacp"; # CFG_NW1_MTU="MTU"; # CFG_NW1_VLAN="VLAN_number | BLANK"; # CFG_NW1_IF="static | dhcp"; # CFG_NW1_IFIP="IP_address | BLANK"; # CFG_NW1_IFNETMASK="Netmask | BLANK"; # CFG_NW1_IFGW="Gateway | BLANK"; # CFG_NW1_IFNAME="Interface_name"; # # Continue with CFG_NW2_* etc.
Or a complete example – in this case knocked up for a CloudStack compute node installation:
####################################### # General settings CFG_HOSTNAME="xs62cn1.mylab.local"; CFG_IP="192.168.0.30"; CFG_NETMASK="255.255.255.0"; CFG_DG="192.168.0.1"; CFG_DNS1="192.168.0.2"; CFG_DNS2="192.168.0.3"; CFG_SEARCHDOMAIN="mylab.local"; CFG_NTP1="ntp.cis.strath.ac.uk"; CFG_PASSWORD="Password123"; CFG_SERVERROLE="POOLMASTER"; CFG_POOLNAME="XS62Pool1"; CFG_POOLMASTER="192.168.0.30"; CFG_POOLMASTERPWD="Password123"; CFG_DOM0MEM=""; CFG_INITIALNIC="eth0"; CFG_EDITION="free"; CFG_LICENSESRV=""; CFG_LICENSEPORT=""; CFG_NW1_NAME="cloud-private"; CFG_NW1_DESC="Cloud private network"; CFG_NW1_TYPE="bond"; CFG_NW1_NICA="eth0"; CFG_NW1_NICB="eth1"; CFG_NW1_BONDMODE="active-backup"; CFG_NW1_MTU=""; CFG_NW1_VLAN="0"; CFG_NW1_IF="none"; CFG_NW2_NAME="cloud-public"; CFG_NW2_DESC="Cloud public network"; CFG_NW2_TYPE="bond"; CFG_NW2_NICA="eth2"; CFG_NW2_NICB="eth3"; CFG_NW2_BONDMODE="active-backup"; CFG_NW2_MTU=""; CFG_NW2_VLAN="0"; CFG_NW2_IF="none"; CFG_NW2_IFIP="none";
Main build script
The main build script follows fairly standard format for linux /etc/init.d scripts, in this case it will run under run level 3 with priority 99. This script will kick off on first reboot following the post script run, and configure the host according to the host configuration file <hostname>.cfg. All build actions are logged in
- /root/build/build.log – main build steps
- /root/build/patches/patchinstall.log – all patch install information.
So – here goes:
#!/bin/bash
#
# Copyright 2014 Dag Sonstebo
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
# Citrix Xenserver 6.2 build and configuration script
#
# Set to start at runlevel 3 with priority 99.
#
# chkconfig: 3 99 99
# description: Citrix Xenserver 6.2 build script
#
######################################################################################################
# BUILD CONSTANTS
#
BUILDVERSION="Citrix Xenserver 6.2 v1.0";
BUILDSERVER="http://192.168.0.100";
BUILDSERVICE="xs62buildsvc";
BUILDFOLDER="/root/build";
BUILDLOG="${BUILDFOLDER}/build.log";
BUILDSTATUSFILE="${BUILDFOLDER}/build.step";
BUILDERROR="9999";
BUILDCOMPLETION="8888";
REBOOTDELAY="10";
POOLJOINRETRIES="600";
POOLJOINRETRYDELAY="30";
RESOLVCONF="/etc/resolv.conf";
BUILDCONFIGSCRIPT=${BUILDFOLDER}/`hostname | cut -d"." -f1`".cfg";
DRIVERSFOLDER="${BUILDFOLDER}/drivers";
PATCHFOLDER="${BUILDFOLDER}/patches";
PATCHFILE="xs62patches.tgz";
PATCHLOG="patchinstall.log";
SPLASHSCREENFILE="xsbuildbackground.png";
# FILELIST=( "path/to/file1.tgz" "path/to/file2.sh" );
DELIMITER="-----------------------------------------------------------------------------------------";
#####################################################################################################
# FUNCTIONS
#
# Interrogate DMI
function CheckHardware()
{
local strSysManufacturer;
local strSysModel;
local strSysSerial;
WriteOutput "INFO ${DELIMITER}";
WriteOutput "INFO Hardware info";
strSysManufacturer=`dmidecode --string system-manufacturer`;
strSysModel=`dmidecode --string system-product-name`;
strSysSerial=`dmidecode --string system-serial-number`;
WriteOutput "INFO System detected: " ${strSysManufacturer}" "${strSysModel};
WriteOutput "INFO System serial: "${strSysSerial};
}
########################################################################################
# Check Xenserver release
function CheckRelease()
{
local strRelease;
WriteOutput "INFO ${DELIMITER}";
WriteOutput "INFO Xenserver release";
strRelease=`grep -i xenserver /etc/redhat-release`;
if [ $strRelease == "" ];
then
WriteOutput "ERROR Script should be run on Citrix Xenservers only. No changes made.";
UpdateBuildStatus ${BUILDERROR};
exit;
else
WriteOutput "INFO Xenserver release detected:" $strRelease;
fi
}
########################################################################################
# Write to log file + console
function WriteOutput()
{
local strLogtimestamp;
strLogtimestamp=`date +"%Y-%m-%d %H:%M:%S"`;
echo -e "[BUILD]" "$*" > /dev/tty0;
echo $strLogtimestamp "$*" >> $BUILDLOG;
}
########################################################################################
# Update hardware drivers downloaded during post install.
function UpdateDrivers(){
local strDriverFile;
local strSysManufacturer;
local strSysModel;
WriteOutput "INFO ${DELIMITER}";
WriteOutput "INFO Driver update";
strSysManufacturer=`dmidecode --string system-manufacturer`;
strSysModel=`dmidecode --string system-product-name`;
case "${strSysManufacturer}" in
#Following added as example only
"Dell Inc.")
case "${strSysModel}" in
"PowerEdge M620")
# Driver specific install here
;;
*)
WriteOutput "INFO No drivers supplied for system model ${strSysManufacturer} ${strSysModel}.";
;;
esac
;;
*)
WriteOutput "INFO No drivers supplied for system manufacturer ${strSysManufacturer}.";
;;
esac
}
########################################################################################
# Install patches on all Xenserver hosts.
# Install all patches from file to prevent problems when joining pool.
function InstallXSPatches()
{
local strXSpatchfile;
local strPatchuuid;
local strLocalhostuuid;
WriteOutput "INFO ${DELIMITER}";
WriteOutput "INFO Patch install";
strLocalhostuuid=`xe host-list name-label=\`hostname\` params=uuid --minimal`;
WriteOutput "INFO Applying Xenserver patches to localhost."
cd ${PATCHFOLDER};
tar zxvf ${PATCHFILE};
for strXSpatchfile in *.xsupdate;
do
WriteOutput "INFO Installing ${strXSpatchfile}.";
strPatchuuid=`xe patch-upload file-name=${strXSpatchfile}`;
WriteOutput "INFO Patch UUID: "${strPatchuuid};
echo ${DELIMITER} >> ${PATCHLOG};
echo ${strXSpatchfile} >> ${PATCHLOG};
echo ${strPatchuuid} >> ${PATCHLOG};
xe patch-apply host-uuid=${strLocalhostuuid} uuid=${strPatchuuid} >> ${PATCHLOG} 2>&1;
done
}
########################################################################################
# Check build progress status
#
function CheckBuildStatus()
{
local strKeyPressed;
if [ -f ${BUILDSTATUSFILE} ];
then
intCurrentBuildStep=`cat $BUILDSTATUSFILE | grep CURRENTSTEP | cut -d":" -f2`;
# If build status at fatal error then quit
if [ "$intCurrentBuildStep" == "${BUILDERROR}" ];
then
# Fatal errror detected in previous step, exit.
WriteOutput "ERROR Build fatal error" $intCurrentBuildStep", exiting."
exit;
fi
else
# If status file not found then exit.
WriteOutput "ERROR No build status file found, exiting.";
exit;
fi
# Check for anyone trying to break out of build process
read -s -n 1 -t 1 strKeyPressed;
if [ "${strKeyPressed}" == "q" ];
then
WriteOutput "WARN Build interrupted by \"q\" keypress, exiting.";
UpdateBuildStatus ${BUILDERROR};
exit;
fi
}
########################################################################################
function UpdateBuildStatus()
{
local strXeLocalhostUuid;
local strXeLocalhostNewdescription;
local strNewDescription;
local strDescReturn;
if [ -f ${BUILDSTATUSFILE} ];
then
echo -e "CURRENTSTEP:"$1 > $BUILDSTATUSFILE;
WriteOutput "INFO Build status updated to" $1;
else
WriteOutput "ERROR No build status file found, exiting.";
exit;
fi
# Update host description field with build information.
strXeLocalhostUuid="";
while [ -z ${strXeLocalhostUuid} ];
do
sleep 5;
strXeLocalhostUuid=`xe host-list name-label=\`hostname\` params=uuid --minimal`;
done
# Parse new description
if [ $1 -eq ${BUILDCOMPLETION} ];
then
strNewDescription=${BUILDVERSION}" (BUILD COMPLETE)";
else
strNewDescription=${BUILDVERSION}" (BUILD INCOMPLETE STEP "$1")";
fi
# Set new host description, wait 5 sec in case toolstack not yet up
strDescReturn="null";
until [ -z ${strDescReturn} ];
do
sleep 5;
strDescReturn=`xe host-param-set name-description="${strNewDescription}" uuid=${strXeLocalhostUuid} 2>&1`;
done
# As above - wait for toolstack to come up
strXeLocalhostNewdescription="null";
until [ "${strXeLocalhostNewdescription}" = "${strNewDescription}" ];
do
sleep 5;
strXeLocalhostNewdescription=`xe host-list uuid=${strXeLocalhostUuid} params=name-description --minimal 2>&1`;
done
WriteOutput "INFO Local host description set to: "${strXeLocalhostNewdescription};
}
########################################################################################
function DownloadAllFiles()
{
local strFile;
local intRetVal;
WriteOutput "INFO ${DELIMITER}";
WriteOutput "INFO File download";
cd ${BUILDFOLDER};
for strFile in "${FILELIST[@]}";
do
wget ${BUILDSERVER}/${strFile};
intRetVal=$?;
if [ ${intRetVal} -eq 0 ];
then
WriteOutput "INFO ${BUILDSERVER}/${strFile} successfully downloaded.";
else
WriteOutput "ERROR Could not download ${BUILDSERVER}/${strFile} (error ${intRetVal}).";
fi
done
}
########################################################################################
function RebootServer()
{
# Waits $REBOOTDELAY before reboot.
sleep ${REBOOTDELAY};
WriteOutput "WARN Rebooting server."
reboot;
}
########################################################################################
function ChangeRootPwd()
{
local intRetVal;
WriteOutput "INFO ${DELIMITER}";
WriteOutput "INFO Changing root password.";
source ${BUILDCONFIGSCRIPT};
echo ${CFG_PASSWORD} | passwd root --stdin;
intRetVal=$?;
if [ ${intRetVal} -eq 0 ];
then
WriteOutput "INFO Root password successfully changed.";
else
WriteOutput "ERROR Could not change root password.";
fi
}
########################################################################################
function ConfigureBasicNetworking()
{
local strXeLocalhostUuid;
WriteOutput "INFO ${DELIMITER}";
WriteOutput "INFO Basic networking";
# Load configuration details.
source ${BUILDCONFIGSCRIPT};
hostname ${CFG_HOSTNAME};
# Update Xenserver settings.
# First retrieve host UUID, then set local host as well as the Xencentre hostname / label
strXeLocalhostUuid=`xe host-list params=uuid --minimal`;
WriteOutput "INFO Localhost UUID: "${strXeLocalhostUuid};
xe host-set-hostname-live host-uuid=${strXeLocalhostUuid} host-name=${CFG_HOSTNAME};
xe host-param-set name-label=${CFG_HOSTNAME} uuid=${strXeLocalhostUuid};
WriteOutput "INFO Hostname changed to "${CFG_HOSTNAME};
# Write searchdomain and nameservers to resolv.conf.
echo -e "search "${CFG_SEARCHDOMAIN}"\nnameserver "${CFG_DNS1}"\nnameserver "${CFG_DNS2} > ${RESOLVCONF};
WriteOutput "INFO "${RESOLVCONF}" updated with new DNS settings.";
}
########################################################################################
function ConfigureNTP()
{
# Back up old ntp.conf file, copy in the new template and write own NTP server at the end
local strTimestamp;
local strBackupfilename;
WriteOutput "INFO ${DELIMITER}";
WriteOutput "INFO NTP";
strTimestamp=`date +"%Y%m%d%H%M%S"`;
strBackupfilename="ntp_"${strTimestamp}".conf";
WriteOutput "INFO Backing up old ntp.conf file to "${strBackupfilename};
mv /etc/ntp.conf /etc/${strBackupfilename};
WriteOutput "INFO Configuring new ntp.conf with time server "${CFG_NTP1};
# Write new ntp.conf
cat > /etc/ntp.conf <> /etc/ntp.conf
# Set hardware synch back in NTP
sed -i 's/SYNC_HWCLOCK=no/SYNC_HWCLOCK=yes/g' /etc/sysconfig/ntpd
# Restart NTP
service ntpd restart;
WriteOutput "INFO New NTP settings configured, ntpd restarted.";
}
########################################################################################
# Install licenses
#
function InstallLicense()
{
WriteOutput "INFO ${DELIMITER}";
WriteOutput "INFO Licensing";
if [ "${CFG_EDITION}" == "free" ]; then
WriteOutput "INFO Configuring free license.";
xe host-apply-edition edition=free;
else
WriteOutput "INFO Configuring host with ${CFG_EDITION} license.";
WriteOutput "INFO License server / port: ${CFG_LICENSESRV}:${CFG_LICENSEPORT}.";
xe host-apply-edition edition=${CFG_EDITION} license-server-address=${CFG_LICENSESRV} license-server-port=${CFG_LICENSEPORT};
fi
}
########################################################################################
# Report kernel release and version
#
function ReportKernel()
{
WriteOutput "INFO ${DELIMITER}";
WriteOutput "INFO Kernel release";
WriteOutput "INFO Current kernel release: "`uname -r`;
WriteOutput "INFO Current kernel version: "`uname -v`;
}
#####################################################################################
# Configure poolmaster networks and bonds
#
function ConfigurePoolmasterNetworks()
{
WriteOutput "INFO ${DELIMITER}";
WriteOutput "INFO Poolmaster networking";
WriteOutput "INFO Configuring networks and bonds on poolmaster.";
# Load all host specific configuration details.
source ${BUILDCONFIGSCRIPT};
# Local scope variables
local strXe_NICA_uuid;
local strXe_NICB_uuid;
local strXe_Network_uuid;
local strXe_Bond_uuid;
local strXe_Bondmaster_uuid;
local strXe_Devicename_detected;
local strXe_VLAN_uuid;
local strXe_VLAN_detected;
local strXe_Localhost_uuid;
local strXe_Poolname_detected;
local strXe_Pool_uuid;
# Parsing variables
local intNetwork;
local strParsedNetname;
local strParsedDesc;
local strParsedNICA;
local strParsedNICB;
local strParsedBondmode;
local strParsedMTU;
local strParsedType;
local strParsedVLAN;
local strParsedIF;
local strParsedIFIP;
local strParsedIFNetmask;
local strParsedIFGW;
local strParsedIFName;
local strXe_IF_uuid;
# Iterate through all networks and bonds specified
intNetwork=0;
while :
do
# Next network
((intNetwork++));
# Reset all variables
strXe_NICA_uuid="null";
strXe_NICB_uuid="null";
strXe_Network_uuid="null";
strXe_Bond_uuid="null";
strXe_Bond_master_uuid="null";
strXe_Devicename_detected="null";
strXe_VLAN_uuid="null";
strXe_VLAN_detected="null";
strXe_IF_uuid="null";
# Parse pointer names
# Using dynamic variables rather that associative arrays
# to make host specific config file format cleaner.
strParsedNetname="CFG_NW${intNetwork}_NAME";
strParsedDesc="CFG_NW${intNetwork}_DESC";
strParsedType="CFG_NW${intNetwork}_TYPE";
strParsedNICA="CFG_NW${intNetwork}_NICA";
strParsedNICB="CFG_NW${intNetwork}_NICB";
strParsedBondmode="CFG_NW${intNetwork}_BONDMODE";
strParsedMTU="CFG_NW${intNetwork}_MTU";
strParsedVLAN="CFG_NW${intNetwork}_VLAN";
strParsedIF="CFG_NW${intNetwork}_IF";
strParsedIFIP="CFG_NW${intNetwork}_IFIP";
strParsedIFNetmask="CFG_NW${intNetwork}_IFNETMASK";
strParsedIFGW="CFG_NW${intNetwork}_IFGW";
strParsedIFName="CFG_NW${intNetwork}_IFNAME";
# Quit if reached last network
if [ -z ${!strParsedNetname} ];
then
WriteOutput "INFO No further networks specified.";
break;
fi
WriteOutput "INFO === Network ${intNetwork} ===============";
# Find the UUIDs of the bond NICs / PIFs
strXe_NICA_uuid=`xe pif-list device=${!strParsedNICA} params=uuid --minimal`;
strXe_NICB_uuid=`xe pif-list device=${!strParsedNICB} params=uuid --minimal`;
WriteOutput "INFO NIC ${!strParsedNICA} UUID: "${strXe_NICA_uuid};
WriteOutput "INFO NIC ${!strParsedNICB} UUID: "${strXe_NICB_uuid};
# Create network and catch uuid
# MTU is optional but xe does not accept empty MTU setting
if [ -z ${!strParsedMTU} ];
then
WriteOutput "INFO Creating network ${!strParsedNetname}, no MTU specified.";
strXe_Network_uuid=`xe network-create name-label=${!strParsedNetname} name-description="${!strParsedDesc}"`;
else
WriteOutput "INFO Creating network ${!strParsedNetname}, MTU ${!strParsedMTU}.";
strXe_Network_uuid=`xe network-create name-label=${!strParsedNetname} name-description="${!strParsedDesc}" MTU=${!strParsedMTU}`;
fi
# Disable auto connect
xe network-param-set other-config:automatic=false uuid=${strXe_Network_uuid};
WriteOutput "INFO Network ${!strParsedNetname} UUID: "${strXe_Network_uuid};
WriteOutput "INFO Network ${!strParsedNetname} type: "${!strParsedType};
# Process bonds, vlans (bonds a prerequisite) and interfaces.
case "${!strParsedType}" in
"BOND" | "bond" )
# Report bond mode
WriteOutput "INFO === Bond ${intNetwork} ===============";
WriteOutput "INFO Bond mode for network ${!strParsedNetname}: ${!strParsedBondmode}."
# Create bond
strXe_Bond_uuid=`xe bond-create network-uuid=${strXe_Network_uuid} pif-uuids=${strXe_NICA_uuid},${strXe_NICB_uuid} mode=${!strParsedBondmode}`;
WriteOutput "INFO Network ${!strParsedNetname} bond UUID: "${strXe_Bond_uuid};
# Get the bond master uuid
strXe_Bond_master_uuid=`xe bond-list uuid=${strXe_Bond_uuid} params=master --minimal`;
strXe_IF_uuid=${strXe_Bond_master_uuid};
WriteOutput "INFO Bond master UUID: "${strXe_Bond_master_uuid};
# Double check the bond:
strXe_Devicename_detected=`xe pif-list uuid=${strXe_Bond_master_uuid} params=device --minimal`;
WriteOutput "INFO ${!strParsedNICA} + ${!strParsedNICB} bond cross check, device detected as:" ${strXe_Devicename_detected};
# Fix searchdomain which isn't persistent after being set just in file
xe pif-param-set uuid=${strXe_Bond_master_uuid} other-config:domain=${CFG_SEARCHDOMAIN};
WriteOutput "INFO Searchdomain set to ${CFG_SEARCHDOMAIN}";
# Plug in new PIF to bring online
xe pif-plug uuid=${strXe_Bond_master_uuid};
;;
"VLAN" | "vlan")
WriteOutput "INFO === VLAN ${intNetwork} ===============";
# Bond must already exist so searching for bond uuid using the specified NICs
strXe_Bond_master_uuid=`xe bond-list slaves=${strXe_NICA_uuid}"; "${strXe_NICB_uuid} params=master --minimal`;
if [ -z ${strXe_Bond_master_uuid} ];
then
strXe_Bond_master_uuid=`xe bond-list slaves=${strXe_NICB_uuid}"; "${strXe_NICA_uuid} params=master --minimal`;
fi
WriteOutput "INFO ${!strParsedNICA} + ${!strParsedNICB} bond master UUID: "${strXe_Bond_master_uuid};
# Create VLAN
strXe_VLAN_uuid=`xe vlan-create network-uuid=${strXe_Network_uuid} pif-uuid=${strXe_Bond_master_uuid} vlan=${!strParsedVLAN}`;
strXe_IF_uuid=${strXe_VLAN_uuid};
WriteOutput "INFO VLAN ${!strParsedVLAN} untagged-PIF UUID: "${strXe_VLAN_uuid};
# Doublecheck
strXe_VLAN_detected=`xe vlan-list untagged-PIF=${strXe_VLAN_uuid} params=tag --minimal`;
strXe_Devicename_detected=`xe network-list PIF-uuids=${strXe_VLAN_uuid} params=name-label --minimal`;
WriteOutput "INFO Network ${strXe_Devicename_detected} created with VLAN ${strXe_VLAN_detected}.";
# Plug in new VLAN to bring online
xe pif-plug uuid=${strXe_VLAN_uuid};
;;
esac #bonds / vlans
# Process interfaces
WriteOutput "INFO === Interface ${intNetwork} ===============";
case "${!strParsedIF}" in
"STATIC" | "static" )
WriteOutput "INFO Configuring IP interface on ${strXe_Devicename_detected}, UUID ${strXe_IF_uuid}.";
xe pif-reconfigure-ip uuid=${strXe_IF_uuid} IP=${!strParsedIFIP} netmask=${!strParsedIFNetmask} gateway=${!strParsedIFGW} mode=static;
xe pif-param-set uuid=${strXe_IF_uuid} other-config:management_purpose=${!strParsedIFName};
xe pif-param-set disallow-unplug=true uuid=${strXe_IF_uuid};
WriteOutput "INFO IP interface configured on ${strXe_Devicename_detected}: ${!strParsedIFIP}/${!strParsedIFNetmask}, GW ${!strParsedIFGW}.";
;;
"DHCP" | "dhcp" )
WriteOutput "INFO Configuring IP interface on ${strXe_Devicename_detected}, UUID ${strXe_IF_uuid}.";
xe pif-reconfigure-ip uuid=${strXe_IF_uuid} mode=dhcp;
xe pif-param-set uuid=${strXe_IF_uuid} other-config:management_purpose=${!strParsedIFName};
xe pif-param-set disallow-unplug=true uuid=${strXe_IF_uuid};
WriteOutput "INFO IP interface configured on ${strXe_Devicename_detected}: DHCP configured.";
;;
"" | "*" )
WriteOutput "INFO No IP interfaces specified.";
;;
esac #interfaces
done #Network number
# Set poolname
strXe_Localhost_uuid=`xe host-list hostname=\`hostname\` params=uuid --minimal`;
strXe_Pool_uuid=`xe pool-list master=${strXe_Localhost_uuid} params=uuid --minimal`;
strXe_Poolname_detected=`xe pool-list master=${strXe_Localhost_uuid} params=name-label --minimal`;
WriteOutput "INFO Changing poolname to ${CFG_POOLNAME}.";
xe pool-param-set uuid=${strXe_Pool_uuid} name-label=${CFG_POOLNAME};
}
#####################################################################################
# Configure slave networks and bonds
#
function ConfigureSlaveNetworks()
{
WriteOutput "INFO ${DELIMITER}";
WriteOutput "INFO Slave networking";
WriteOutput "INFO Configuring networks and bonds on slave.";
# Load all host specific configuration details.
source ${BUILDCONFIGSCRIPT};
# Local scope variables
local strXe_NICA_uuid;
local strXe_NICB_uuid;
local strXe_Network_uuid;
local strXe_Bond_uuid;
local strXe_Bondmaster_uuid;
local strXe_Devicename_detected;
local strXe_VLAN_uuid;
local strXe_VLAN_detected;
local strXe_Localhost_uuid;
# Parsing variables
local intNetwork;
local strParsedNetname;
local strParsedDesc;
local strParsedNICA;
local strParsedNICB;
local strParsedBondmode;
local strParsedMTU;
local strParsedType;
local strParsedVLAN;
local strParsedIF;
local strParsedIFIP;
local strParsedIFNetmask;
local strParsedIFGW;
local strParsedIFName;
local strXe_IF_uuid;
# Iterate through all networks and bonds specified
intNetwork=0;
while :
do
# Next network
((intNetwork++));
# Reset all variables
strXe_NICA_uuid="null";
strXe_NICB_uuid="null";
strXe_Network_uuid="null";
strXe_Bond_uuid="null";
strXe_Bond_master_uuid="null";
strXe_Devicename_detected="null";
strXe_VLAN_uuid="null";
strXe_VLAN_detected="null";
strXe_IF_uuid="null";
strXe_Localhost_uuid="null";
# Parse pointer names
# Using dynamic variables rather that associative arrays
# to make host specific config file format cleaner.
strParsedNetname="CFG_NW${intNetwork}_NAME";
strParsedDesc="CFG_NW${intNetwork}_DESC";
strParsedType="CFG_NW${intNetwork}_TYPE";
strParsedNICA="CFG_NW${intNetwork}_NICA";
strParsedNICB="CFG_NW${intNetwork}_NICB";
strParsedBondmode="CFG_NW${intNetwork}_BONDMODE";
strParsedMTU="CFG_NW${intNetwork}_MTU";
strParsedVLAN="CFG_NW${intNetwork}_VLAN";
strParsedIF="CFG_NW${intNetwork}_IF";
strParsedIFIP="CFG_NW${intNetwork}_IFIP";
strParsedIFNetmask="CFG_NW${intNetwork}_IFNETMASK";
strParsedIFGW="CFG_NW${intNetwork}_IFGW";
strParsedIFName="CFG_NW${intNetwork}_IFNAME";
# Quit if reached last network
if [ -z ${!strParsedNetname} ];
then
WriteOutput "INFO No further networks specified.";
break;
fi
WriteOutput "INFO === Network ${intNetwork} ===============";
# Double check bonds
WriteOutput "INFO === Bond ${intNetwork} ===============";
# Find the UUIDs of the bond NICs / PIFs
strXe_Localhost_uuid=`xe host-list hostname=\`hostname\` params=uuid --minimal`;
strXe_NICA_uuid=`xe pif-list host-uuid=${strXe_Localhost_uuid} device=${!strParsedNICA} params=uuid --minimal`;
strXe_NICB_uuid=`xe pif-list host-uuid=${strXe_Localhost_uuid} device=${!strParsedNICB} params=uuid --minimal`;
WriteOutput "INFO NIC ${!strParsedNICA} UUID: "${strXe_NICA_uuid};
WriteOutput "INFO NIC ${!strParsedNICB} UUID: "${strXe_NICB_uuid};
# Find bond masteruuid
# Required for both bonds and vlans
strXe_Bond_master_uuid=`xe bond-list slaves=${strXe_NICA_uuid}"; "${strXe_NICB_uuid} params=master --minimal`;
if [ -z ${strXe_Bond_master_uuid} ];
then
strXe_Bond_master_uuid=`xe bond-list slaves=${strXe_NICB_uuid}"; "${strXe_NICA_uuid} params=master --minimal`;
fi
strXe_IF_uuid=${strXe_Bond_master_uuid};
WriteOutput "INFO ${!strParsedNICA} + ${!strParsedNICB} bond master UUID: "${strXe_Bond_master_uuid};
# Find bond device name
strXe_Devicename_detected=`xe pif-list uuid=${strXe_Bond_master_uuid} params=device --minimal`;
WriteOutput "INFO ${!strParsedNICA} + ${!strParsedNICB} bond cross check, device detected as:" ${strXe_Devicename_detected};
# Process bonds, vlans (bonds a prerequisite) and interfaces.
case "${!strParsedType}" in
"BOND" | "bond" )
# Doublecheck based on network name
strXe_Bond_uuid=`xe pif-list host-name-label=\`hostname\` network-name-label=${!strParsedNetname} params=uuid --minimal`;
WriteOutput "INFO ${!strParsedNetname} network detected as PIF with UUID ${strXe_Bond_uuid}";
# Fix searchdomain which isn't persistent after being set just in file
xe pif-param-set uuid=${strXe_Bond_master_uuid} other-config:domain=${CFG_SEARCHDOMAIN};
WriteOutput "INFO Searchdomain set to ${CFG_SEARCHDOMAIN}";
# Plug in new PIF to bring online
xe pif-plug uuid=${strXe_Bond_master_uuid};
;;
"VLAN" | "vlan")
WriteOutput "INFO === VLAN ${intNetwork} ===============";
# Doublecheck VLAN
strXe_VLAN_uuid=`xe vlan-list tagged-PIF=${strXe_Bond_master_uuid} tag=${!strParsedVLAN} params=untagged-PIF --minimal`;
# Catch untagged PIF to set IP later
# Updating this from just bond above
strXe_IF_uuid=${strXe_VLAN_uuid};
# Device name
strXe_Devicename_detected=`xe network-list PIF-uuids:contains=${strXe_VLAN_uuid} params=name-label --minimal`;
WriteOutput "INFO VLAN ${!strParsedVLAN} attached to network ${strXe_Devicename_detected}.";
# Plug in new VLAN to bring online
xe pif-plug uuid=${strXe_VLAN_uuid};
;;
esac #bonds / vlans
# Process interfaces
WriteOutput "INFO === Interface ${intNetwork} ===============";
case "${!strParsedIF}" in
"STATIC" | "static" )
WriteOutput "INFO Configuring IP interface on ${strXe_Devicename_detected}, UUID ${strXe_IF_uuid}.";
xe pif-reconfigure-ip uuid=${strXe_IF_uuid} IP=${!strParsedIFIP} netmask=${!strParsedIFNetmask} gateway=${!strParsedIFGW} mode=static;
xe pif-param-set uuid=${strXe_IF_uuid} other-config:management_purpose=${!strParsedIFName};
xe pif-param-set disallow-unplug=true uuid=${strXe_IF_uuid};
WriteOutput "INFO IP interface configured on ${strXe_Devicename_detected}: ${!strParsedIFIP}/${!strParsedIFNetmask}, GW ${!strParsedIFGW}.";
;;
"DHCP" | "dhcp" )
WriteOutput "INFO Configuring IP interface on ${strXe_Devicename_detected}, UUID ${strXe_IF_uuid}.";
xe pif-reconfigure-ip uuid=${strXe_IF_uuid} mode=dhcp;
xe pif-param-set uuid=${strXe_IF_uuid} other-config:management_purpose=${!strParsedIFName};
xe pif-param-set disallow-unplug=true uuid=${strXe_IF_uuid};
WriteOutput "INFO IP interface configured on ${strXe_Devicename_detected}: DHCP configured.";
;;
"" | "*" )
WriteOutput "INFO No IP interfaces specified.";
;;
esac #interfaces
done #Network number
}
#####################################################################################################
# Configure management eth ip address
# For slaves this is required to do the initial pool join, after this the join operation takes care of bonds etc.
# For poolmasters this is in Xen6 required to create the bond, without private IP on one NIC the bond doesn't come online.
#
function ConfigureEthIP()
{
source ${BUILDCONFIGSCRIPT};
local strXe_NICA_uuid;
WriteOutput "INFO ${DELIMITER}";
WriteOutput "INFO Initial NIC IP config";
WriteOutput "INFO Configuring management NIC IP on ${CFG_INITIALNIC}.";
# Find the UUIDs of the initial NIC
strXe_NICA_uuid=`xe pif-list device=${CFG_INITIALNIC} params=uuid --minimal`;
WriteOutput "INFO ${CFG_INITIALNIC} detected as "${strXe_NICA_uuid};
# Configure IP address
xe pif-reconfigure-ip uuid=${strXe_NICA_uuid} mode=static IP=${CFG_IP} gateway=${CFG_DG} netmask=${CFG_NETMASK} DNS=${CFG_DNS1},${CFG_DNS2};
xe host-management-reconfigure pif-uuid=${strXe_NICA_uuid};
WriteOutput "INFO ${CFG_INITIALNIC} configured as management interface with IP address "${CFG_IP}"/"${CFG_NETMASK};
}
#################################################################################################################
# Pool join
# Checks the specified poolmaster and join if poolmaster build is complete
#
function JoinPool()
{
source ${BUILDCONFIGSCRIPT};
WriteOutput "INFO ${DELIMITER}";
WriteOutput "INFO Pool join";
local intRetryCounter;
local strRemoteHostname;
WriteOutput "INFO Attempting to join poolmaster with IP "${CFG_POOLMASTER};
intRetryCounter="0";
strRemoteBuildStatus="";
while [ ${intRetryCounter} -ne ${POOLJOINRETRIES} ];
do
# Try to ping poolmaster
PingIP ${CFG_POOLMASTER};
if [ $? -eq 0 ];
then
WriteOutput "INFO Poolmaster "${CFG_POOLMASTER}" is reachable.";
strRemoteBuildStatus=`xe host-list address=${CFG_POOLMASTER} params=name-description --minimal -u root -pw ${CFG_POOLMASTERPWD} -s ${CFG_POOLMASTER}`;
if [ "${strRemoteBuildStatus}" == "${BUILDVERSION} (BUILD COMPLETE)" ];
then
# Using host description to check if build is complete
WriteOutput "INFO Poolmaster build complete: " ${strRemoteBuildStatus};
# Adding delay to prevent slaves joinging pool too quickly after poolmaster is completed
sleep 60;
xe pool-join master-address=${CFG_POOLMASTER} master-username=root master-password=${CFG_POOLMASTERPWD};
if [ $? -eq 0 ];
then
WriteOutput "INFO Successfully joined pool.";
else
WriteOutput "ERROR Problems joining pool.";
UpdateBuildStatus ${BUILDERROR};
exit;
fi
# wait for 30 seconds to allow xapi to complete all joining actions.
sleep 30;
# on join - successful or not - break out of the while loop
break;
else
WriteOutput "WARN Poolmaster build not yet complete, waiting "${POOLJOINRETRYDELAY}" seconds:" ${strRemoteBuildStatus};
fi
else
WriteOutput "INFO Can not reach poolmaster "${CFG_POOLMASTER}", waiting "${POOLJOINRETRYDELAY}" seconds.";
fi
# increment retry counter
((intRetryCounter++));
sleep ${POOLJOINRETRYDELAY};
done
#Reached max retries
if [ ${intRetryCounter} -eq ${POOLJOINRETRIES} ];
then
WriteOutput "ERROR Maximum pool join retries ("${POOLJOINRETRIES}") reached, giving up.";
UpdateBuildStatus ${BUILDERROR};
fi
}
#################################################################################################
# Ping IP address, return 1 for unreachable, 0 for reachable
function PingIP()
{
if ! ping -c 1 -w 1 "$1" &>/dev/null ;
then
# Unreachable
return 1;
else
# Response
return 0;
fi
}
#################################################################################################
# Set Dom0 memory according to config
#
function Xen62Dom0Config()
{
source ${BUILDCONFIGSCRIPT};
WriteOutput "INFO ${DELIMITER}";
WriteOutput "INFO Dom0 memory configuration";
local intMemoryLimit;
if [ "${CFG_DOM0MEM}" ]; then
# Make sure memory limit is between 400 (Citrix recommended minimum) and 4096
# See XS62 admin guide for more details
if [ ${CFG_DOM0MEM} -lt 400 ];
then
intMemoryLimit="400";
elif [ ${CFG_DOM0MEM} -gt 4096 ];
then
intMemoryLimit="4096";
else
intMemoryLimit=${CFG_DOM0MEM};
fi
# Configure new memory limit
/opt/xensource/libexec/xen-cmdline --set-xen dom0_mem=${intMemoryLimit}M,max:${intMemoryLimit}M;
WriteOutput "INFO Host DOM0 memory limit set to ${intMemoryLimit}, host rebooting for limit to take effect.";
else
WriteOutput "INFO No host DOM0 memory limit specified.";
fi
}
function PostBuildHousekeeping()
{
source ${BUILDCONFIGSCRIPT};
WriteOutput "INFO ${DELIMITER}";
WriteOutput "INFO Housekeeping";
# Switch build service off
chkconfig ${BUILDSERVICE} off;
WriteOutput "INFO Build service disabled: "`chkconfig --list ${BUILDSERVICE}`;
# Restarting Xapi just to make sure all OK
WriteOutput "INFO Restarting Xapi.";
/opt/xensource/bin/xe-toolstack-restart;
# Sort out splashy back to original Citrix splash screen
cd /usr/share/splashy/themes/citrix-theme
mv background.png ${SPLASHSCREENFILE};
mv ctxbackground.orig background.png;
cd ${BUILDFOLDER};
# Tidy files no longer required
if [ "${CFG_SERVERROLE}" = "SLAVE" ];
then
WriteOutput "INFO Removing patch files not required.";
rm -f ${PATCHFOLDER}/${PATCHFILE};
fi
WriteOutput "INFO BUILD IS NOW COMPLETE.";
}
#################################################################################################################################
# MAIN SCRIPT EXECUTION
case "$1" in
start)
#############################################################################################
#
# Install execution: upon completion every step update the status file to the
# next build step. This keeps the build steps persistent across reboots - i.e.
# make sure next build step is updated BEFORE reboot, otherwise the server will continue rebooting.
# For fatal errors the build status will be updated to ${BUILDERROR}, the CheckBuildStatus function
# will check for this and stop all script execution. Normal script execution completion is
# indicated with a build status of ${BUILDCOMPLETION}, at which point this build service will be disabled.
#
#
if [ ! -f ${BUILDSTATUSFILE} ];
then
echo -e "CURRENTSTEP:1" > $BUILDSTATUSFILE;
fi
CheckBuildStatus;
while [ ${intCurrentBuildStep} -ne ${BUILDCOMPLETION} ];
do
case "$intCurrentBuildStep" in
1) #Prep, download files etc.
CheckRelease;
CheckHardware;
ReportKernel;
# DownloadAllFiles;
ChangeRootPwd;
ConfigureNTP;
InstallLicense;
UpdateBuildStatus 2;
;;
2) #Configure basic networking
ConfigureBasicNetworking;
UpdateBuildStatus 3;
;;
3) #Patch all servers from file and reboot. Now done before pool join to avoid join errors.
InstallXSPatches;
UpdateBuildStatus 4;
RebootServer;
exit;
;;
4) #Configure networks and bonds on poolmaster.
#Networks and bonds are auto created when slave join pool hence these are double checked only.
source ${BUILDCONFIGSCRIPT};
case "${CFG_SERVERROLE}" in
POOLMASTER)
WriteOutput "INFO Configuring poolmaster networking.";
ConfigureEthIP;
ConfigurePoolmasterNetworks;
;;
SLAVE)
WriteOutput "INFO Configuring slave networking.";
ConfigureEthIP;
JoinPool;
# Sleep to allow host to catch up with poolmaster
sleep 60;
ConfigureSlaveNetworks;
;;
*)
WriteOutput "ERROR No pool role detected, exiting.";
UpdateBuildStatus ${BUILDERROR};
exit;
;;
esac #server role
UpdateBuildStatus 5;
;;
5) # Dom0 mem config
Xen62Dom0Config;
UpdateBuildStatus 6;
RebootServer;
exit;
;;
6) #Host has been rebooted, patch drivers (which rely on patches being installed)
#WriteOutput "INFO Server back up after reboot ("`uptime | cut -d"," -f1 | cut -d" " -f3,4,5`")";
UpdateDrivers;
UpdateBuildStatus 7;
RebootServer;
exit;
;;
7) #Tidy for now.
WriteOutput "INFO Server back up after reboot ("`uptime | cut -d"," -f1 | cut -d" " -f3,4,5`")";
PostBuildHousekeeping;
UpdateBuildStatus ${BUILDCOMPLETION};
;;
esac
# Check current build status before next iteration.
CheckBuildStatus;
done #Loop for progressing through build
;;
stop)
;;
status)
;;
*)
echo $"Usage: start|status}"
;;
esac
exit
blog.sonstebo.com 