Citrix XenServer/XenCenter Installation

Everyone wants to build their XenServer box, define VMs and do cool stuff.  But there needs to be a logical approach to this effort and I would like to explore that avenue here.  Here are the basic but very needed steps:

1.  BIOS settings:

  • VT-D enabled
  • IOMMU enabled
  • IGPU multi set to ENABLE

2.  build the xenserver (download the xenserver ISO image and burn to DVD)  This image will contain 3 important components:
a.  the XenServer – installed on your dedicated pc (it will be a Linux varian that gets installed)
b.  the XenCenter – this is installed only on a windows pc running some current version of Windows.
c.  the xs-tools.iso – this gets installed on your virtual machines. These tools must be installed on every VM creagted

3.  the logical build is:

. XenServer
. XenCenter
. update with all of the latest hotfixes.
. define a single VM – includes network configuration, etc.
. install the xs-tools.iso for your VMs (Linux or Windows).

INSTALL the XenServer:

You will install the XenServer on a dedicated PC.  No other software should be on this PC.  Load the XenServer DVD (iso image) into your dedicated PC and power up.  Your bios should define a boot logic that tries to load a DVD if one is present.  I am not going to provide information on the various questions that are asked during the install because it is straight forward..  There is a readme/install paper on the XenSever site that is worth a read before beginning.

When the install of the XenServer is done you will be told to remove the DVD and boot.  You will then be presented with XenWindow.

INSTALL XenCenter

XenCenter is installed on a separate physical PC running Windows.  It will provide a graphical interface into your XenServer running on its own box and will allow you to define the VMs (Virtual Machines) on XenServer.  XenCenter is on the same DVD as is XenServer.

Now take the XenServer DVD that you just built the server with and put in a windows box.   Do not boot the windows box with this DVD in the tray unless you want a second XenServer and no Windows Box.

Using the Windows file explorer go to the DVD device and go to the “client_install” directory/folder on the DVD.  Double-click on the XenCenter.msi file to begin the install of XenCenter.  Follow the on screen instructions.

Install latest hotfixes

Find the XenCenter icon on the PC where you just installed XenCenter and launch it if does not auto launch.

Look at the top right.  You will see something about “System Alerts”.  These are usually referring to bug fixes/patches that are missing from your current XenServer that you just installed.  They are not referring to patches for XenCenter even though there might be one notice telling you about a “newer” XenCenter than you have.  Most of these patches will be installed on your XenServer HOST.
By clicking on this “link” you will launch a window detailing the patches that need installing.  NOW PAY ATTENTION.  Under the XenCenter “tools –> install update” it implies that you can install these patches.  This is NOT true with the “FREE” version.  It will not allow you to select the host on which you want to install them.  This is a neat feature but reserved for the “purchased” product only.  This means you will have to do this work by hand.  A bit of a pain but not really difficult.  I will explain.:

If you will “click” on the “System Alerts” icon at the top right you will get a list of needed fixes.  In the list is a link to the XenServer Website where you can download these fixes in the form of a “RAR” file.  Associated with these links are specific instruction on how to install the fixes/patches.  For the most part they all install the same way.  My way is based upon downloading these fixes to a windows box, unwinding the rar file, moving the resultant “XXXX##X##.xsupdate” files to a directory on the XenServer HOST machine, and then installing them.  I created a directory called hot_fixes and move them there from the windows machine.

You only need the files ending in “.xsupdate”.  Now when you download these fixes you need to open a new tab in your browser for each one so you can refer to the installation procedures.  I am now going to provide you with the 5 steps for installing each patch.  Please read the 5 steps before attempting so you get an idea of the “flow”.  These commands are an example of one patch I needed.
The numbers will change.  Do NOT cut and paste my example.  Also some patches my require a different install procedure so it is important that you read those instruction on the website.  For the most part I found them to install pretty much the same way.  But….

To update XenServer with hotfixes you must use the “xe” commands on the XenServer HOST from the command line.  Remember, you  CANNOT do it with XenCenter unless you have a licensed version.

Short summary of install patch sequence for the XS62E002 fix:

1.  xe patch-upload file-name=XS62E002.xsupdate
2.  59128f15-92cd-4dd9-8fbe-a0115d1b07a2
3.  xe patch-pool-apply uuid=59128f15-92cd-4dd9-8fbe-a0115d1b07a2
4.  xe patch-list name-label=XS62E002

If the update has been successful, the hosts field will contain the UUIDs of the hosts this patch was successfully applied to. This should be a complete list of all hosts in the pool.

Also PAY ATTENTION to the suggested “to do” thing.  I forgot to copy this so its a bit vague but it usually says to reboot the host after the patch.

Back in the “System Alert” window if still open do a refresh and the patch you just installed should be “gone”, i.e. it is installed and not showing up.  Also in XenCenter click on your XenServer host in the left hand pane and select the “General” tab —> Updates (may have to scroll down).  Here will be a list of all installed updates.

Define a virtual host

The following VM discussion is primarily for Linux only because Linux can support both types of VMs whereas Windows can only support HVM.

Regardless of which OS you will install on a VM now is the time to place that DVD in the XenServer’s DVD tray and close it. After you have defined your VM the DVD will run in the new VM’s console tab where you will be able to interact by entering user name, hostname, etc.

Also note that in the process of defining your new VM there a several steps the wizard takes you through. Some may not be available, i.e., GPU if you don’t have one installed.  Most of these attributes/step can be modified later, although CPU and Memory would be nice to do now.

Step 1:  Template

You can define two types of VMs:

HVM-hardware virtualization machine
PVM-para virtualization machine

the difference:  HVM – domains are based upon emulation of hardware within software
PVM – no hardware emulation at all

here is how to define the type of VM you want – this is determined by what you want to do.

For example if you want to pass through devices you are most likely going to need an HVM.
Remember Windows is HVM based hence the ability to pass-through GPUs.  But in Linux
if you want to pass-through PCI/USB devices you need to build/define an HVM.  “pass-through”
will not work if you define a Linux VM as a PVM.

Using XenCenter to define a VM:

define an HVM:
click “New VM”
select “Other Install media”  <—  THIS WILL FORCE an HVM
choose the original ISO CD image from DVD drive (or NFS ISO storage repository)

define a PVM:
click “New VM”
select a template like Debian Squeeze 6 32 bits” (By using predefined templates you will force a PVM VM)
choose the original ISO CD image drom DVD drive

Here is a test of VM type:  after you have defined the VM and installed the OS login to the OS and type:
“lspci”

on the HVM machine you will get a list of your emulated devices, i.e.,

lspci returns:

00:00.0 Host bridge: Intel Corporation 440FX – 82441FX PMC [Natoma] (rev 02)
00:01.0 ISA bridge: Intel Corporation 82371SB PIIX3 ISA [Natoma/Triton II]
00:01.1 IDE interface: Intel Corporation 82371SB PIIX3 IDE [Natoma/Triton II]
00:01.2 USB controller: Intel Corporation 82371SB PIIX3 USB [Natoma/Triton II] (rev 01)
00:01.3 Bridge: Intel Corporation 82371AB/EB/MB PIIX4 ACPI (rev 01)
00:02.0 VGA compatible controller: Cirrus Logic GD 5446
00:03.0 SCSI storage controller: XenSource, Inc. Xen Platform Device (rev 0

on the PVM

lspci returns:

nothing

Step 2:  Name – make up a short one but meaningful name i.e., win7_32bit or centos6_54 in the case of linux.

Step 3:  Installation Media – where is it?  a dvd in the tray you loaded earlier or an ISO on the network.

Step 4:  Home Server

Step 5:  CPU and Memory – how many virtual CPUs do you want and how much memory for each one.  NOTE:  you may run into an issue on a Windows virtual machine in that you define 2 or more but after the install of Windows completes you only have one (1) CPU.  We will cover how to fix this later.

Step 6:  GPU – if you have one installed on the XenServer host.  For now select “None’’.  Can be changed later.

Step 7: Storage

Step 8: Networking

Step 9: Finish

XenCenter will show you your new VM in the left pane.  It should be running.  If not right click on it and click “start”.  Open the “Console” tab for your new VM and you should see the DVD that you placed earlier in the tray loading.  You then interact as you normally would during installation of your OS.

INSTALL THE XS-TOOLS FOR XenServer HOST

XS-TOOLS is an absolute requirement for all VMs regardless of the OS installed. You cannot skip doing this installation.

For the install of XS-TOOLS to be successful you need to ensure that the install of Windows onto your VM has Microsoft >NET Framework 4.0 or higher.  This is needed not only for XS-TOOLS but also later when you install your GPU drivers.  If you do not have 4.0 version installed (look at the “Programs and Features” list on your Windows VM) then now would be a good time to run Windows Update (make sure your networking is operational – if using DHCP you probably set but if not you would need to define it now).  When Windows Update completes boot the machine from the “Start” button in the console tab on XenCenter

This is the hard part:  Where is the xs-tools.iso?  It took me hours of “googling” to find it.  It is in one of two places:

1.  On the XenServer host in the directory /opt/xensource/packages/iso you will find the xs-tools.iso files.  Also note that this location is where any new versions of xs-tools will be located.  This can happen when installing hotfixes.  hotfixes also provide new xs-tool in addition to XenServer patches.

2.  This file also exists on the XenServer ISO disk in DVD_drive:XenServer-6.2.0/packages.main directory.

Copy this file to a directory and unwind it using winzip.

Burn the resulting ISO image to a DVD.  you will be putting this DVD into the drive on your XenServer HOST  machine so that it can be accessed by your VMs later.

XS-TOOLS ON LINUX INSTALL:

now ssh to the virtual machine and login

now become root  sudo -s

cd /mnt
mkdir xs-tools
ls
mkdir xs-tools
mount /dev/xvdd /mnt/xs-tools
mount: block device /dev/xvdd is write-protected, mounting read-only
BE SURE YOU ARE IN THE Linux directory in the next stop
cd /mnt/xs-tools/Linux
pwd —>  BE SURE YOU ARE IN THE Linux directory
bash install.sh
Detected `Ubuntu 12.04.3 LTS’ (ubuntu version 12).

The following changes will be made to this Virtual Machine:
* update arp_notify sysctl.conf.
* packages to be installed/upgraded:
– xe-guest-utilities_6.2.0-1133_amd64.deb

Continue? [y/n] y

Selecting previously unselected package xe-guest-utilities.
(Reading database … 59532 files and directories currently installed.)
Unpacking xe-guest-utilities (from …/xe-guest-utilities_6.2.0-1133_amd64.deb) …
Setting up xe-guest-utilities (6.2.0-1133) …
Mounting xenfs on /proc/xen: OK
Detecting Linux distribution version: OK
Starting xe daemon:  OK
Processing triggers for ureadahead …

You should now reboot this Virtual Machine.

XS-TOOLS ON WINDOWS INSTALL:

stop any vm that is using the dvd drive, i.e., if it is mounted in a Linux vm it will not be available to windows.

start the windows vm in not running.

go to the vm console and log into windows.

Launch windows explorer

under “computer” you should see your dvd drive

click on it

double click on the install wizard (installwizard.msi)

 

ADDING an additional tool to XenServer host

If you are considering do pass-through of usb devices you should install the usb utilities.  Here is how:

ssh into your XenServer host as root

yum install usbutils –enablerepo=base <—install the utilities.

test:

lsusb

{listed usbdevices}

 

What to do if your Windows install only shows 1 CPU in task-manager when you defined 2 or more CPUs when defining the VM

It is suggested that to fix this problem you use “msconfig” which is part of your Windows distribution.  Do not use this solution it is completely wrong for what we are doing and from what I have read it is not the solution to other problems.  If you look in device manager on the windows VM you might see two cpus defined but windows task manager shows we are only using one.  here is how to fix this ( you may have to change VCPUs-max value depending upon your CPU):

  1. place your windows VM into a halt state (right click on it and “stop” it)
  2. xe vm-list name-label={your VM name}
  3. from the above step grab the UUID # (79c9794d-fda9-9a8c-c5d7-c19b929bd1d8 for example)
  4. xe vm-param-set platform:cores-per-socket=4 uuid=number from step 2
  5. xe vm-param-set VCPUs-max=8 uuid=number from step2
  6. xe vm-param-set VCPUs-at-startup=2 uuid=number from step2
  7. restart the VM and check win’s task manger to see if you now have 2 cpus from step 6.

Pass-through

GPU pass-through to Windows

  • you have correctly setup your bios for VT-d or IOMMU
  • to see the GPUs on your XenServer host do:  xe pgpu-list
    • to see if IOMMU chipset feature is enabled
      • xe host-list – capture the uuid number it is quite long
      • xe host-param-get uuid=#_from above step param-name=chipset-info param-key=iommu
        • if “false” is returned then it is NOT enabled and you need to enable it in the bios
  • place your GPU driver disk in the XenServer tray and install the drivers.  In the Console tab for your windows VM look at the DVD drive at the top if it says <empty> then select the down arrow and choose “dvd drive on your-host”.  the dvd should load and you can now install your drivers.
  • look at the windows “Device Manager” it should show your GPU.  One device might have a “yellow bang” and that is ok AS LONG AS IT IS NOT YOUR GPU DEVICE.
  • your networking is setup.

So far you have been able to get to your Window VM through the “Console Tab” in XenServer.  However you are about to do select your GPU for pass through.  When you do this the Console window will state that you are in pass-through mode and you cannot use the Console window.  You need to login another way.  So you now need to install a VNC server on your Windows VM first or you will not be able to login when pass-through is established.  I suggest “REALVNC” from HERE.  Get both the server and the client.  You will install the server on your Windows VM and the client on another windows pc that you will use to login to your Windows VM.  Read about it before installing.  You might think that you can use the Windows built in Remote desktop.  You cannot.  It is incompatible with pass-through and your version of windows might not have it.  Install your VNC server now.

  • stop your windows VM
  • right click on the windows VM in the left pane and select properties
  • select GPU – now look closely you should see up/down arrows on the right side if your BIOS allows multi monitors/graphics cards.  select the one that is your GPU device.
  • start the VM
  • watch the console window you will see it reboot and the the Windows logo appears.  You can no longer login with this console window.  So start your VNC client on a windows pc and use it to log in to your windows VM.
  • when logged in look at device manager to be sure it looks ok.  remember a “yellow bang” by another graphics card is ok.  See if device manager correctly defines you GPU.
  • download and install BOINC if your doing distributed computing.

 

Audio pass-through to Linux

NOTE:  This must be done on an HVM (see above).

login to your Linux virtual host
lspci | grep -i Audio
nothing should be returned
exit

login to your XenServer Host as root

lspci | grep -i Audio

00:03.0 Audio device: Intel Corporation Xeon E3-1200 v3/4th Gen Core Processor HD Audio Controller (rev 06)

00:1b.0 Audio device: Intel Corporation 8 Series/C220 Series Chipset High Definition Audio Controller (rev 05)

01:00.1 Audio device: Advanced Micro Devices, Inc. [AMD/ATI] Cape Verde/Pitcairn HDMI Audio [Radeon HD 7700/7800 Series]

02:00.1 Audio device: Advanced Micro Devices, Inc. [AMD/ATI] Caicos HDMI Audio [Radeon HD 6400 Series]

I am going to pass through the audio device with the address of: 00:1b.0

Now notice that we need to expand this address to: 0000:00:1b.0

Get a list of your all virtual machines on the XenServer HOST.

A lot of output follows but this is the VM I want:

xe vm-list

o
o
o
uuid ( RO) : 8e7a5486-515d-34c8-60e5-1c6c9bc41d24
name-label ( RW): ubuntu1204_HVM
power-state ( RO): running

Now pass through the audio device:
xe vm-param-set other-config:pci=0/0000:00:1b.0 uuid=8e7a5486-515d-34c8-60e5-1c6c9bc41d24

In the above note the: 0/0000:00:1b.0 each PCI address starts with a “#/”, with each address is separated by a comma if passing multiple devices.
So if I were going to pass through two devices it would look like the following:

Start the counter at 0 and increase by one for each device you add. Here is the string I would use for multiple devices use:

0/0000:06:00.0,1/0000:00:1a.0

Note the 0/ and 1/ separated by a comma you can do this while the virtual machine is running but it will not take effect until you do a “stop/shutdown” followed by a “start”. This is NOT the same as a “restart”.

In the event you need to remove the device above do the following:

xe vm-param-remove param-name=other-config param-key=pci uuid=”UUID of selected VM”

shutdown the VM

start the vm

login to the VM node

lspci | grep -i Audio

00:05.0 Audio device: Intel Corporation 8 Series/C220 Series Chipset High Definition Audio Controller (rev 05)

Note the device address on the VM.  It is not the same as on the XenServer noted at the beginning.

test:

aplay {path_to_some_wav_file}
speaker should respond.

6 comments on “Citrix XenServer/XenCenter Installation”

  1. Marcelo Viana Reply

    Help me!

    [root@xenlocal ~]# xe vm-param-set other-config:pci=0/0000:00:1b.0 uuid=203c0584-3985-5a6a-f63f-b406d69e9fd2

    [root@xenlocal ~]# xe vm-start uuid=203c0584-3985-5a6a-f63f-b406d69e9fd2
    The server failed to handle your request, due to an internal error. The given message may give details useful for debugging the problem.
    message: xenopsd internal error: Xenctrl.Error(“38: Function not implemented”)

  2. Ron Reply

    Marcelo,

    What is your operating system and what type of device are you trying to pass through? Also are you sure that the “pci=0/0000:00:1b.0” address is the correct address for your devcie? In my example the “00:1b.0” is the correct address for me (an audio device) but might not be for you.

  3. Mani Reply

    Hi,
    thank you for the post. it is very useful, can you please share the link to download xs-tools6.2 . because i am unable to locate it. please share the url or upload on google one drive.
    thank you.
    Mani

  4. Ron Reply

    Mani, Please scroll up to “INSTALL THE XS-TOOLS FOR XenServer HOST” on this page. Reading down it will tell you where xs-tools is located for this release level of XenServer. If you are using a later release the “tools” might be located in a different location. My guess is that it is still supplied on the ISO. CD to your .iso file on your DVD and check the location I refer to above. Unfortunately I am no longer using XenServer so I will not be able to help you much beyond what is in this “readme”.

Leave A Reply

Your email address will not be published. Required fields are marked *


*

Bad Behavior has blocked 449 access attempts in the last 7 days.