How to enable nested virtualization on Google Cloud

Google Cloud supports nested virtualization using the KVM hypervisor on Linux instances. This nested virtualization feature allows you to run one or more virtual machines inside a Linux virtual machine by leveraging Intel VT-x processor, It means VMs inside of VMs. The only requirement is that the VM has to run on an Intel Haswell or newer CPU. Here are the steps, how to enable nested virtualization on google cloud.

  • ESX and Xen hypervisors are not supported.
  • Nested virtualization does not currently support Windows instances.

Google has verified nested virtualization tests which work perfectly using the following Linux distros. The list of available regions and zones for the CPU platforms available in each zone.

– CentOS 7 with kernel version 3.10
– Debian 9 with kernel version 4.9
– Debian 8 with kernel version 3.16
– RHEL 7 with kernel version 3.10
– SLES 12.2 with kernel version 4.4
– SLES 12.1 with kernel version 3.12
– Ubuntu 16.04 LTS with kernel version 4.4
– Ubuntu 14.04 LTS with kernel version 3.13

How to Enable nested virtualization on Google Cloud

You can enable nested virtualization using ‘gcloud’ command-line tool or the API. It would be required to create a custom image with the special license key required for virtualization. We have created a new nested virtualization image using an existing image as a source image.

Log in your Google Cloud Console

Click top right corner - > Activate Google Cloud Shell
google cloud shell

GCP shell

List all compute images including deprecated.

$ gcloud compute images list --show-deprecated
gcloud list images

gcloud list images

You can use any one of the image as the source image to enable nested virtualization.


$ gcloud compute images create <nested_image_name> \
--source-image=<os_image_name> --source-image-project=<source_image_project> \

For example, the following command creates an image named ‘gcp-nested-vm-image‘ from an existing centos image named ‘centos-7-v20170719‘.

$ gcloud compute images create gcp-nested-vm-image \
--source-image=centos-7-v20170719 --source-image-project=centos-cloud \

Once done. Create a VM instance with the custom image in a zone that supports Haswell or higher.

Using the Console.

Login Compute Engine -> VM instances -> CREATE INSTANCE

Using gcloud command.

$ gcloud compute instances create example-nested-vm --zone us-central1-b \
--image gcp-nested-vm-image

Once VM created, connect the VM SSH and check that nested virtualization is enabled by running the following command.

# grep -cw vmx /proc/cpuinfo


# cat /proc/cpuinfo | grep vmx

A non-zero response confirms that nested virtualization is enabled.

You have enabeld the nested virtualization on your google cloud. Now you can start a nested VM in many different ways like qemu-kvm installation.



ESXi adding datastore Error Stack Call “HostDatastoreSystem.QueryVmfsDatastoreCreateOptions”

Most of the datacenter has given the hosts which already used and properly not cleaned to the new customers. Please note this whenever you are going to dismantle any storage, make sure you properly cleaned before given to server providers. The following error we received while we create a datastore for the new ESXi host in vSphere client.

Error:Cannot change the host configuration. Error Stack Call “HostDatastoreSystem.QueryVmfsDatastoreCreateOptions” for object “ha-datastoresystem”

There are various situations which might lead to pre-existing info on the disk. It was an issue with pre-existing or incompatible information on the hard disks. when you try to re-purpose existing storage devices is ensuring that all data and existing partitions have been completely removed.

ESXi adding datastore errors

ESXi adding datastore Error

This issue has resolved in ESXi using partedUtil tool, you need to run the following command for each disk that you’re having issues.

NOTE: This will ERASE ALL DATA on the disk in question so be careful to select the right disks!

List all the existing devices

[[email protected]:~] /usr/lib/vmware/vm-support/bin/

To get each device information.

[[email protected]:~] esxcli storage core device smart get -d <device-id>

Run partedUtil command to fix Error:Cannot change the host configuration. Error Stack Call “HostDatastoreSystem.QueryVmfsDatastoreCreateOptions” for object “ha-datastoresystem”

[[email protected]:~] partedUtil mklabel /dev/disks/<disk id> msdos



[[email protected]:~] /usr/lib/vmware/vm-support/bin/
SMART Information for disks.

Device: t10.ATA_____WDC_WD1003FAYX2D01Y7B1________________________WD2DWCAE35326661
Parameter Value Threshold Worst
Health Status OK N/A N/A
Media Wearout Indicator N/A N/A N/A
Write Error Count N/A N/A N/A
Read Error Count 200 51 200
Power-on Hours 42 0 42
Power Cycle Count 100 0 100
Reallocated Sector Count 200 140 200
Raw Read Error Rate 200 51 200
Drive Temperature 114 0 107
Driver Rated Max Temperature N/A N/A N/A
Write Sectors TOT Count 200 0 200
Read Sectors TOT Count 200 0 200
Initial Bad Block Count N/A N/A N/A
[[email protected]:~] partedUtil mklabel /dev/disks/t10.ATA_____WDC_WD1003FAYX2D01Y7B1________________________WD2DWCAE35326661 msdos

You have done. Add new datastore via the vSphere Client or vCenter Storage -> Add Storage.

This is also can be done via the vSphere Web Client, vCenter has introduced in vSphere 6.0 Update 1 to help assist with this procedure.

You can find this new option (icon with disk and eraser) by clicking onto a specific ESXi host and then select the Manage->Storage Adapters and then be highlighting the specific storage device you wish to erase as seen in the screenshot below.

Before proceeding to erase any disk, review twice that is the correct disk.

Fix vCenter storage errors

vCenter storage errors

Once you have finished you can go back to the vSphere client and add the disks successfully, no reboots required.



Configure multiple Tomcat instances on single server

Learn how to set up several different instances of an application on your Tomcat server, and manipulate each of these instances independently. This article will explain to you how to configure multiple Tomcat instances on single server.

Many times we come to a situation where we need to modify the server configuration such that it is specific to an application. And if we got more than one such applications and we want each application to have it’s own defined configuration options, then it needs some sort of configuration. In this tutorial, I am going to discuss the changes you should make to have different instances of tomcat for each application.


Install JAVA

[[email protected] ~]# sudo yum install java-1.8.0-openjdk.x86_64
[[email protected] ~]# sudo java -version
openjdk version “1.8.0_141”
OpenJDK Runtime Environment (build 1.8.0_141-b16)
OpenJDK 64-Bit Server VM (build 25.141-b16, mixed mode)

[[email protected] ~]# sudo cd /opt/; wget
[[email protected] ~]# sudo tar -zxvf apache-tomcat-8.0.52.tar.gz
[[email protected] ~]# sudo cd apache-tomcat-8.0.52

Create the subdirectories of each instance directory.

[[email protected] ~]# sudo mkdir /var/lib/apache-tomcat-8.0.52

[[email protected] ~]# sudo mkdir -p /var/lib/apache-tomcat-8.0.52/{custserv,prodserv}

[[email protected] ~]# sudo mkdir -p /var/lib/apache-tomcat-8.0.52/custserv/{logs,conf,temp,webapps,work}
[[email protected] ~]# sudo mkdir -p /var/lib/apache-tomcat-8.0.52/prodserv/{logs,conf,temp,webapps,work}

Copy the conf directory from the default Tomcat installation into each instance.

[[email protected] ~]# sudo cp -r /opt/apache-tomcat-8.0.52/conf/* /var/lib/apache-tomcat-8.0.52/custserv/conf
[[email protected] ~]# sudo cp -r /opt/apache-tomcat-8.0.52/conf/* /var/lib/apache-tomcat-8.0.52/prodserv/conf

Create symbolic links to the default bin and lib locations.

[[email protected] ~]# sudo ln -s /opt/apache-tomcat-8.0.52/bin /var/lib/apache-tomcat-8.0.52/custserv/bin
[[email protected] ~]# sudo ln -s /opt/apache-tomcat-8.0.52/bin /var/lib/apache-tomcat-8.0.52/prodserv/bin

[[email protected] ~]# sudo ln -s /opt/apache-tomcat-8.0.52/lib /var/lib/apache-tomcat-8.0.52/custserv/lib
[[email protected] ~]# sudo ln -s /opt/apache-tomcat-8.0.52/lib /var/lib/apache-tomcat-8.0.52/prodserv/lib

Create /usr/lib/systemd/system/tomcat-custserv.service and /usr/lib/systemd/system/tomcat-prodserv.service to configure the environment for each instance.

[[email protected] ~]# sudo vi /usr/lib/systemd/system/tomcat-custserv.service
Description=Apache Tomcat Web Application Container


Environment='CATALINA_OPTS=-Xms512M -Xmx1024M -server -XX:+UseParallelGC'

ExecStop=/bin/kill -15 $MAINPID



Update same configuration for /usr/lib/systemd/system/tomcat-prodserv.service and change path.

Make sure user has created and updated for tomcat folders.

[[email protected] ~]# sudo useradd -s /bin/nologin -g tomcat -d /opt/apache-tomcat-8.0.52 tomcat
[[email protected] ~]# sudo cd /opt/apache-tomcat-8.0.52
[[email protected] ~]# sudo chgrp -R tomcat conf
[[email protected] ~]# sudo chown -R tomcat logs/ temp/ webapps/ work/ bin/ /lib

Change default ports in prodserv/conf/server.xml and make sure ports 8080,8005,8009 are not conflicting with custserv/conf/server.xml

[[email protected] ~]# sudo systemctl start tomcat-prodserv.service
[[email protected] ~]# sudo systemctl start tomcat-custserv.service