CloudStack - Revision history

Leo: /* Rebuilding UI */

2025-07-10T17:49:48Z

Rebuilding UI

← Older revision		Revision as of 17:49, 10 July 2025
Line 650:		Line 650:

	=== Rebuilding UI ===		=== Rebuilding UI ===
	~~Instructions~~ to ~~rebuild~~ the UI ~~are available on~~ the [https://github.com/apache/cloudstack/tree/main/ui README file under the ui directory.]		CloudStack's web interface is bundled with the pre-built cloudstack-ui package. If you need to make any custom changes to the UI, you can follow the build instructions from the [https://github.com/apache/cloudstack/tree/main/ui README file under the ui directory.]

	~~In summary, to rebuild~~ the ~~CloudStack~~ UI~~, such as to test~~ a ~~new feature or bug fix~~, you ~~must compile~~ the ~~VueJS + Ant design files~~ and ~~place~~ have the ~~CloudStack management server serve these files~~. ~~This is accomplished by doing the following:~~		Building the UI is a straight forward process once you have all the necessary software dependencies in place. Once the UI is built, you can then install it on the management server and have it served instead of the 'stock' bundled UI.

	~~# On~~ a server with npm installed~~, clone~~ the CloudStack ~~git~~ repo and ~~checkout~~ the ~~branch with~~ the ~~UI fix~~/~~feature~~		==== Building the UI ====
	~~# Navigate to <code>cloudstack~~/ui/</code>		You will need a server with npm installed along with a copy of the CloudStack repo. The simplest way I've found to accomplish this is to create the following Docker image and then run the build process within the container.
	# ~~Run <code>~~npm ~~install<~~/~~code>~~		{{Highlight
	~~# Run <code>~~npm run build~~</code>~~		\| code = FROM rockylinux/rockylinux:8.10

			RUN curl -sL https://rpm.nodesource.com/setup_16.x {{!}} bash -
			RUN yum -y install nodejs
			\| lang = dockerfile
			}}
			Clone the CloudStack repo (I placed it in /tmp in this example) and then run the following:
			{{Highlight
			\| code = # docker build -t cs-ui-builder --progress=plain --no-cache .

			# docker run --rm -ti \
			-v /tmp/cloudstack:/cloudstack \
			-v /tmp/npm:/root/.npm cs-ui-builder \
			bash -c "cd /cloudstack/ui; npm install; npm run build"
			\| lang = terminal
			}}
			We make a volume for the .npm directory to help speed up subsequent builds as the npm dependencies are cached, but this is entirely optional.

			==== Installing the UI ====
	# Copy the <code>dist/</code> directory to <code>/usr/share/cloudstack-management/webapp/</code>		# Copy the <code>dist/</code> directory to <code>/usr/share/cloudstack-management/webapp/</code>
	# Edit <code>/etc/cloudstack/management/server.properties</code> and make sure that <code>webapp.dir</code> is set to: <code>webapp.dir=/usr/share/cloudstack-management/webapp</code>		# Edit <code>/etc/cloudstack/management/server.properties</code> and make sure that <code>webapp.dir</code> is set to: <code>webapp.dir=/usr/share/cloudstack-management/webapp</code>

Leo: /* Re-add existing KVM bare metal host */

2025-06-18T15:56:54Z

Re-add existing KVM bare metal host

← Older revision		Revision as of 15:56, 18 June 2025
Line 609:		Line 609:
	#<code>rm -rf /var/lib/libvirt/qemu</code>		#<code>rm -rf /var/lib/libvirt/qemu</code>
	#You may need to edit <code>/etc/sysconfig/libvirtd</code> to not use the listen flag. This might prevent libvirtd (and subsequently cloudstack-agent) from starting.		#You may need to edit <code>/etc/sysconfig/libvirtd</code> to not use the listen flag. This might prevent libvirtd (and subsequently cloudstack-agent) from starting.
	#Edit <code>/etc/cloudstack/agent/agent.properties</code> and remove the keystore passphrase, any UUIDs ~~and GUIDs~~, cluster/pod/zone, and the host.		#Edit <code>/etc/cloudstack/agent/agent.properties</code> and remove the keystore passphrase, any UUIDs, cluster/pod/zone, and the host. You should keep the guid or regenerate it with uuidgen. You should also keep the public/private/guest network devices set.
	#Restart with <code>systemctl start cloudstack-agent</code> (libvirt should come up automatically as it's a dependency). Ensure that it comes up OK.		#Restart with <code>systemctl start cloudstack-agent</code> (libvirt should come up automatically as it's a dependency). Ensure that it comes up OK.

Leo: /* Upgrade CloudStack */

2024-01-23T22:36:50Z

Upgrade CloudStack

← Older revision		Revision as of 22:36, 23 January 2024
Line 752:		Line 752:

	=== Upgrade CloudStack ===		=== Upgrade CloudStack ===
	~~To upgrade~~ CloudStack ~~to the next major release, you should first update the management server~~, ~~then secondly~~ the ~~agents, and lastly, check if you need to~~ upgrade ~~any system VMs or virtual routers~~. ~~Ideally, you should have the same CloudStack version across the entire cluster~~ to ~~avoid any weird issues due to mismatched versions~~. ~~If you do have mismatched versions~~, ~~you might find yourself being unable to start a virtual router properly because the virtual router version is higher than the underlying CloudStack agent doing the orchestration~~.		Before upgrading CloudStack, review the upgrade instructions from CloudStack's documentation. For 4.17 to 4.18, see: https://docs.cloudstack.apache.org/en/4.18.0.0/upgrading/upgrade/upgrade-4.17.html

	~~==== Updating~~ the ~~management server ====~~		In a nutshell, upgrading CloudStack for KVM hosts requires the following steps:
	~~Here are the general~~ steps ~~to upgrade the management server.~~

	# ~~Make sure you back up your CloudStack database before you begin~~. ~~This is necessary if you wish to rollback the update because new versions typically also require a database migration which cannot~~ be ~~reverted easily and older CloudStack versions will refuse to start with an updated database.~~		# Before upgrading, load the next systemvm template image. System templates are available from: http://download.cloudstack.org/systemvm/. The systemvm template for KVM should be named something like: <code>systemvm-kvm-4.18.0</code>. When adding the template, specify qcow2 as its format.
	~~# Stop the management server~~: <code>~~systemctl stop cloudstack~~-~~management~~</code>		# Backup your CloudStack and usage database. {{Highlight
	# ~~Apply the update: <~~code>~~yum~~ -~~y update cloudstack\*</code~~>		\| code = $ mysqldump -u root -p -R cloud > cloud-backup_$(date +%Y-%m-%d-%H%M%S)
	# Manually unmount any CloudStack mounts. I had to do when I upgraded from 4.16 to 4.17 since it prevented CloudStack from starting~~: <~~code>umount /var/cloudstack/mnt/*~~</code>~~		$ mysqldump -u root -p cloud_usage > cloud_usage-backup_$(date +%Y-%m-%d-%H%M%S)
	# Ensure that the appropriate system VM template exists. This is a problem if you built a custom RPM package and didn't include it. If it's missing, download the appropriate version from http://download.cloudstack.org/systemvm. <code>ls -~~al /usr/share/~~cloudstack~~-management/templates/systemvm/</code>.~~		\| lang = terminal
	# ~~Restart the management server: <code>~~systemctl start cloudstack-management~~</code>,~~		}}
	# Watch the ~~CloudStack~~ logs ~~for any issues preventing a startup:~~ <code>tail -f /var/log/cloud/management/*log</code>		# If you have outstanding system packages to upgrade, do so now (excluding CloudStack packages) and reboot.
			# Stop the CloudStack Management server. Manually unmount any CloudStack mounts. I had to do when I upgraded from 4.16 to 4.17 since it prevented CloudStack from starting. Upgrade the cloudstack-management and cloudstack-common packages. Restart CloudStack. {{Highlight
			\| code = # systemctl stop cloudstack-management
			# umount /var/cloudstack/mnt/*
			# yum -y update cloudstack\*
			# systemctl start cloudstack-management
			\| lang = terminal
			}}
			# Ensure things are running. Watch the logs in <code>tail -f /var/log/cloud/management/*log</code>. Verify that the management server can still communicate with the hosts.
			# For each CloudStack host, drain it of hosts, stop the cloudstack-agent service, do a full upgrade, reboot. {{Highlight
			\| code = # systemctl stop cloudstack-agent
			# yum -y update cloudstack-agent

	~~==== Updating~~ the ~~CloudStack agents ====~~		## Restart the service or reboot just to make sure the host can come up by itself
	~~Updating the agent is straight forward and shouldn't require you~~ to ~~drain VMs from~~ the ~~node.~~		## reboot
			# systemctl restart cloudstack-agent
	# ~~Stop CloudStack agent: <code>systemctl stop cloudstack-agent</code>~~		\| lang = terminal
	# ~~Update the CloudStack agent: <code>yum -y update cloudstack-agent</code>~~		}}
	# ~~Restart the CloudStack agent: <code>~~systemctl restart cloudstack-agent~~</code>~~

	==== Updating the system VMs ====		==== Updating the system VMs ====

Leo: /* Rebuilding UI */

2024-01-23T18:07:39Z

Rebuilding UI

← Older revision		Revision as of 18:07, 23 January 2024
Line 650:		Line 650:

	=== Rebuilding UI ===		=== Rebuilding UI ===
	To rebuild the CloudStack ~~management console~~ UI (such as to test a new feature or bug fix), do the following:		Instructions to rebuild the UI are available on the [https://github.com/apache/cloudstack/tree/main/ui README file under the ui directory.]

			In summary, to rebuild the CloudStack UI, such as to test a new feature or bug fix, you must compile the VueJS + Ant design files and place have the CloudStack management server serve these files. This is accomplished by doing the following:

	# On a server with npm installed, clone the CloudStack git repo and checkout the branch with the UI fix/feature		# On a server with npm installed, clone the CloudStack git repo and checkout the branch with the UI fix/feature
Line 657:		Line 659:
	# Run <code>npm run build</code>		# Run <code>npm run build</code>
	# Copy the <code>dist/</code> directory to <code>/usr/share/cloudstack-management/webapp/</code>		# Copy the <code>dist/</code> directory to <code>/usr/share/cloudstack-management/webapp/</code>
			# Edit <code>/etc/cloudstack/management/server.properties</code> and make sure that <code>webapp.dir</code> is set to: <code>webapp.dir=/usr/share/cloudstack-management/webapp</code>

			Restart the CloudStack management service and then reload the console page. Ensure that the vue app isn't cached.

			==== Installing and using CloudStack's prebuilt UI ====
			The cloudstack-ui package contains the prebuilt CloudStack UI. The location of the UI files are placed under <code>/usr/share/cloudstack-ui</code>.

			As of CloudStack 4.18, when I tried using this prebuilt package, I had to do the following things:

	~~Reload the console page~~. ~~Ensure that~~ the ~~vue app isn~~'t ~~cached~~.		* Edit <code>/etc/cloudstack/management/server.properties</code> and set <code>webapp.dir=/usr/share/cloudstack-ui</code>
			* In <code>/usr/share/cloudstack-ui</code>, run: <code>find . -type d -exec chmod -v o+x {} \;</code> because the directories aren't executable by the 'cloud' user.
			* Create /usr/share/cloudstack-ui/WEB-INF and place this [https://github.com/apache/cloudstack/blob/main/client/src/main/webapp/WEB-INF/web.xml web.xml file] within. Otherwise, requests to the API break.

	===Usage server===		===Usage server===

Leo: /* CloudStack Agent */

2023-11-30T18:54:51Z

CloudStack Agent

← Older revision		Revision as of 18:54, 30 November 2023
Line 283:		Line 283:
	}}		}}

	Configure <code>qemu</code> and <code>libvirtd</code>.{{Highlight		Configure <code>qemu</code> and <code>libvirtd</code>. If you need some starting configs, try the following:{{Highlight
	\| code = ## edit /etc/libvirt/qemu.conf		\| code = ## edit /etc/libvirt/qemu.conf
	vnc_listen=0.0.0.0		vnc_listen=0.0.0.0

	## edit /etc/libvirt/libvirtd.conf		## edit /etc/libvirt/libvirtd.conf
	~~listen_tls = 0~~
	listen_tcp = 1		listen_tcp = 1
	tcp_port = "16509"		tcp_port = "16509"
			listen_tls = 0
			tls_port = "16514"
	auth_tcp = "none"		auth_tcp = "none"
	mdns_adv = 0		mdns_adv = 0
Line 301:		Line 302:
	LIBVIRTD_ARGS="--listen"		LIBVIRTD_ARGS="--listen"
	\| lang = terminal		\| lang = terminal
	}}Start the CloudStack agent. ~~The CloudStack~~ agent should also ~~automatically bring up libvirtd~~ (it's a service dependency).		}}Start the CloudStack agent. Verify that the cloudstack-agent is running. At this point libvirtd should also running (it's a service dependency).

			With the agent running on the node, you should now be able to add the node to the CloudStack cluster. This process will rewrite the <code>libvirtd.conf</code> file and it should set <code>listen_tcp=0</code> and <code>listen_tls=1</code> for you (so that libvirt traffic such as migrations are done via TLS rather than basic TCP).

	====Allow sudo access====		====Allow sudo access====

Leo at 18:36, 30 November 2023

2023-11-30T18:36:47Z

← Older revision		Revision as of 18:36, 30 November 2023
Line 1,138:		Line 1,138:
	Using this network offering, I was able to create a shared network in the advanced networking zone that has a NAT service which is visible to all accounts. The only issue with this approach is that there isn't a way to create a port forwarding for a specific VM because the account that owns this network is 'system'.		Using this network offering, I was able to create a shared network in the advanced networking zone that has a NAT service which is visible to all accounts. The only issue with this approach is that there isn't a way to create a port forwarding for a specific VM because the account that owns this network is 'system'.

			=== Libvirtd can't start due to expired certificate ===
			For some reason, the host stopped renewing agent certs with the management server. As a result, libvirtd will not restart. I only noticed this after rebooting an affected node after migrating all the VMs off the system.
			{{Highlight
			\| code = # libvirtd -l
			2023-11-30 18:16:25.116+0000: 39448: info : libvirt version: 8.0.0, package: 22.module+el8.9.0+1405+b6048078 (infrastructure@rockylinux.org, 2023-07-31-18:01:38, )
			2023-11-30 18:16:25.116+0000: 39448: info : hostname: cs1
			2023-11-30 18:16:25.116+0000: 39448: error : virNetTLSContextCheckCertTimes:142 : The server certificate /etc/pki/libvirt/servercert.pem has expired
			\| lang = terminal
			}}
			Note that the certficate file <code>/etc/pki/libvirt/servercert.pem</code> symlinks to <code>/etc/cloudstack/agent/cloud.crt</code>. The certificates and private keys under <code>/etc/cloudstack/agent/cloud*</code> are generated by the CloudStack management server and then sent to and saved by the agent<ref>Certificate saved by the agent: https://github.com/apache/cloudstack/blob/cb62ce67671699fa01564b3b4b0d3d83eb3d5acb/agent/src/main/java/com/cloud/agent/Agent.java#L671</ref>.

			Since the node is already out of service, the easiest fix here is to [[CloudStack#Re-add existing KVM bare metal host\|re-add this KVM bare metal host]] back into CloudStack again.

	==Open-ended questions==		==Open-ended questions==

Leo: /* Renewing SSL certificate */

2023-07-14T22:13:42Z

Renewing SSL certificate

← Older revision		Revision as of 22:13, 14 July 2023
Line 574:		Line 574:
	## Create keystore and import your certificate into it.		## Create keystore and import your certificate into it.
	# openssl pkcs12 -in combined.crt -export -out combined.pkcs12		# openssl pkcs12 -in combined.crt -export -out combined.pkcs12
	# keytool -importkeystore -srckeystore combined.pkcs12 -srcstoretype PKCS12 -destkeystore ~~/etc/cloudstack/management/combined~~.pkcs12 -deststoretype pkcs12		# keytool -importkeystore -srckeystore combined.pkcs12 -srcstoretype PKCS12 -destkeystore ssl_keystore.pkcs12 -deststoretype pkcs12

	## Move the keystore over the existing one as defined in the server config. You may want to backup the old one just in case.		## Move the keystore over the existing one as defined in the server config. You may want to backup the old one just in case.
	# mv ~~combined~~.pkcs12 /etc/cloudstack/management/ssl_keystore.pkcs12		# mv /etc/cloudstack/management/ssl_keystore.pkcs12 /etc/cloudstack/management/ssl_keystore.pkcs12-org
			# mv ssl_keystore.pkcs12 /etc/cloudstack/management/ssl_keystore.pkcs12

	## Convert your key to pkcs8 if you haven't already done so. Use the same password for both commands.		## Convert your key to pkcs8 if you haven't already done so. Use the same password for both commands.

Leo: /* Renewing SSL certificate */

2023-07-14T22:12:40Z

Renewing SSL certificate

← Older revision		Revision as of 22:12, 14 July 2023
Line 569:		Line 569:
	## Note the keystore location that's defined in your configs		## Note the keystore location that's defined in your configs
	# grep https.keystore /etc/cloudstack/management/server.properties		# grep https.keystore /etc/cloudstack/management/server.properties
			https.keystore=/etc/cloudstack/management/ssl_keystore.pkcs12
			https.keystore.password=xxxxxxx

	## Create keystore and import your certificate into it.		## Create keystore and import your certificate into it.
Line 577:		Line 579:
	# mv combined.pkcs12 /etc/cloudstack/management/ssl_keystore.pkcs12		# mv combined.pkcs12 /etc/cloudstack/management/ssl_keystore.pkcs12

	## Convert your key to pkcs8 if you haven't already done so. Use ~~'password' as~~ the password		## Convert your key to pkcs8 if you haven't already done so. Use the same password for both commands.
	# openssl pkcs8 -topk8 -in server.key -out server.pkcs8.key-encrypted		# openssl pkcs8 -topk8 -in server.key -out server.pkcs8.key-encrypted
	# openssl pkcs8 -in server.pkcs8.key-encrypted -out server.pkcs8.key		# openssl pkcs8 -in server.pkcs8.key-encrypted -out server.pkcs8.key

Leo: /* Preparing your SSL certificates */

2023-07-14T22:11:17Z

Preparing your SSL certificates

@@ Line 559: / Line 559: @@
 ==== Enable HTTPS on the console proxy ====
 If you enable SSL on the management console, you will also need to enable SSL for the console proxies for the VNC web sockets to work properly. If your management console certificate (from the previous sections) contain a Subject Alternative Name (SAN) or is a wildcard certificate that includes your console proxy's DNS name, SSL for the console proxy should be working. If your certificates do not include the console proxy's DNS name, you will need to obtain another SSL certificate and add it to the SSL keystore and upload it to CloudStack using the same instructions above.
 == Tasks ==

Leo: /* Change guest VM CPU flags */

2023-03-23T17:40:03Z

Change guest VM CPU flags

← Older revision		Revision as of 17:40, 23 March 2023
Line 826:		Line 826:

	* '''custom''': This is the default and it defaults to the x86_qemu64 feature set defined in <code>/usr/share/libvirt/cpu_map/x86_qemu64.xml</code>. You may select a different CPU map by specifying <code>guest.cpu.model</code>.		* '''custom''': This is the default and it defaults to the x86_qemu64 feature set defined in <code>/usr/share/libvirt/cpu_map/x86_qemu64.xml</code>. You may select a different CPU map by specifying <code>guest.cpu.model</code>.
	* '''host-model''': Uses a CPU model compatible with your host. Most feature flags are available.		* '''host-model''': Uses a CPU model compatible with your host. Most feature flags are available. Guest CPUs will identify itself as a generic CPU of that family such as <code>Intel Xeon Processor (Icelake)</code> (note the lack of '(R)' after Intel and Xeon brands and no specific CPU model number).
	* '''host-passthrough''': Use CPU passthrough; feature flags match exactly. Migrations only work with matching CPUs and may still fail when using this mode.		* '''host-passthrough''': Use CPU passthrough; feature flags match exactly. Migrations only work with matching CPUs and may still fail when using this mode. Guest CPUs will identify itself as the underlying CPU in that hypervisor (such as <code>Intel(R) Xeon(R) Gold 5320 CPU @ 2.20GHz</code>)

	For CloudStack clusters with identical CPUs, it's recommended to use host-model. I've tried using host-passthrough on matching hosts using a Intel Xeon Silver 4316 and migrations sometimes fail and the VM requires a reset to be brought back up.		For CloudStack clusters with identical CPUs, it's recommended to use host-model. I've tried using host-passthrough on matching hosts using a Intel Xeon Silver 4316 and migrations sometimes fail and the VM requires a reset to be brought back up.