Apache CloudStack Ubuntu/KVM Guide for x86_64 and ARM64

This guide covers the installation and build of Apache CloudStack IaaS cloud on Ubuntu using KVM, for both x86_64 and ARM64 architectures.

If you’re new to Apache CloudStack, here a video for you: CloudStack 101: The Best Way to Build Your Private Cloud

Getting Started

First install Ubuntu 24.04/later LTS on your x86_64 or ARM64 system.

On ARM64 hosts, ensure 64-bit mode and GIC may need to be enabled in some systems, such as on Raspberry Pi 4/5:

/boot/firmware/config.txt

[all]
arm_64bit=1
enable_gic=1

# NVME speed config
dtparam=nvme
dtparam=pciex1_1=gen3
dtparam=pciex1_gen=3

# Current config
usb_max_current_enable=1
max_usb_current=1

dtoverlay=vc4-kms-v3d
disable_fw_kms_setup=1

Ensure that the universe repository is enabled in /etc/apt/sources.list.

Install basic packages:

    apt-get install openntpd openssh-server sudo vim htop tar

Ensure that KVM is available at /dev/kvm or by running kvm-ok:

    apt-get install cpu-checker

    #kvm-ok
    INFO: /dev/kvm exists
    KVM acceleration can be used

Optional: Intel based systems may benefit from CPU microcode install/update:

    apt-get install intel-microcode

Allow the root user for ssh access using password, fix /etc/ssh/sshd_config to set PermitRootLogin yes and restart ssh using systemctl restart ssh.

Change and remember the root password:

    passwd root

Network Setup

Next, setup host networking using Linux bridges that can handle CloudStack’s public, guest, management and storage traffic. For simplicity, a single bridge cloudbr0 can be used for all traffic types on the same physical network. Install bridge utilities using:

    apt-get install bridge-utils

This guide assumes that you’re in a 192.168.1.0/24 network which is a typical RFC1918 private network.

Ubuntu Netplan Config

Starting Ubuntu bionic, admins can use netplan to configure networking. The default installation creates a file at /etc/netplan/50-cloud-init.yaml that you should comment, and create a file at /etc/netplan/01-netcfg.yaml applying your network and interface/name specific changes:

/etc/netplan/01-netcfg.yaml

     network:
       version: 2
       renderer: networkd
       ethernets:
         eno1:
           dhcp4: false
           dhcp6: false
           optional: true
       bridges:
         cloudbr0:
           addresses: [192.168.1.10/24]
           routes:
            - to: default
              via: 192.168.1.1
           nameservers:
             addresses: [1.1.1.1, 8.8.8.8]
           interfaces: [eno1]
           dhcp4: false
           dhcp6: false
           parameters:
             stp: false
             forward-delay: 0

/etc/netplan/01-netcfg.yaml

  ethernets:
    eno1:
      match:
        macaddress: 00:01:2e:4f:f7:d0
      mtu: 1550
      dhcp4: false
      dhcp6: false
    enp3s0:
      mtu: 1550

Apply network config and reboot/re-ssh:

    netplan generate
    netplan apply

Repo Setup

Setup the repository on your Ubuntu host:

    mkdir -p /etc/apt/keyrings
    wget -O- http://packages.shapeblue.com/release.asc | gpg --dearmor | sudo tee /etc/apt/keyrings/cloudstack.gpg > /dev/null

    echo deb [signed-by=/etc/apt/keyrings/cloudstack.gpg] http://packages.shapeblue.com/cloudstack/upstream/debian/4.22 / > /etc/apt/sources.list.d/cloudstack.list
    apt-get update -y

This should be run on all hosts where CloudStack software packages (management server, usage server, agent etc) needs to be installed, esp if you’ve multiple hosts.

Management Server Setup

Install CloudStack management server and MySQL server: (run as root, once the CloudStack repository is setup)

    apt-get update -y
    apt-get install cloudstack-management mysql-server

Optionally, if you want usage server you can run:

    apt-get install cloudstack-usage

Database Setup

Make a note of the MySQL server’s root user password. Configure InnoDB settings in mysql server’s /etc/mysql/mysql.conf.d/mysqld.cnf:

    [mysqld]

    server_id = 1
    sql-mode="STRICT_TRANS_TABLES,NO_ENGINE_SUBSTITUTION,ERROR_FOR_DIVISION_BY_ZERO,NO_ZERO_DATE,NO_ZERO_IN_DATE,NO_ENGINE_SUBSTITUTION"
    innodb_rollback_on_timeout=1
    innodb_lock_wait_timeout=600
    max_connections=1000
    log-bin=mysql-bin
    binlog-format = 'ROW'

    #default-authentication-plugin=mysql_native_password

Restart MySQL server and setup database:

    systemctl restart mysql
    cloudstack-setup-databases cloud:cloud@localhost --deploy-as=root: -i 192.168.1.10

Storage Setup

Install NFS server:

    apt-get install nfs-kernel-server quota

Create exports:

    echo "/export  *(rw,async,no_root_squash,no_subtree_check)" > /etc/exports
    mkdir -p /export/primary /export/secondary
    exportfs -a

Configure and restart NFS server:

    sed -i -e 's/^RPCMOUNTDOPTS="--manage-gids"$/RPCMOUNTDOPTS="-p 892 --manage-gids"/g' /etc/default/nfs-kernel-server
    sed -i -e 's/^STATDOPTS=$/STATDOPTS="--port 662 --outgoing-port 2020"/g' /etc/default/nfs-common
    echo "NEED_STATD=yes" >> /etc/default/nfs-common
    sed -i -e 's/^RPCRQUOTADOPTS=$/RPCRQUOTADOPTS="-p 875"/g' /etc/default/quota
    service nfs-kernel-server restart

For ARM64 hosts, seed an appropriate arm64 based systemvm template manually from the management server for the specific CloudStack version, for example for 4.22:

wget http://download.cloudstack.org/systemvm/4.22/systemvmtemplate-4.22.0-aarch64-kvm.qcow2.bz2
/usr/share/cloudstack-common/scripts/storage/secondary/cloud-install-sys-tmplt \
          -m /export/secondary -f systemvmtemplate-4.22.0-aarch64-kvm.qcow2.bz2 -h kvm \
          -o localhost -r cloud -d cloud

Future, CloudStack releases are working towards automating installation and upgrades of ARM64 systemvmtemplates.

KVM Host Setup

Install KVM and CloudStack agent, configure libvirt:

    apt-get install qemu-kvm cloudstack-agent

Enable VNC for console proxy:

    sed -i -e 's/\#vnc_listen.*$/vnc_listen = "0.0.0.0"/g' /etc/libvirt/qemu.conf

On Ubuntu 22.04/24.04 or later, add LIBVIRTD_ARGS="--listen" to /etc/default/libvirtd:

    echo LIBVIRTD_ARGS=\"--listen\" >> /etc/default/libvirtd

The traditional socket/listen based configuration may not be supported with CloudStack, we can get the old behaviour using following:

    systemctl mask libvirtd.socket libvirtd-ro.socket libvirtd-admin.socket libvirtd-tls.socket libvirtd-tcp.socket
    systemctl restart libvirtd

Configure libvirt to connect to libvirtd and not to per-driver daemons, especially important on newer distros such as Ubuntu 24.04 (and EL9) by editing the /etc/libvirt/libvirt.conf and adding the following:

/etc/libvirt/libvirt.conf

      remote_mode="legacy"

Configure default libvirtd config:

    echo 'listen_tls=0' >> /etc/libvirt/libvirtd.conf
    echo 'listen_tcp=1' >> /etc/libvirt/libvirtd.conf
    echo 'tcp_port = "16509"' >> /etc/libvirt/libvirtd.conf
    echo 'mdns_adv = 0' >> /etc/libvirt/libvirtd.conf
    echo 'auth_tcp = "none"' >> /etc/libvirt/libvirtd.conf
    systemctl restart libvirtd

On ARM64 hosts, please ensure the following options are set in the /etc/cloudstack/agent/agent.properties: (you may need to check/re-add these for your KVM host, post zone deployment)

guest.cpu.arch=aarch64
guest.cpu.mode=host-passthrough
host.cpu.manual.speed.mhz=1500

The host.cpu.manual.speed.mhz must be configured because Linux on ARM64 isn’t able to report the correct CPU speed on some ARM64 hosts.

By default 1GB of host memory is reserved for agent/host usage, which you can change/reduce it using the following in the agent.properties file, for example:

host.reserved.mem.mb=800

On certain hosts where you may be running docker and other services, you may need to add the following in /etc/sysctl.conf and then run sysctl -p:

/etc/sysctl.conf

    net.bridge.bridge-nf-call-arptables = 0
    net.bridge.bridge-nf-call-iptables = 0

    sysctl -p

Optional: If you’re using a non-compliant server vendor or virtual environment, in some cases you may need to configure unique host ID for libvirt:

    apt-get install uuid
    UUID=$(uuid)
    echo host_uuid = \"$UUID\" >> /etc/libvirt/libvirtd.conf
    systemctl restart libvirtd

Firewall Setup

By default ufw may be disabled and this may not be required. If your system uses ufw (you can check using ufw status), you may run the following:

    ufw allow mysql
    ufw allow proto tcp from any to any port 22
    ufw allow proto tcp from any to any port 1798
    ufw allow proto tcp from any to any port 16509
    ufw allow proto tcp from any to any port 16514
    ufw allow proto tcp from any to any port 5900:6100
    ufw allow proto tcp from any to any port 49152:49216

Alternatively, you can configure firewall rules using iptables for your management network:

    # configure firewall rules to allow useful ports
    NETWORK=192.168.1.0/24
    iptables -A INPUT -s $NETWORK -m state --state NEW -p udp --dport 111 -j ACCEPT
    iptables -A INPUT -s $NETWORK -m state --state NEW -p tcp --dport 111 -j ACCEPT
    iptables -A INPUT -s $NETWORK -m state --state NEW -p tcp --dport 2049 -j ACCEPT
    iptables -A INPUT -s $NETWORK -m state --state NEW -p tcp --dport 32803 -j ACCEPT
    iptables -A INPUT -s $NETWORK -m state --state NEW -p udp --dport 32769 -j ACCEPT
    iptables -A INPUT -s $NETWORK -m state --state NEW -p tcp --dport 892 -j ACCEPT
    iptables -A INPUT -s $NETWORK -m state --state NEW -p tcp --dport 875 -j ACCEPT
    iptables -A INPUT -s $NETWORK -m state --state NEW -p tcp --dport 662 -j ACCEPT
    iptables -A INPUT -s $NETWORK -m state --state NEW -p tcp --dport 8250 -j ACCEPT
    iptables -A INPUT -s $NETWORK -m state --state NEW -p tcp --dport 8080 -j ACCEPT
    iptables -A INPUT -s $NETWORK -m state --state NEW -p tcp --dport 9090 -j ACCEPT
    iptables -A INPUT -s $NETWORK -m state --state NEW -p tcp --dport 16514 -j ACCEPT

    apt-get install iptables-persistent

You must check and disable apparmour:

    # Disable apparmour on libvirtd
    ln -s /etc/apparmor.d/usr.sbin.libvirtd /etc/apparmor.d/disable/
    ln -s /etc/apparmor.d/usr.lib.libvirt.virt-aa-helper /etc/apparmor.d/disable/
    apparmor_parser -R /etc/apparmor.d/usr.sbin.libvirtd
    apparmor_parser -R /etc/apparmor.d/usr.lib.libvirt.virt-aa-helper

Start Control Plane

Start your cloud management control plane by setting up the CloudStack management server:

    cloudstack-setup-management
    systemctl status cloudstack-management
    tail -f /var/log/cloudstack/management/management-server.log

After the CloudStack management server is up, log into the management server using the default credentials (username admin and password password) on the URL:

http://`192.168.1.10(i.e. the cloudbr0-IP)`:8080/client CloudStack Login Page

Deploy Your Private Cloud

The following is an example of how you can setup an advanced zone (without security groups) in a typical (home/private) 192.168.1.0/24 network.

Deploy DataCenter

Create Zone:

Go to Infrastructure > Zone and click on add zone button, select advanced zone and provide following configuration:
```
Name - any name
Public DNS 1 - 8.8.8.8
Internal DNS1 - 192.168.1.1
Hypervisor - KVM
```
Setup Network

Use the default, which is VLAN isolation method on a single physical nic (on the host) that will carry all traffic types (management, public, guest etc).

Note: If you’ve iproute2 installed and host’s physical NIC MTUs configured, you can used VXLAN as well.

Public traffic configuration:
```
Gateway - 192.168.1.1
Netmask - 255.255.255.0
VLAN/VNI - (leave blank for vlan://untagged or in case of VXLAN use vxlan://untagged)
Start IP - 192.168.1.20
End IP - 192.168.1.50
```

Pod Configuration:

Name - any name
Gateway - 192.168.1.1
Start/end reserved system IPs - 192.168.1.51 - 192.168.1.80

Guest traffic:
```
VLAN/VNI range: 700-900
```
Create a cluster with following:
```
Name - any name
Hypervisor - Choose KVM
```
Add your default/first host:
```
Hostname - 192.168.1.10
Username - root
Password - <password for root user, or leave blank to add by public key>
```
Note: root user ssh-access is disabled by default, please enable it. The recommended approach is to add the KVM host using ssh public-key based access, add the management server SSH public key which is usually at /var/cloudstack/management/.ssh/id_rsa.pub to the root user of the KVM host(s) at /root/.ssh/authorized_keys.

Add primary storage:

Name - any name
Scope - zone-wide
Protocol - NFS
Server - 192.168.1.10
Path - /export/primary

Add secondary storage:

Provider - NFS
Name - any name
Server - 192.168.1.10
Path - /export/secondary

Next, click Launch Zone which will perform following actions:

Create Zone
Create Physical networks:
  - Add various traffic types to the physical network
  - Update and enable the physical network
  - Configure, enable and update various network provider and elements such as the virtual network element
Create Pod
Configure public traffic
Configure guest traffic (vlan range for physical network)
Create Cluster
Add host
Create primary storage (also mounts it on the KVM host)
Create secondary storage
Complete zone creation

Enable Zone

Finally, confirm and enable the zone. Wait for the system VMs to come up under Infrastructure -> System VMs, then you can proceed using your IaaS cloud.

Register Templates

You can register publicly available cloud-init enabled guest templates such as:

Distro	x86_64	ARM64
Ubuntu 24.04	https://cloud-images.ubuntu.com/releases/24.04/release/ubuntu-24.04-server-cloudimg-amd64.img	https://cloud-images.ubuntu.com/releases/24.04/release/ubuntu-24.04-server-cloudimg-arm64.img
Ubuntu 22.04	https://cloud-images.ubuntu.com/releases/22.04/release/ubuntu-22.04-server-cloudimg-amd64.img	https://cloud-images.ubuntu.com/releases/22.04/release/ubuntu-22.04-server-cloudimg-arm64.img
Debian 12	https://cloud.debian.org/images/cloud/bookworm/latest/debian-12-genericcloud-amd64.qcow2	https://cloud.debian.org/images/cloud/bookworm/latest/debian-12-genericcloud-arm64.qcow2
AlmaLinux 9	https://repo.almalinux.org/almalinux/9/cloud/x86_64/images/AlmaLinux-9-GenericCloud-latest.x86_64.qcow2	https://repo.almalinux.org/almalinux/9/cloud/aarch64/images/AlmaLinux-9-GenericCloud-latest.aarch64.qcow2
OpenSUSE 15	https://download.opensuse.org/distribution/leap/15.5/appliances/openSUSE-Leap-15.5-Minimal-VM.x86_64-Cloud.qcow2	https://download.opensuse.org/distribution/leap/15.5/appliances/openSUSE-Leap-15.5-Minimal-VM.aarch64-Cloud.qcow2

Add Networks

With the configured and deployed zone, once System VMs are up and running, you can create isolated network or VPC for your account for the built-in admin account or any new user account you may created that can be later used to deploy instances.

For simplicity, you may also create a shared network in the unused IP address space of your home/private network. For example, create shared network using API/CLI or UI under Network > Guest Networks > Add Network > Shared using:

Physical Network - cloudbr0
VLAN/VNI - untagged
IPv4 Gateway - 192.168.1.1
IPv4 Netmask - 255.255.255.0
IPv4 Start IP - 192.168.1.81
IPv4 End IP - 192.168.1.100
DNS1 - 192.168.1.1
DNS2 - 1.1.1.1

Deploying Instance

In the UI, an instance can be created using the top navigation “Create” button or via Compute > Instances > Add Instance where you can specify a minimum of:

Zone - select the zone
Arch - select your arch (x86_64 or ARM64)
Image - select the registed template
Compute Offering - select the offering (new ones can be created later on)
Networks - select one or more network where you want to deploy this instance
SSH key pairs - select one or more ssh public key pairs that you may have added
Name - (optional) name of the instance

… and hit Launch Instance to create your instance!

Integrations

CloudStack has several integrations, the following is a list of some of the popular or official ones:

CLI: https://github.com/apache/cloudstack-cloudmonkey
Terraform Provider: https://github.com/apache/cloudstack-terraform-provider
Ansible: https://github.com/ngine-io/ansible-collection-cloudstack
Kubernetes integration: https://github.com/apache/cloudstack-kubernetes-provider
Kubernetes Cluster API Provider: https://github.com/kubernetes-sigs/cluster-api-provider-cloudstack
Go SDK: https://github.com/apache/cloudstack-go (for developers)
Benchmarking tool: https://github.com/apache/cloudstack-csbench
More here: https://cloudstack.apache.org/integrations

Where to go next?

If you’re stuck on to something and have questions, then:

Join and ask on the CloudStack users@ mailing list: https://cloudstack.apache.org/mailing-lists
Start a discussion sharing the details of your problem: https://github.com/apache/cloudstack/discussions