K22: Virtual Machine (VM) Setup Instructions
For kernel development and experimentation in this class, you will be working with Linux inside a controlled environment on your own machine using a Linux VM.
This tutorial explains how to get a recent Ubuntu distribution and run a VM with it, as well as how to obtain, compile, install, and run a recent Linux kernel inside that VM. Finally, we will create and run our first kernel patch!
The steps detailed in this tutorial are a strong recommendation from us, based on what we used on our own machines and found to work reliably. Feel free to use and setup any VM of your liking, as long as (1) you can make it boot with a recent Ubuntu distribution, and most importantly (2) be able to obtain, edit, patch, compile, install, and run a patched (modified) kernel.
Option-1: Set up a VM with VMware (Windows and Ubuntu users)
In what follows in this section, we will be describing steps regarding using VMware Workstation as the hypervisor, and assume that you are running it on a Linux-based system.
Download and Install VMware
1) Create an account at Broadcom.com. (Your username is your email!)
2) Log in to Broadcom Support Portal.
3) To use the free edition of VMware Workstation Pro, go to Download VMware Workstation Pro (Free) page and download the latest release. (You have to download and save the .bundle file on your computer.)
On your local machine
4) Navigate the folder where the VMware Workstation bundle installer is located. (By default, it is at the Downloads directory.)
5) Make the file executable and run the corresponding installer for your system.
chmod +x VMware-Workstation-Full-####-####.architecture.bundle
sudo ./VMware-Workstation-Full-####-####.architecture.bundle
6) Follow the instructions displayed on the screen to complete the installation.
7) After the installation finishes, reboot your machine.
Download an Ubuntu ISO Image
We will work with Ubuntu 26.04.1 LTS.
- If your computer has an x86 CPU, download the ubuntu-26.04.1-desktop-amd64.iso.
- If your computer has an ARM CPU, download the ubuntu-26.04.1-desktop-arm64.iso.
Enable Hardware Virtualization
Visit your motherboard manufacturer’s website (or your laptop’s manufacturer’s website if you are using a laptop) and find the page for your specific device. In the manuals of the manufacturer you will find the way to enter the UEFI/BIOS menu settings of your device and edit them. Look for settings with names like Virtualization Support, VMX, Enable virtualization. It should be a boolean toggle (true/false or enabled/disabled) which you want to ensure is turned on (enabled, true).
Create a VM
Now we are ready to create a Linux VM.
1) Run VMware at the background with the command: /usr/bin/vmware &.
2) To create a new VM using the Ubuntu ISO image that you downloaded, go to the top-left corner, click File, and then select New Virtual Machine.
3) In the Virtual Machine Configuration window, select Typical (recommended).
4) In the Install operating system from: window, choose Use ISO image and provide the path to the ISO image you downloaded. Fill in the required information in the Personalize Linux section.
Warning: This password will be used to log into the VM, so make sure to remember it!
5) In the Virtual Machine Name window, leave everything as is, and click Next.
6) In the Disk Size section, choose a size of at least 32 GB and select Store virtual disk as a single file.
7) In the final section, click Customize Hardware and configure the following settings
-
CPU: 4 or more cores are highly recommended, if your host machine has at least 4 physical cores. -
RAM: 6 GB or more is highly recommended, if your host machine has at least 8 GB of RAM.
8) Click close and after the configuration click Finish to create your new VM.
Some notes for the Ubuntu installation:
-
What do you want to do with Ubuntu?: Install Ubuntu -
How would you like to install Ubuntu?: Interactive Installation -
What apps would you like to install to start with?: Default Selection -
Install recommended proprietary software?: No need to select anything -
How do you want to install Ubuntu?: Erase disk and install Ubuntu -
Encryption and file system: No encryption
Finally click Install.
After a while, hopefully your VM is ready! Make sure you can boot and log into it before proceeding to the next section.
Enable Clipboard Sharing (Host ↔ Guest)
To enable clipboard sharing between the host and the guest virtual machine, open a terminal in the VM and run:
sudo apt update && sudo apt install open-vm-tools open-vm-tools-desktop
sudo systemctl enable --now open-vm-tools
Afterwards, log out of your session and log back in. Clipboard sharing between the host and the guest should now be enabled.
Kernel Compilation & Installation
Warning: All the steps of this sub-section must be run inside the VM!
Preparation
Boot up the VM and from a shell inside the VM execute the following commands to install the required dependencies for building the kernel:
sudo apt update
sudo apt install build-essential git bc python3 bison flex rsync libelf-dev libncurses-dev dwarves libdw-dev gawk libssl-dev openssh-server htop universal-ctags python3-ply python3-git
Next, clone the source code for Linux kernel v7.2
git clone --depth=1 --branch v7.2 https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git linux-7.2
cd linux-7.2
Warning: all subsequent commands from this guide should be run from the root directory of the kernel source!
Read the first lines of the Makefile to verify you downloaded the correct kernel version:
head -n5 Makefile
# SPDX-License-Identifier: GPL-2.0
VERSION = 7
PATCHLEVEL = 2
SUBLEVEL = 0
EXTRAVERSION =
Kernel Configuration
Before building the kernel, you must configure it by selecting the features, drivers and other options that will be included in the compiled kernel. This configuration is stored in the .config file in the root directory of the kernel source.
We will remove any existing configuration files and create our own based on the configuration file of the kernel currently in use.
Warning: it would be a good idea to run this step using a stable stock kernel, such as the one provided by default by your distribution, to ensure you proceed with a proper configuration.
make mrproper
make olddefconfig
A .config file should now be visible in the current directory. This file holds kernel build options, each set to a specific value. You can view its contents with cat .config. There are two main approaches to edit the configuration file:
- Run
make menuconfigto use an interactive menu, or - Use the
scripts/configtool:
scripts/config --set-str <option> <value> # set an option
scripts/config --state <option> # read an option
Make the following changes to your configuration file:
scripts/config --set-str CONFIG_LOCALVERSION "-k22"
scripts/config --set-str SYSTEM_TRUSTED_KEYS ""
scripts/config --set-str SYSTEM_REVOCATION_KEYS ""
The first command gives your kernel a custom name to distinguish it from other installed kernels. You should expect your custom kernel to have a name similar to 7.2-k22. The lattercommands instruct the kernel not to enforce built-in signature verification of modules/firmware against specific trusted keys.
Finally, run the following command to disable unused kernel modules from your configuration file, reducing kernel compilation time:
yes '' | make localmodconfig
Kernel Build
Warning: run this step as non-root user!
Build the kernel using all cores allocated to the VM with:
make -j`nproc`
Depending on the amount of available cores and the capabilities of your machine, this step may take a while.
Kernel Installation
Install your custom kernel and kernel modules with:
sudo make modules_install && sudo make install
If all goes well, you must be able to verify that the following 3 files are in /boot:
ls /boot | grep "k22"
initrd.img-7.2-k22
System.map-7.2-k22
vmlinuz-7.2-k22
Boot the Custom Kernel
Before rebooting the machine, we need to modify the bootloader’s (GRUB) configuration file to allow us to select the kernel we want to boot with.
Open /etc/default/grub as root with a text editor and:
- Replace
GRUB_TIMEOUT_STYLE=hiddenwithGRUB_TIMEOUT_STYLE=menu - Replace
GRUB_TIMEOUT=0withGRUB_TIMEOUT=15
Save and close the file and then run:
sudo update-grub
Reboot the machine and from the GRUB menu select Advanced Options for Ubuntu . Choose the kernel post-fixed with -k22. You can always select the Ubuntu option from the GRUB menu to boot with the stock kernel, which is recommended for development.
Once the VM boots up, verify you are running your custom kernel with:
uname -r
7.2-k22
Option-2: Set up a VM with KVM/QEMU (Ubuntu users)
This setup option is an alternative for those who wish to obtain access to tools like a debugger, and it can only be used if your host machine is GNU/Linux.
Enable Hardware Virtualization
We need to enable hardware level virtualization support via the motherboard UEFI/BIOS firmware settings.
Visit your motherboard manufacturer’s website (or your laptop’s manufacturer’s website if you are using a laptop) and find the page for your specific device. In the manuals of the manufacturer you will find the way to enter the UEFI/BIOS menu settings of your device and edit them. Look for settings with names like Virtualization Support, VMX, Enable virtualization. It should be a boolean toggle (true/false or enabled/disabled) which you want to ensure is turned on (enabled, true).
You can find out if you correctly enabled virtualization by running this command on your host OS and making sure the output is non-zero:
grep -E -c '(vmx|svm)' /proc/cpuinfo
Enable the KVM Module on your Host Kernel
To check if the KVM kernel module on the host OS is properly installed, run the following command:
lsmod | grep kvm
If the output looks like this, (perhaps with kvm_amd instead of kvm_intel) you can skip the rest of this step.
kvm_intel 286720 0
kvm 667648 1 kvm_intel
- If you have an Intel CPU and root access to the host OS, you should run the following command to enable the KVM module:
sudo modprobe kvm kvm_intel - If you have an AMD CPU and root access to the host OS, you should run the following command to enable the KVM module:
sudo modprobe kvm kvm_amd
After you are done with this, rerun the command lsmod | grep kvm and ensure the output shows the kvm module and one of { kvm_intel, kvm_amd }. If it does, move to the next step. If not, please contact us so we can help you.
Install QEMU
Unlike the normal setup, where we use VMware, we need to install the virtual machine environment we will be using.
We will be installing QEMU, which has builtin support for KVM and debugging. QEMU is commonly used among kernel developers as it can be quickly launched and has near native performance allowing for quick iteration.
We highly encourage exploring these tools outside of what we explain in here, as they are used very often in the industry and having extensive knowledge of them can significantly enhance the OS level development experience.
If your host is on Ubuntu, you can install the above software by running:
sudo apt install qemu-system-x86 cpu-checker
Run the following command and ensure that the output says everything is fine. Otherwise, you either made a mistake in the previous steps. Retry them. If even after that your setup does not work, contact us on Piazza so we may help you.
kvm-ok
Kernel Configuration & Installation
A big benefit of this tutorial is that kernel compilation will be done in the host computer instead of inside the virtual machine which will allow us to save some time during compilations.
Download an Ubuntu ISO Image
We will work with Ubuntu 26.04.1 LTS
- If your computer has an x86 CPU, download the ubuntu-26.04.1-desktop-amd64.iso.
- If your computer has an ARM CPU, download the ubuntu-26.04.1-desktop-arm64.iso.
Create a Virtual Disk
We will create our virtual disk, which, we will be using throughout the rest of the tutorial.
In a directory that you can easily recall (do NOT use the linux kernel directory as you may want to clean it later), run the following command
qemu-img create -f qcow2 rootdisk.qcow2 32G # replace 32G with however much storage you want
We recommend using at least 20G, and 32G is probably as much as you will need. (We have run into problems with running out of disk space, and thought something is wrong with our kernel build. Thus, always keep in mind the amount of storage you have allocated.)
Boot the Custom Kernel on the VM
We will use the virtual disk created during the previous step to run our virtual machine in TTY mode. That means we will only get console output and have no access to UI.
This is a good format for quick development iteration and testing.
Follow the instructions from Option-1 above to compile a fresh copy of the kernel on your host machine. Then, in the root directory of the kernel, run the following command, substituting any placeholders for what you want to use:
qemu-system-x86_64 -m 4096 \
-drive format=qcow2,file=./rootdisk.qcow2 \
-enable-kvm \
-smp 8 \
-audio driver=pa,model=virtio \
-cdrom <path-to-your-ISO> # substitute the path to your ubuntu ISO image
This should now start outputting kernel boot log and finally, bring you to the Ubuntu installation wizard.
Install a Fresh OS into the VM
In this step you will need to follow the steps the Ubuntu installer has in order to install Ubuntu in the virtual machine. For more information, see this.
At this point, you should now have a fully functioning virtual machine running Ubuntu 26.04.1 LTS.
Boot a Custom Kernel
To boot a custom kernel build in our VM, what we need to do is go to the root path of the linux kernel clone and from there run the following command:
qemu-system-x86_64 -kernel ./arch/x86/boot/bzImage \
-append "nokaslr console=ttyS0 root=/dev/sda2" \
-nographic \
-m 4096 \
-drive format=qcow2,file=<path-to-your-virtual-disk> \
-enable-kvm \
-smp 8
You should now see TTY output in the terminal, showing kernel logs and then be prompted for a login user and password. Congratulations, you have now run your custom kernel build in a virtual machine!
Debugging
With this setup, we also have the ability to connect to our virtual machine with a GDB session and debug the kernel. To do that, make sure you have gdb installed.
We also want to make sure our kernel is configured with debugging features enabled. There a bunch of them you can take a look at. For now we will need the following:
./scripts/config --enable CONFIG_GDB_SCRIPTS # enabled GDB scripts
./scripts/config --enable CONFIG_DEBUG_INFO # compile with debug info
./scripts/config --enable CONFIG_DEBUG_INFO # compile with debug info
./scripts/config --enable CONFIG_DEBUG_INFO_DWARF_TOOLCHAIN_DEFAULT # compile with debug info
./scripts/config --disable CONFIG_DEBUG_INFO_REDUCED # compile with full debug info
./scripts/config --enable CONFIG_FRAME_POINTER # keep frame pointer info
./scripts/config --disable CONFIG_RANDOMIZE_BASE # allow for disabling kernel address space layout randomization (kaslr)
We then need to create the kernel GDB scripts. To do that, run the following command:
make scripts_gdb
echo "add-auto-load-safe-path /path/to/linux-build" > ~/.gdbinit # replace the placeholder path with the path to your linux build
Then just start you virtual machine, making sure to add the -s and -S flags.
qemu-system-x86_64 -kernel ./arch/x86/boot/bzImage \
-append "nokaslr console=ttyS0 root=/dev/sda2" \
-nographic \
-m 4096 \
-drive format=qcow2,file=<path-to-your-virtual-disk> \
-enable-kvm \
-smp 8 \
-s \
-S # stop and wait for GDB connection
Then, start a GDB instance in a separate terminal. To do that, go to your kernel root folder and run
gdb ./vmlinux
A gdb terminal will now open. In that terminal, type
(gdb) target remote :1234
This should now connect you to kernel instance running in the virtual machine. You can make sure that everything is correct, by running
(gdb) continue
The virtual machine should now start booting. For more information regarding how to use GDB to debug a program, look here.
GDB is going to be an exceptionally useful tool throughout your kernel development journey. It can save a lot of time and help understand a lot of conditions. There are numerous other tools one can use to make kernel development and debugging easier but we won’t go into too much detail regarding those. Feel free to explore and use anythingyou deem useful.
Mounting your Virtual Disk
In some cases, like when installing modules or headers from a custom kernel build, it’s necessary for the host to be able to access the filesystem in the virtual disk. To do that, we will use an NBD (network block device).
First we need to enable the NBD module on the host. To do that, run:
sudo modprobe nbd
We then need to connect our virtual disk to an NBD device. We will use the qemu-nbd tool to do that. Run:
sudo qemu-nbd -c /dev/nbd0 <path-to-your-virtual-disk>
You can now mount the partitions of your virtual disk to your host:
sudo mount /dev/nbd0p2 /mnt # mount the second partition of the virtual disk
To install headers to your VM, you need to run the headers_install rule AND modify the path of the installation as shown below:
make headers_install INSTALL_HDR_PATH=/mnt/usr
You can install modules to your VM in a similar way, by running:
make modules_install INSTALL_MOD_PATH=/mnt
You now need to unmount your nbd device and then disconnect it from the virtual disk (you VM cannot boot with its disk mounted on the host):
sudo umount /dev/nbd0p2
sudo qemu-nbd -d /dev/nbd0
You can now safely boot back into your VM and the new headers as well as the new modules should be properly installed.
Option-3: Set up a VM with UTM (MacOS users)
Finally, this option covers the setup for MacOS users, including a debugger setup since it will come in handy for most of the assignments.
Install UTM Virtual Machine software
You can install UTM with brew (recommended) or from its website.
For installation using brew, just type in you terminal.
brew install utm
Do not obtain the framework from the App Store, as it is not free there for technical reasons. The versions that can be installed with the recommended methods are more than complete and fit for the purposes of this course.
Download an Ubuntu ISO Image
We will work with Ubuntu 26.04.1 LTS
- If your computer has an x86 CPU, download the ubuntu-26.04.1-desktop-amd64.iso.
- If your computer has an ARM CPU, download the ubuntu-26.04.1-desktop-arm64.iso.
VM Setup
Once downloaded, open UTM. On the top, you will see a “+” button, press it and choose Virtualize. Emulation is not forbidden but neither necessary. If you choose to emulate, make sure you have the ISO for the architecture of your choosing.
When asking for an Operating System, choose “Linux”. You are recommended to provide at least:
4 CPU Cores6 GBs of RAM
For Display Output leave as default.
On the next page, choose:
- NOT to use
Apple Virtualization - Boot from
ISOimage - at
PathselectBrowseand choose the recently downloaded image which should be in yourDownloadsfolder
Provide at least 32 GB of disk space, but more is recommended. You also do not need to choose to share a directory, unless you want to.
Lastly, name your VM, save it and you are good to go!
Booting and Configuration
Once you boot your VM you will have to install some packages, the kernel directory and configure it. Follow the instructions from this part of the tutorial.
GDB setup
Sometimes, when things don’t go as planned (which will happen very often), it might will be hard to troubleshoot and debug your kernel. This is why we recommend setting up your VM to work with GDB sessions. If you do not have gdb, you can install it via brew.
In UTM, select your vm from the side bar. On the top right choose to “Edit selected VM”. Under the QEMU option go to Arguments and scroll to the bottom. We will add two argumentsthat will tell QEMU we want to
-
-s: create a GDB session and -
-S: stop and wait for a GDB connection.
Choose New and add the flags above as two different commands. Note that the flag should be on the “dark bar” since it is not completely specified by utm. Click save and you aredone.
For the rest, follow this part of the tutorial
Our First Kernel Patch
We will now make our first Linux kernel patch to get used to the kernel development workflow. We need the kernel to print a Hello World message whenever the VM boots (i.e., when the kernel starts). This message should be displayed in the kernel message buffer, used by the kernel to display useful messages, such as:
- Boot Messages
- Kernel Errors and Warnings
- Runtime logs
To do this, we will edit the start_kernel() function in the init/main.c file. This is the C entry point of the Linux kernel and is called by the architecture-specific assembly start up code, after the CPU and basic hardware have been initialized. The function “ends” with a call to rest_init(), which performs the remaining initializations and calls into the scheduler. Therefore rest_init() should never return.
We will print a log message in start_kernel(), just before rest_init() is called. The logging function we will use is called printk(). It works similar to printf but its messages are all printed to the kernel message buffer.
Insert the following line of code just before the call to rest_init():
printk(KERN_INFO "I miss Alexis Delis\n");
Warning: this step should be performed in your custom kernel tree inside the VM!
Make sure you save the file, re-build and re-install the kernel with the following commands:
make -j`nproc`
sudo make modules_install && sudo make install
Note: You do not need to re-configure the kernel every time you make a change. However you should re-compile any changed files and re-install the modified kernel using the commandsabove whenever you are ready to test your changes.
Reboot the machine and once again select your custom -k22 kernel from the GRUB menu. The new kernel may show a -dirty suffix, which is normal. Run the following command to searchthe kernel logs for your custom message:
sudo dmesg | grep "Delis"
[ 0.148119] I miss Alexis Delis
Authors
K22 Fall ‘26 crew!