derek/gem5
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

util: Use a Multi-Machine Vagrantfile

Browse Source 
This patch removed the bespoke "vm_manager.sh" script in favor of a
Multi-Machine Vagrantfile.

With this the users needs to only change the variables in Vagrantfile
then use the standard `vagrant` commands to launch the VMs/Runners.

Change-Id: Ida5d2701319fd844c6a5b6fa7baf0c48b67db975
This commit is contained in:
Bobby R. Bruce
2023-10-04 18:45:53 -07:00
parent 0e5c6d9f50
commit 6571a54a65
3 changed files with 43 additions and 126 deletions
Download Patch File Download Diff File Expand all files Collapse all files

68
util/github-runners-vagrant/README.md
View File

@@ -1,6 +1,6 @@
# Setting up a Github Actions Runner with Vagrant # Setting up a Github Actions Runner with Vagrant
This directory provides a way to setup a Github Actions runner using Vagrant to host the runner in a Virtual machine. This directory provides a way to setup Github Actions runners using Vagrant to host them in Virtual machines.
This tutorial has been written with the assumption of running on a machine with Ubuntu 22.04. This tutorial has been written with the assumption of running on a machine with Ubuntu 22.04.
Setting up a runner on a different OS may require some changes. Setting up a runner on a different OS may require some changes.
@@ -21,13 +21,6 @@ sudo apt-get install qemu libvirt-daemon-system libvirt-clients ebtables dnsmasq
sudo apt purge vagrant-libvirt sudo apt purge vagrant-libvirt
``` ```
## Set up the Vagrant for the GitHub repository
First, generate a Personal Access Token, which you can create [here](https://github.com/settings/tokens)
Make sure to set admin permissions on this token, then replace the of `<PERSONAL ACCESS TOKEN>` in the Vagrantfile with your token.
Next, replace instances of `<GITHUB_ORG>` with your GitHub organization you wish to add this runner.
## Install Vagrant Plugins ## Install Vagrant Plugins
Once everything is set properly, set the `VAGRANT_HOME` environment variable to the directory in which the Vagrant files and other scripts are stored (i.e., the CWD). Once everything is set properly, set the `VAGRANT_HOME` environment variable to the directory in which the Vagrant files and other scripts are stored (i.e., the CWD).
@@ -45,62 +38,35 @@ vagrant plugin install vagrant-libvirt
vagrant plugin install vagrant-reload vagrant plugin install vagrant-reload
``` ```
## Creating the virtual machine ## Creating the virtual machines
The Vagrantfile in this directory defines a VM that can be used to create a GitHub Actions runner. The Vagrantfile in this directory defines the VMs that can be built and used to create a GitHub Actions runner.
It has 4-cores, 16GB of RAM, and 60GB of disk space. This standard VM has 4-cores, 16GB of RAM, and 60GB of disk space.
This is sufficient to both compile gem5 and run most simulations. This is sufficient to both compile gem5 and run most simulations.
Each VM on your host system must have a unique name. At the top of the Vagrantfile, there are a few variables that must be set prior to creating the VMs.
Give the VM to be created a unique name by setting the `<VM NAME>` variables in the Vagrantfile you wish to utilize.
Then run: * `NUM_RUNNERS`: The number of runners to create.
* `PERSONAL_ACCESS_TOKEN`: The GitHub personal access token to use.
You can generate a Personal Access Token [here](https://github.com/settings/tokens)
Make sure to set admin permissions on this token.
* `GITHUB_ORG`: The GitHub organization to add the runners to.
E.g., if the URL to your organization is https://github.com/orgs/gem5, then the variable should be set to "gem5".
* `HOSTNAME` : The hostname of the VM to be created (note, this will be appended with a number to create a unique hostname for each VM).
E.g., if set to `my-machine` and the number of runners set to `2`, two VMs will be created.
One called `my-machine-1` and the other `my-machine-2`.
When set simply run:
```sh ```sh
vagrant up --provider=libvirt vagrant up --provider=libvirt
``` ```
This should automatically create your machine, as well as configure and start up a Github Actions runner. This should automatically create your machines then configure and start up a Github Actions runner in each.
You can check the status of the runner here: https://github.com/organizations/{GITHUB_ORG}/settings/actions/runners You can check the status of the runner here: https://github.com/organizations/{GITHUB_ORG}/settings/actions/runners
If the runner ever shows as offline, you can rerun the `vagrant up --provider=libvirt` command to make sure everything is working properly. If the runner ever shows as offline, you can rerun the `vagrant up --provider=libvirt` command to make sure everything is working properly.
If you wish to create more than one runner you must edit the `<VM NAME>` in the Vagrant file.
## Helper scripts
The "vm_manager" script can be used to set up multiple builder and runner VMs.
To use this script simply modify the `NUM_RUNNERS` and `RUNNER_PREFIX` variables to the desired values.
Then run the script with:
```sh
./vm_manager.sh
```
This script will create any VMs that don't already exist and ensure those that do exists are running.
If you wish to destroy all the VMs you can run:
```sh
./vm_manager.sh destroy
```
**Note:** This script assumes "VAGRANT_HOME" is set to the CWD.
## Improving stability
Occasionally GitHub runner services, or VMs, go down. This is often silent and
usually only noticable from going to the GitHub repo page "settings" -> "actions" -> "runners" and observing the status.
When the VMs or the service stop working they need restarted.
To do so you can sun `./vm_manager.sh`. This will cycle through the VMs and execute a `vagrant up` command.
This does one of three things depending on the state of the VM:
1. If the VM is down this will bring the VM back online and start the GitHub runner service.
2. If the VM is up but the GitHub runner service is down, this will start the GitHub runner service.
3. If the VM is up and the GitHub runner service is running (i.e., everything is fine) then this does nothing.
Given there is no harm in running this command frequently, we recommend setting up a cron job to automatically execute `./vm_manager.sh` every few hours.
## Troubleshooting ## Troubleshooting
### The default libvirt disk image storage pool is on the wrong drive ### The default libvirt disk image storage pool is on the wrong drive

47
util/github-runners-vagrant/Vagrantfile vendored
View File

@@ -27,34 +27,39 @@
# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE # (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. # OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
NUM_RUNNERS=0 # Set this to the desired number of runners.
PERSONAL_ACCESS_TOKEN="<PERSONAL ACCESS TOKEN>" PERSONAL_ACCESS_TOKEN="<PERSONAL ACCESS TOKEN>"
GITHUB_ORG="<GITHUB_ORG>" GITHUB_ORG="<GITHUB_ORG>"
HOSTNAME="<VM NAME>" HOSTNAME="<VM NAME>"
Vagrant.configure("2") do |config| Vagrant.configure("2") do |config|
config.vm.box = "generic/ubuntu2204"
config.vm.box_check_update = true
config.vm.define "#{HOSTNAME}"
config.vm.hostname = "#{HOSTNAME}"
config.ssh.username = "vagrant" config.ssh.username = "vagrant"
config.ssh.password = "vagrant" config.ssh.password = "vagrant"
config.vm.provider "libvirt" do |vb| (1..NUM_RUNNERS).each do |i|
# Customize the amount of cpus, memory, and storage on the VM:
vb.cpus = "4".to_i config.vm.define "#{HOSTNAME}-#{i}" do |runner|
vb.memory = "16384".to_i runner.vm.hostname = "#{HOSTNAME}-#{i}"
vb.machine_virtual_size = 128 # 128G is the minimum. runner.vm.box = "generic/ubuntu2204"
runner.vm.box_check_update = true
runner.vm.provider "libvirt" do |vb|
# Customize the amount of cpus, memory, and storage on the VM:
vb.cpus = "4".to_i
vb.memory = "16384".to_i
vb.machine_virtual_size = 128 # 128G is the minimum.
end
# sets up vm
runner.vm.provision :shell, path: "provision_root.sh"
runner.vm.provision :shell, privileged: false, path: "provision_nonroot.sh"
# The provision_root.sh adds the vagrant user to the docker group, so we need to reload the VM.
runner.vm.provision :reload
# Copy the "action-run.sh" script from the host to the VM.
runner.vm.provision "file", source: "./action-run.sh", destination: "/tmp/action-run.sh"
runner.vm.provision :shell, privileged: false, inline: "cp /tmp/action-run.sh ."
# Execute the actions-run.sh script on every boot. This configures the and starts the runner.
runner.vm.provision :shell, privileged: false, run: 'always', inline: "./action-run.sh #{PERSONAL_ACCESS_TOKEN} #{GITHUB_ORG} >> action-run.log 2>&1 &"
end
end end
# sets up vm
config.vm.provision :shell, path: "provision_root.sh"
config.vm.provision :shell, privileged: false, path: "provision_nonroot.sh"
# The provision_root.sh adds the vagrant user to the docker group, so we need to reload the VM.
config.vm.provision :reload
# Copy the "action-run.sh" script from the host to the VM.
builder.vm.provision "file", source: "./action-run.sh", destination: "/tmp/action-run.sh"
builder.vm.provision :shell, privileged: false, inline: "cp /tmp/action-run.sh ."
# Execute the actions-run.sh script on every boot. This configures the and starts the runner.
config.vm.provision :shell, privileged: false, run: 'always', inline: "./action-run.sh #{PERSONAL_ACCESS_TOKEN} #{GITHUB_ORG} >> action-run.log 2>&1 &"
end end

54
util/github-runners-vagrant/vm_manager.sh
View File

@@ -1,54 +0,0 @@
#!/bin/bash
# Copyright (c) 2023 The Regents of the University of California
# All rights reserved.
#
# Redistribution and use in source and binary forms, with or without
# modification, are permitted provided that the following conditions are
# met: redistributions of source code must retain the above copyright
# notice, this list of conditions and the following disclaimer;
# redistributions in binary form must reproduce the above copyright
# notice, this list of conditions and the following disclaimer in the
# documentation and/or other materials provided with the distribution;
# neither the name of the copyright holders nor the names of its
# contributors may be used to endorse or promote products derived from
# this software without specific prior written permission.
#
# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
# A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
# OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
# THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
NUM_RUNNERS=3
RUNNER_PREFIX_PREFIX="$(hostname)"
SCRIPT_DIR=$( cd -- "$( dirname -- "${BASH_SOURCE[0]}" )" &> /dev/null && pwd )
export VAGRANT_HOME=${SCRIPT_DIR}
param="up"
if [[ $# -ge 1 ]]; then
param=$1
if [[ "${param}" != "destroy" ]] && [[ "${param}" != "shutdown" ]]; then
echo "Only valid parameters are 'destroy' and 'shutdown' to destroy all VMs or shutdown all VMs"
exit 1
fi
fi
for (( i=1; i<=NUM_RUNNERS; i++ )); do
sed -i "s/HOSTNAME=.*/HOTNAME=\"${RUNNER_PREFIX}-${i}\"/g" Vagrantfile
if [[ "${param}" == "destroy" ]]; then
VAGRANT_VAGRANTFILE=Vagrantfile vagrant destroy -f
elif [[ "${param}" == "shutdown" ]]; then
VAGRANT_VAGRANTFILE=Vagrantfile vagrant halt -f
else
VAGRANT_VAGRANTFILE=Vagrantfile vagrant up --provider=libvirt
fi
done