From 6571a54a65a1e65f9118d0a217c61db7c9611dad Mon Sep 17 00:00:00 2001 From: "Bobby R. Bruce" Date: Wed, 4 Oct 2023 18:45:53 -0700 Subject: [PATCH] util: Use a Multi-Machine Vagrantfile This patch removed the bespoke "vm_manager.sh" script in favor of a Multi-Machine Vagrantfile. With this the users needs to only change the variables in Vagrantfile then use the standard `vagrant` commands to launch the VMs/Runners. Change-Id: Ida5d2701319fd844c6a5b6fa7baf0c48b67db975 --- util/github-runners-vagrant/README.md | 68 ++++++----------------- util/github-runners-vagrant/Vagrantfile | 47 +++++++++------- util/github-runners-vagrant/vm_manager.sh | 54 ------------------ 3 files changed, 43 insertions(+), 126 deletions(-) delete mode 100755 util/github-runners-vagrant/vm_manager.sh diff --git a/util/github-runners-vagrant/README.md b/util/github-runners-vagrant/README.md index 0b93ff6693..ea7fd16957 100644 --- a/util/github-runners-vagrant/README.md +++ b/util/github-runners-vagrant/README.md @@ -1,6 +1,6 @@ # Setting up a Github Actions Runner with Vagrant -This directory provides a way to setup a Github Actions runner using Vagrant to host the runner in a Virtual machine. +This directory provides a way to setup Github Actions runners using Vagrant to host them in Virtual machines. This tutorial has been written with the assumption of running on a machine with Ubuntu 22.04. Setting up a runner on a different OS may require some changes. @@ -21,13 +21,6 @@ sudo apt-get install qemu libvirt-daemon-system libvirt-clients ebtables dnsmasq sudo apt purge vagrant-libvirt ``` -## Set up the Vagrant for the GitHub repository - -First, generate a Personal Access Token, which you can create [here](https://github.com/settings/tokens) -Make sure to set admin permissions on this token, then replace the of `` in the Vagrantfile with your token. - -Next, replace instances of `` with your GitHub organization you wish to add this runner. - ## Install Vagrant Plugins Once everything is set properly, set the `VAGRANT_HOME` environment variable to the directory in which the Vagrant files and other scripts are stored (i.e., the CWD). @@ -45,62 +38,35 @@ vagrant plugin install vagrant-libvirt vagrant plugin install vagrant-reload ``` -## Creating the virtual machine +## Creating the virtual machines -The Vagrantfile in this directory defines a VM that can be used to create a GitHub Actions runner. -It has 4-cores, 16GB of RAM, and 60GB of disk space. +The Vagrantfile in this directory defines the VMs that can be built and used to create a GitHub Actions runner. +This standard VM has 4-cores, 16GB of RAM, and 60GB of disk space. This is sufficient to both compile gem5 and run most simulations. -Each VM on your host system must have a unique name. -Give the VM to be created a unique name by setting the `` variables in the Vagrantfile you wish to utilize. +At the top of the Vagrantfile, there are a few variables that must be set prior to creating the VMs. -Then run: +* `NUM_RUNNERS`: The number of runners to create. +* `PERSONAL_ACCESS_TOKEN`: The GitHub personal access token to use. +You can generate a Personal Access Token [here](https://github.com/settings/tokens) +Make sure to set admin permissions on this token. +* `GITHUB_ORG`: The GitHub organization to add the runners to. +E.g., if the URL to your organization is https://github.com/orgs/gem5, then the variable should be set to "gem5". +* `HOSTNAME` : The hostname of the VM to be created (note, this will be appended with a number to create a unique hostname for each VM). +E.g., if set to `my-machine` and the number of runners set to `2`, two VMs will be created. +One called `my-machine-1` and the other `my-machine-2`. + +When set simply run: ```sh vagrant up --provider=libvirt ``` -This should automatically create your machine, as well as configure and start up a Github Actions runner. +This should automatically create your machines then configure and start up a Github Actions runner in each. You can check the status of the runner here: https://github.com/organizations/{GITHUB_ORG}/settings/actions/runners If the runner ever shows as offline, you can rerun the `vagrant up --provider=libvirt` command to make sure everything is working properly. -If you wish to create more than one runner you must edit the `` in the Vagrant file. - -## Helper scripts - -The "vm_manager" script can be used to set up multiple builder and runner VMs. -To use this script simply modify the `NUM_RUNNERS` and `RUNNER_PREFIX` variables to the desired values. -Then run the script with: - -```sh -./vm_manager.sh -``` - -This script will create any VMs that don't already exist and ensure those that do exists are running. - -If you wish to destroy all the VMs you can run: - -```sh -./vm_manager.sh destroy -``` - -**Note:** This script assumes "VAGRANT_HOME" is set to the CWD. - -## Improving stability - -Occasionally GitHub runner services, or VMs, go down. This is often silent and -usually only noticable from going to the GitHub repo page "settings" -> "actions" -> "runners" and observing the status. -When the VMs or the service stop working they need restarted. -To do so you can sun `./vm_manager.sh`. This will cycle through the VMs and execute a `vagrant up` command. -This does one of three things depending on the state of the VM: - -1. If the VM is down this will bring the VM back online and start the GitHub runner service. -2. If the VM is up but the GitHub runner service is down, this will start the GitHub runner service. -3. If the VM is up and the GitHub runner service is running (i.e., everything is fine) then this does nothing. - -Given there is no harm in running this command frequently, we recommend setting up a cron job to automatically execute `./vm_manager.sh` every few hours. - ## Troubleshooting ### The default libvirt disk image storage pool is on the wrong drive diff --git a/util/github-runners-vagrant/Vagrantfile b/util/github-runners-vagrant/Vagrantfile index 2a891b9688..3446f703dd 100644 --- a/util/github-runners-vagrant/Vagrantfile +++ b/util/github-runners-vagrant/Vagrantfile @@ -27,34 +27,39 @@ # (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE # OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. +NUM_RUNNERS=0 # Set this to the desired number of runners. PERSONAL_ACCESS_TOKEN="" GITHUB_ORG="" HOSTNAME="" Vagrant.configure("2") do |config| - config.vm.box = "generic/ubuntu2204" - config.vm.box_check_update = true - config.vm.define "#{HOSTNAME}" - config.vm.hostname = "#{HOSTNAME}" config.ssh.username = "vagrant" config.ssh.password = "vagrant" - config.vm.provider "libvirt" do |vb| - # Customize the amount of cpus, memory, and storage on the VM: - vb.cpus = "4".to_i - vb.memory = "16384".to_i - vb.machine_virtual_size = 128 # 128G is the minimum. + (1..NUM_RUNNERS).each do |i| + + config.vm.define "#{HOSTNAME}-#{i}" do |runner| + runner.vm.hostname = "#{HOSTNAME}-#{i}" + runner.vm.box = "generic/ubuntu2204" + runner.vm.box_check_update = true + + runner.vm.provider "libvirt" do |vb| + # Customize the amount of cpus, memory, and storage on the VM: + vb.cpus = "4".to_i + vb.memory = "16384".to_i + vb.machine_virtual_size = 128 # 128G is the minimum. + end + + # sets up vm + runner.vm.provision :shell, path: "provision_root.sh" + runner.vm.provision :shell, privileged: false, path: "provision_nonroot.sh" + # The provision_root.sh adds the vagrant user to the docker group, so we need to reload the VM. + runner.vm.provision :reload + # Copy the "action-run.sh" script from the host to the VM. + runner.vm.provision "file", source: "./action-run.sh", destination: "/tmp/action-run.sh" + runner.vm.provision :shell, privileged: false, inline: "cp /tmp/action-run.sh ." + # Execute the actions-run.sh script on every boot. This configures the and starts the runner. + runner.vm.provision :shell, privileged: false, run: 'always', inline: "./action-run.sh #{PERSONAL_ACCESS_TOKEN} #{GITHUB_ORG} >> action-run.log 2>&1 &" + end end - - # sets up vm - config.vm.provision :shell, path: "provision_root.sh" - config.vm.provision :shell, privileged: false, path: "provision_nonroot.sh" - # The provision_root.sh adds the vagrant user to the docker group, so we need to reload the VM. - config.vm.provision :reload - # Copy the "action-run.sh" script from the host to the VM. - builder.vm.provision "file", source: "./action-run.sh", destination: "/tmp/action-run.sh" - builder.vm.provision :shell, privileged: false, inline: "cp /tmp/action-run.sh ." - # Execute the actions-run.sh script on every boot. This configures the and starts the runner. - config.vm.provision :shell, privileged: false, run: 'always', inline: "./action-run.sh #{PERSONAL_ACCESS_TOKEN} #{GITHUB_ORG} >> action-run.log 2>&1 &" - end diff --git a/util/github-runners-vagrant/vm_manager.sh b/util/github-runners-vagrant/vm_manager.sh deleted file mode 100755 index 280ee431bc..0000000000 --- a/util/github-runners-vagrant/vm_manager.sh +++ /dev/null @@ -1,54 +0,0 @@ -#!/bin/bash - -# Copyright (c) 2023 The Regents of the University of California -# All rights reserved. -# -# Redistribution and use in source and binary forms, with or without -# modification, are permitted provided that the following conditions are -# met: redistributions of source code must retain the above copyright -# notice, this list of conditions and the following disclaimer; -# redistributions in binary form must reproduce the above copyright -# notice, this list of conditions and the following disclaimer in the -# documentation and/or other materials provided with the distribution; -# neither the name of the copyright holders nor the names of its -# contributors may be used to endorse or promote products derived from -# this software without specific prior written permission. -# -# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS -# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT -# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR -# A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT -# OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, -# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT -# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, -# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY -# THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT -# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE -# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - -NUM_RUNNERS=3 -RUNNER_PREFIX_PREFIX="$(hostname)" - -SCRIPT_DIR=$( cd -- "$( dirname -- "${BASH_SOURCE[0]}" )" &> /dev/null && pwd ) -export VAGRANT_HOME=${SCRIPT_DIR} - -param="up" -if [[ $# -ge 1 ]]; then - param=$1 - if [[ "${param}" != "destroy" ]] && [[ "${param}" != "shutdown" ]]; then - echo "Only valid parameters are 'destroy' and 'shutdown' to destroy all VMs or shutdown all VMs" - exit 1 - fi -fi - - -for (( i=1; i<=NUM_RUNNERS; i++ )); do - sed -i "s/HOSTNAME=.*/HOTNAME=\"${RUNNER_PREFIX}-${i}\"/g" Vagrantfile - if [[ "${param}" == "destroy" ]]; then - VAGRANT_VAGRANTFILE=Vagrantfile vagrant destroy -f - elif [[ "${param}" == "shutdown" ]]; then - VAGRANT_VAGRANTFILE=Vagrantfile vagrant halt -f - else - VAGRANT_VAGRANTFILE=Vagrantfile vagrant up --provider=libvirt - fi -done