Skip to content
Snippets Groups Projects

Draft: feat: Add getting started docs

1 file
+ 171
0
Compare changes
  • Side-by-side
  • Inline
+ 171
0
# Getting started
## Prerequisites
1. Get yourself an account on our openstack cloud - cloud.rc, by following the instructions from the following link https://docs.rc.uab.edu/uab_cloud/
2. Make sure you have a project with owner or maintainer roles
## HPC factory overview
Why we need it ?
- This project provides build and deploy harness for your applications
- This repo provides a way to define the cluster you want to deploy your application against.
So you built your application and deployed it. Now what ?
- You would need to test the science gateway against a cluster, so set up an adhoc cluster.
Refer to the documentation for our [Cluster on Demand](https://gitlab.rc.uab.edu/rc/cod-heat-stack)
> Note: Make sure you have deployed a working cluster before you move on to next steps.
-------
## Define a packer template
- Follow the example we have in the hpc-factory for defining a packer template. This example template will build an image with commands defined in the provisioner.
TODO - Link the README instructions to install packer etc.
## Setup a CI/CD pipeline
- Make sure you have a runner available to run your jobs.
- Go to **Settings > CI/CD** and expand **Runners**
- If you do not have an active runner, [create a runner](Setup-a-runner)
- Create a .gitlab-ci.yml file at the root of your repository.
- Define a pipeline in the yaml file created above [following the template](https://gitlab.rc.uab.edu/hpc-factory)
- Pipelines are made up of jobs and stages.
- Stages define the order of execution. Typical stages might be `build`, `test`, and `deploy`.
- Jobs specify the tasks to be performed in each stage. For example, a job can compile or test code.
- Pipelines can be triggered by various events, like merges, commits, web or can be on schedule.
- Define your cicd variables
- Two types of variables exist: custom variables and predefined.
- Custom variables are user-defined. Create and manage them in the GitLab UI(settings -> cicd -> Variables), API, or in .gitlab-ci.yml files.
- Predefined variables are automatically set by GitLab and provide information about the current job, pipeline, and environment.
- Setup a scheduled pipeline
- Build -> pipeline schedules
- Set pipeline vars to override user-defined cicd vars if needed
- Run the scheduled pipeline.
## Setup a runner
A runner is an agent in the GitLab Runner application that runs jobs in a GitLab CI/CD pipeline. Jobs are defined in the `.gitlab-ci.yml` file and assigned to available runners.
GitLab has three types of runners:
- Shared: Available to all groups and projects in a GitLab instance.
- Group: Available to all projects and subgroups in a group.
- Project: Associated with specific projects. Typically, project runners are used by one project at a time.
### Step 1: Create an access token
You will first need to create an access token. A [personal access token](https://docs.gitlab.com/ee/user/profile/personal_access_tokens.html) for an administrator account will allow you to create runners at the instance, group, and project levels.
If you only need to create a group or project runner, then it is best to use a group access token or project access token, respectively. For a group or project, navigate to `Settings -> Access Tokens` and create a token. You must specify a name, the token expiration date, role, and scope. For the role, select `Owner`; for the scopes, select `create_runner`
> Warning: This access token is only visible once in the UI. Copy the access token to a secure location
### Step 2: Create a runner config using the access token
Now that you have a scoped token it is time to create a runner config using this token.
You only need to run one of these commands based on your requirement and scope of the token you generated.
``` sh
# Group runner config
curl -sX POST https://<GITLAB_URL>/api/v4/user/runners \
--data runner_type=<RUNNER_TYPE> \
--data "group_id=<target_group_id>" \
--data "description=gitlab-ci-runner" \
--data "tag_list=<your comma-separated tags>" \
--header "PRIVATE-TOKEN: <your_access_token>"
```
``` sh
# Project runner config
curl -sX POST https://<GITLAB_URL>/api/v4/user/runners \
--data runner_type=<RUNNER_TYPE> \
--data "project_id=<target_project_id>" \
--data "description=gitlab-ci-runner" \
--data "tag_list=<your comma-separated tags>" \
--header "PRIVATE-TOKEN: <your_access_token>"
```
``` sh
# Shared runner config
curl -sX POST https://<GITLAB_URL>/api/v4/user/runners \
--data runner_type=<RUNNER_TYPE> \
--data "group_id=<target_group_or_project_id>" \
--data "description=software-eng-docker-builds-runner" \
--data "tag_list=<your comma-separated tags>" \
--header "PRIVATE-TOKEN: <your_access_token>"
```
The newly created runner configuration is visible in the GitLab UI. As the actual runner has not yet been configured, the status displayed is `Never contacted`.
The API will return a message with the following fields: `id`, `token`, and `token_expires_at`. You must save the value for the `token` as it will only be displayed once.
> Note: This auth token can be reused to register multiple runners and they all will be grouped together. Each runner in the group is uniquely identified by their `system_id`.
### Step3: Create a runner VM
```sh
# Set runner name and floating IP
export RUNNER_NAME="ci-runner-img-build"
export FLOATING_IP=<your-floating-ip>
# Create the server and capture the instance ID
openstack server create --flavor <your-flavor> \
--image <your-image> \
--key-name <your-openstack-keypair> \
--network <your-build-network> \
--security-group default \
--security-group <your-security-group> \
$RUNNER_NAME
# Get the instance ID of the created server
RUNNER_INSTANCE_ID=$(openstack server list --name $RUNNER_NAME -f value -c ID)
# Add the floating IP to the server
openstack server add floating ip $RUNNER_INSTANCE_ID $FLOATING_IP
```
### Step4: Install and register runner
The following steps need to run on the runner VM.
`ssh ubuntu@$FLOATING_IP`
``` sh
# Set variables here - replace these with your actual values
GITLAB_INSTANCE_URL="https://gitlab.rc.uab.edu/"
# Update and install necessary packages
sudo apt-get update
sudo apt-get install -y ca-certificates curl gnupg lsb-release
# Create Docker GPG key and repo
sudo install -m 0755 -d /etc/apt/keyrings
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /etc/apt/keyrings/docker.gpg
sudo chmod a+r /etc/apt/keyrings/docker.gpg
echo \
"deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.gpg] https://download.docker.com/linux/ubuntu \
$(lsb_release -cs) stable" | sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
# Install Docker Engine
sudo apt-get install -y docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin
# Install GitLab Runner binary
sudo curl -L --output /usr/local/bin/gitlab-runner https://gitlab-runner-downloads.s3.amazonaws.com/latest/binaries/gitlab-runner-linux-amd64
sudo chmod +x /usr/local/bin/gitlab-runner
# Create a GitLab Runner user
sudo useradd --comment 'GitLab Runner' --create-home gitlab-runner --shell /bin/bash
# Install GitLab Runner as a service
sudo gitlab-runner install --user=gitlab-runner --working-directory=/home/gitlab-runner
sudo gitlab-runner start
# Register GitLab Runner
sudo gitlab-runner register --non-interactive \
--url "$GITLAB_URL" \
--registration-token "$RUNNER_TOKEN"\
--executor "docker" \
--docker-image "alpine:latest" \
--description "OpenStack Runner" \
--tag-list "build, deploy"
# Verify Docker is installed
sudo docker run hello-world
```
Ensure the runner is active and connected.
`Settings -> CICD -> Runners -> Assigned Runners`
Loading