Skip to content
Snippets Groups Projects
Krish Moodbidri's avatar
Krish Moodbidri authored
Update README with improvements and corrections

See merge request !156
a93b8ea1

RabbitMQ User Registration Agents


Overview

This project automates user registration workflows at UAB using RabbitMQ to route tasks between systems like the web interface, CLI tools, databases, and email services. It ensures tasks like assigning user IDs, validating data, and sending notifications happen in sequence without manual intervention.


Key Components

1. The RegUsr Exchange

  • Type: Topic exchange (routes messages using routing_key patterns).
  • Purpose: Acts as the central hub for all registration-related messages.

2. Core Scripts

  • self_reg_app (Web UI): Starts the process by sending a request<queuename> message with user data.
  • create_account.py (CLI): Triggers backend tasks (e.g., UID/GID assignment, email subscriptions).

3. Queues & Their Jobs

Queue Name What It Does
get next uid gid Reserves a unique UID/GID for the user (uses SQLite to track IDs).
subscribe mail list Adds the user’s email to mailing lists (e.g., department announcements).
git commit Logs configuration changes to Git (e.g., new user added).
notify user Sends emails/SMS to users (e.g., "Your account is ready").
task_manager Coordinates tasks like retrying failed steps or updating logs.

4. Data Flow

  1. A user submits details via the Web UI (self_reg_app).

  2. A request<queuename> message is sent to RegUsr with fields:

    { "username", "queuename", "email", "fullname", "reason" }  
    
  3. The system:

    • Assigns UID/GID via SQLite (get next uid gid queue).

    • Validates data with a verify<queuename> message.

    • Sends a completed<queuename> message with success/failure status.

    • Notifies the user and logs the event.


Setup & Usage

Prerequisites

  • Launch an alma9 instance on openstack, ssh into it and run the following commands

Before proceeding, set these environment variables:

export RABBITMQ_USER="reggie"                  # RabbitMQ admin username
export RABBITMQ_PASSWORD="secure_password"     # RabbitMQ admin password
export CELERY_VHOST="adduser"                  # Celery-dedicated vhost
export CELERY_USER="celery_user"               # Celery service username
export CELERY_PASSWORD="celery_pass"           # Celery service password

1.1 Configure Package Repositories

Import security keys

sudo rpm --import https://github.com/rabbitmq/signing-keys/releases/download/2.0/rabbitmq-release-signing-key.asc
sudo rpm --import https://packagecloud.io/rabbitmq/erlang/gpgkey
sudo rpm --import https://packagecloud.io/rabbitmq/rabbitmq-server/gpgkey

Erlang Repository (AlmaLinux 8)

sudo tee /etc/yum.repos.d/rabbitmq_erlang.repo <<'EOL'
[rabbitmq_erlang]
name=RabbitMQ Erlang
baseurl=https://packagecloud.io/rabbitmq/erlang/el/8/$basearch
gpgcheck=1
gpgkey=https://packagecloud.io/rabbitmq/erlang/gpgkey
       https://github.com/rabbitmq/signing-keys/releases/download/2.0/rabbitmq-release-signing-key.asc
enabled=1
EOL

RabbitMQ Server Repository

sudo tee /etc/yum.repos.d/rabbitmq_server.repo <<'EOL'
[rabbitmq_server]
name=RabbitMQ Server
baseurl=https://packagecloud.io/rabbitmq/rabbitmq-server/el/8/$basearch
gpgcheck=1
gpgkey=https://packagecloud.io/rabbitmq/rabbitmq-server/gpgkey
enabled=1
EOL

1.2 Install Packages

sudo yum install -y socat logrotate
sudo yum install -y erlang-25.3.2 rabbitmq-server-3.12.6
sudo systemctl enable --now rabbitmq-server

2. RabbitMQ Configuration

2.1 Create Admin User & Permissions

sudo rabbitmqctl add_user $RABBITMQ_USER $RABBITMQ_PASSWORD
sudo rabbitmqctl set_user_tags $RABBITMQ_USER administrator
sudo rabbitmqctl set_permissions -p / $RABBITMQ_USER ".*" ".*" ".*"

2.2 Configure Celery Environment

sudo rabbitmqctl add_vhost $CELERY_VHOST
sudo rabbitmqctl add_user $CELERY_USER $CELERY_PASSWORD
sudo rabbitmqctl set_permissions -p $CELERY_VHOST $CELERY_USER ".*" ".*" ".*"

3. Agent Service Setup

3.1 Deploy Agent Codebase

AGENT_DIR="/cm/shared/rabbitmq_agents"
sudo git clone https://gitlab.rc.uab.edu/rc/rabbitmq_agents.git $AGENT_DIR
cd $AGENT_DIR
sudo git checkout main  # Or specific commit/tag

3.2 Configure Python Environment

sudo python3 -m venv $AGENT_DIR/venv
sudo $AGENT_DIR/venv/bin/pip install -r $AGENT_DIR/requirements.txt

3.3 Create Agent Configuration

Create $AGENT_DIR/rabbit_config.py with:

# rabbit_config.py
host = 'localhost'           # Or your cluster host
port = 5672
vhost = '$CELERY_VHOST'      # Use actual vhost name
user = '$CELERY_USER'        # Celery username
password = '$CELERY_PASSWORD'

3.4 Systemd Service Setup for All Agents

OHPC Account Agent:

sudo tee /etc/systemd/system/ohpc_account_agent.service <<EOL
[Unit]
After=rabbitmq-server.service

[Service]
Type=simple
StartLimitInterval=0
Restart=on-failure
User=root
WorkingDirectory=/cm/shared/rabbitmq_agents/dev_rmq_agents
Environment="PYTHONPATH=/cm/shared/rabbitmq_agents/"
Environment="PATH=/cm/shared/rabbitmq_agents/dev_rmq_agents/venv/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin"
ExecStart=/cm/shared/rabbitmq_agents/venv/bin/python -u ohpc_account_create.py

[Install]
WantedBy=multi-user.target
EOL

OOD Account Agent:

sudo tee /etc/systemd/system/ood_account_agent.service <<EOL
[Unit]
After=rabbitmq-server.service

[Service]
Type=simple
StartLimitInterval=0
Restart=on-failure
User=root
WorkingDirectory=/cm/shared/rabbitmq_agents/dev_rmq_agents
Environment="PYTHONPATH=/cm/shared/rabbitmq_agents/"
Environment="PATH=/cm/shared/rabbitmq_agents/dev_rmq_agents/venv/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin"
ExecStart=/cm/shared/rabbitmq_agents/venv/bin/python -u ood_account_create.py

[Install]
WantedBy=multi-user.target
EOL

Slurm Agent:

sudo tee /etc/systemd/system/slurm_agent.service <<EOL
[Unit]
After=rabbitmq-server.service

[Service]
Type=simple
StartLimitInterval=0
Restart=on-failure
User=root
WorkingDirectory=/cm/shared/rabbitmq_agents/dev_rmq_agents
Environment="PYTHONPATH=/cm/shared/rabbitmq_agents/"
Environment="PATH=/cm/shared/rabbitmq_agents/dev_rmq_agents/venv/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin"
ExecStart=/cm/shared/rabbitmq_agents/venv/bin/python -u slurm_agent.py

[Install]
WantedBy=multi-user.target
EOL

4. Network Configuration

Disable Firewall (Testing Only)

sudo systemctl stop firewalld
sudo systemctl disable firewalld

5. Validation & Testing

# Verify Services 
systemctl is-active rabbitmq-server ood_account_agent 

# Check RabbitMQ Users  
sudo rabbitmqctl list_users 

# Inspect Agent Logs 
journalctl -u ood_account_agent -f  --since  "5m ago"

Error Handling

  • Failures (e.g., duplicate email) are reported in the completed<queuename> message’s errmsg field.

  • The user reg event logger tracks all registration attempts. Check logs at /var/log/user_reg.log.