Skip to content
GitLab
Explore
Sign in
Register
Primary navigation
Search or go to…
Project
R
rabbitmq_agents
Manage
Activity
Members
Labels
Plan
Issues
14
Issue boards
Milestones
Wiki
Code
Merge requests
8
Repository
Branches
Commits
Tags
Repository graph
Compare revisions
Snippets
Build
Pipelines
Jobs
Pipeline schedules
Artifacts
Deploy
Releases
Package Registry
Container Registry
Model registry
Operate
Environments
Terraform modules
Monitor
Incidents
Analyze
Value stream analytics
Contributor analytics
CI/CD analytics
Repository analytics
Model experiments
Help
Help
Support
GitLab documentation
Compare GitLab plans
Community forum
Contribute to GitLab
Provide feedback
Keyboard shortcuts
?
Snippets
Groups
Projects
Show more breadcrumbs
rc
rabbitmq_agents
Merge requests
!156
Update README with improvements and corrections
Code
Review changes
Check out branch
Download
Patches
Plain diff
Merged
Update README with improvements and corrections
krish94/rabbitmq_agents:feat-almalinux-readme
into
main
Overview
0
Commits
5
Pipelines
2
Changes
1
Merged
Krish Moodbidri
requested to merge
krish94/rabbitmq_agents:feat-almalinux-readme
into
main
2 months ago
Overview
0
Commits
5
Pipelines
2
Changes
1
Expand
Fixed JSON code block syntax and structure in Data Flow section
Added hyperlinks to CentOS setup steps for RabbitMQ configuration
Added vhost config
👍
0
👎
0
Merge request reports
Compare
main
version 1
909ed6bc
2 months ago
main (base)
and
version 1
latest version
c6b96aa5
5 commits,
2 months ago
version 1
909ed6bc
5 commits,
2 months ago
1 file
+
191
−
53
Inline
Compare changes
Side-by-side
Inline
Show whitespace changes
Show one file at a time
README.md
+
191
−
53
Options
@@ -49,67 +49,205 @@ This project automates user registration workflows at UAB using **RabbitMQ** to
###
Prerequisites
-
RabbitMQ
server
running
with
the
`RegUsr`
exchange.
-
SQLite
database
for
UID/GID
storage.
*
Launch
an
alma
9
instance
on
openstack,
ssh
into
it
and
run
the
following
commands
*Before
proceeding,
set
these
environment
variables:*
```
export RABBITMQ_USER="reggie" # RabbitMQ admin username
export RABBITMQ_PASSWORD="secure_password" # RabbitMQ admin password
export CELERY_VHOST="adduser" # Celery-dedicated vhost
export CELERY_USER="celery_user" # Celery service username
export CELERY_PASSWORD="celery_pass" # Celery service password
```
### 1.1 Configure Package Repositories
# Import security keys
```
sudo rpm --import https://github.com/rabbitmq/signing-keys/releases/download/2.0/rabbitmq-release-signing-key.asc
sudo rpm --import https://packagecloud.io/rabbitmq/erlang/gpgkey
sudo rpm --import https://packagecloud.io/rabbitmq/rabbitmq-server/gpgkey
```
# Erlang Repository (AlmaLinux 8)
```
sudo tee /etc/yum.repos.d/rabbitmq_erlang.repo <<'EOL'
[rabbitmq_erlang]
name=RabbitMQ Erlang
baseurl=https://packagecloud.io/rabbitmq/erlang/el/8/$basearch
gpgcheck=1
gpgkey=https://packagecloud.io/rabbitmq/erlang/gpgkey
https://github.com/rabbitmq/signing-keys/releases/download/2.0/rabbitmq-release-signing-key.asc
enabled=1
EOL
```
# RabbitMQ Server Repository
```
sudo tee /etc/yum.repos.d/rabbitmq_server.repo <<'EOL'
[rabbitmq_server]
name=RabbitMQ Server
baseurl=https://packagecloud.io/rabbitmq/rabbitmq-server/el/8/$basearch
gpgcheck=1
gpgkey=https://packagecloud.io/rabbitmq/rabbitmq-server/gpgkey
enabled=1
EOL
```
### 1.2 Install Packages
```
sudo yum install -y socat logrotate
sudo yum install -y erlang-25.3.2 rabbitmq-server-3.12.6
sudo systemctl enable --now rabbitmq-server
```
## 2. RabbitMQ Configuration
### 2.1 Create Admin User & Permissions
```
sudo rabbitmqctl add_user $RABBITMQ_USER $RABBITMQ_PASSWORD
sudo rabbitmqctl set_user_tags $RABBITMQ_USER administrator
sudo rabbitmqctl set_permissions -p / $RABBITMQ_USER ".
*" ".*
" ".
*
"
```
### 2.2 Configure Celery Environment
```
sudo rabbitmqctl add_vhost $CELERY_VHOST
sudo rabbitmqctl add_user $CELERY_USER $CELERY_PASSWORD
sudo rabbitmqctl set_permissions -p $CELERY_VHOST $CELERY_USER ".
*" ".*
" ".
*
"
```
## 3. Agent Service Setup
### 3.1 Deploy Agent Codebase
```
AGENT_DIR="/cm/shared/rabbitmq_agents"
sudo git clone https://gitlab.rc.uab.edu/rc/rabbitmq_agents.git $AGENT_DIR
cd $AGENT_DIR
sudo git checkout main # Or specific commit/tag
```
### 3.2 Configure Python Environment
```
sudo python3 -m venv $AGENT_DIR/venv
sudo $AGENT_DIR/venv/bin/pip install -r $AGENT_DIR/requirements.txt
```
### 3.3 Create Agent Configuration
Create `$AGENT_DIR/rabbit_config.py` with:
```
# rabbit_config.py
host = 'localhost' # Or your cluster host
port = 5672
vhost = '$CELERY_VHOST' # Use actual vhost name
user = '$CELERY_USER' # Celery username
password = '$CELERY_PASSWORD'
```
### 3.4 Systemd Service Setup for All Agents
OHPC Account Agent:
```
sudo tee /etc/systemd/system/ohpc_account_agent.service <<EOL
[Unit]
After=rabbitmq-server.service
[Service]
Type=simple
StartLimitInterval=0
Restart=on-failure
User=root
WorkingDirectory=/cm/shared/rabbitmq_agents/dev_rmq_agents
Environment="PYTHONPATH=/cm/shared/rabbitmq_agents/"
Environment="PATH=/cm/shared/rabbitmq_agents/dev_rmq_agents/venv/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin"
ExecStart=/cm/shared/rabbitmq_agents/venv/bin/python -u ohpc_account_create.py
[Install]
WantedBy=multi-user.target
EOL
```
OOD Account Agent:
```
sudo tee /etc/systemd/system/ood_account_agent.service <<EOL
[Unit]
After=rabbitmq-server.service
###
Configuration
Steps
[Service]
Type=simple
StartLimitInterval=0
Restart=on-failure
User=root
WorkingDirectory=/cm/shared/rabbitmq_agents/dev_rmq_agents
Environment="PYTHONPATH=/cm/shared/rabbitmq_agents/"
Environment="PATH=/cm/shared/rabbitmq_agents/dev_rmq_agents/venv/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin"
ExecStart=/cm/shared/rabbitmq_agents/venv/bin/python -u ood_account_create.py
1
.
**Bind
Queues
to
`RegUsr`
Exchange**:
Use
these
routing
keys:
-
`request<queuename>`
-
`completed<queuename>`
-
`verify<queuename>`
_(Replace
`<queuename>`
with
your
queue’s
name,
e.g.,
`request_user_reg`)_
2
.
**Deploy
Agents**:
-
Run
the
Web
UI
(`self_reg_app`)
for
user
submissions.
-
Execute
`create_account.py`
to
process
tasks
(e.g.,
UID
assignment).
3
.
**Monitor
Queues**:
Use
RabbitMQ’s
management
UI
or
CLI
tools
to
check:
-
`task_manager`
for
workflow
progress.
-
`notify
user`
for
delivery
status
of
emails/SMS.
[Install]
WantedBy=multi-user.target
EOL
```
----------
Slurm Agent:
##
Error
Handling
```
sudo tee /etc/systemd/system/slurm_agent.service <<EOL
[Unit]
After=rabbitmq-server.service
-
Failures
(e.g.,
duplicate
email)
are
reported
in
the
`completed<queuename>`
message’s
`errmsg`
field.
-
The
`user
reg
event
logger`
tracks
all
registration
attempts.
Check
logs
at
`/var/log/user_reg.log`.
[Service]
Type=simple
StartLimitInterval=0
Restart=on-failure
User=root
WorkingDirectory=/cm/shared/rabbitmq_agents/dev_rmq_agents
Environment="PYTHONPATH=/cm/shared/rabbitmq_agents/"
Environment="PATH=/cm/shared/rabbitmq_agents/dev_rmq_agents/venv/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin"
ExecStart=/cm/shared/rabbitmq_agents/venv/bin/python -u slurm_agent.py
----------
[Install]
WantedBy=multi-user.target
EOL
```
##
Example
Workflow
**Scenario**:
A
researcher
registers
via
the
Web
UI.
## 4. Network Configuration
### Disable Firewall (Testing Only)
1
.
Web
UI
sends
`request_researcher_reg`
with
their
details.
2
.
System
assigns
UID/GID
from
SQLite.
3
.
A
`verify_researcher_reg`
message
ensures
data
is
valid.
4
.
On
success:
```
sudo systemctl stop firewalld
sudo systemctl disable firewalld
```
## 5. Validation & Testing
```
# Verify Services
systemctl is-active rabbitmq-server ood_account_agent
# Check RabbitMQ Users
sudo rabbitmqctl list_users
# Inspect Agent Logs
journalctl -u ood_account_agent -f --since "5m ago"
```
## Error Handling
- Failures (e.g., duplicate email) are reported in the `completed<queuename>` message’s `errmsg` field.
-
`completed_researcher_reg`
marks
`success:
True`.
-
`notify_researcher_reg`
triggers
a
confirmation
email.
5
.
On
failure:
- The `user reg event logger` tracks all registration attempts. Check logs at `/var/log/user_reg.log`.
-
`errmsg`
lists
issues
(e.g.,
"Email already exists"
).
----------
Loading