Compare revisions

Krish Moodbidri · Krish Moodbidri · Krish Moodbidri · Krish Moodbidri · Krish Moodbidri · Krish Moodbidri
--- a/README.md
+++ b/README.md
-# rabbitmq_agents
-This repo keeps different rabbitmq agents that help in account creation on OHPC system.
+# RabbitMQ User Registration Agents 
-It has 2 branches ```develop``` and ```production``` , that houses agents based on where they are launched
+---
-## Using RCRMQ class
+## Overview
+This project automates user registration workflows at UAB using **RabbitMQ** to route tasks between systems like the web interface, CLI tools, databases, and email services. It ensures tasks like assigning user IDs, validating data, and sending notifications happen in sequence without manual intervention.  
- First, rename `rabbit_config.py.example` to `rabbit_config.py`
+---
- Modify config file, at least the `Password` needs to be your own passwod
+## Key Components 
+### 1. **The `RegUsr` Exchange** 
+- **Type**: Topic exchange (routes messages using `routing_key` patterns).  
+- **Purpose**: Acts as the central hub for all registration-related messages.  
- In your code:
+### 2. **Core Scripts** 
+- **`self_reg_app` (Web UI)**: Starts the process by sending a `request<queuename>` message with user data.  
+- **`create_account.py` (CLI)**: Triggers backend tasks (e.g., UID/GID assignment, email subscriptions).  
+### 3. **Queues & Their Jobs** 
+| Queue Name               | What It Does                                                                 |  
+|--------------------------|-----------------------------------------------------------------------------|  
+| `get next uid gid`        | Reserves a unique UID/GID for the user (uses SQLite to track IDs).          |  
+| `subscribe mail list`     | Adds the user’s email to mailing lists (e.g., department announcements).    |  
+| `git commit`              | Logs configuration changes to Git (e.g., new user added).                   |  
+| `notify user`             | Sends emails/SMS to users (e.g., "Your account is ready").                  |  
+| `task_manager`            | Coordinates tasks like retrying failed steps or updating logs.              |  
+### 4. **Data Flow** 
+1. A user submits details via the **Web UI** (`self_reg_app`).  
+2. A `request<queuename>` message is sent to `RegUsr` with fields:  
+   ```json  
+   { "username", "queuename", "email", "fullname", "reason" }  
+3.  The system:
+    -   Assigns UID/GID via SQLite (`get next uid gid`  queue).
+    -   Validates data with a  `verify<queuename>`  message.
+    -   Sends a  `completed<queuename>`  message with success/failure status.
+    -   Notifies the user and logs the event.
+----------
+## Setup & Usage
+### Prerequisites
+* Launch an alma9 instance on openstack, ssh into it and run the following commands
+*Before proceeding, set these environment variables:*
+```
+export RABBITMQ_USER="reggie"                  # RabbitMQ admin username
+export RABBITMQ_PASSWORD="secure_password"     # RabbitMQ admin password
+export CELERY_VHOST="adduser"                  # Celery-dedicated vhost
+export CELERY_USER="celery_user"               # Celery service username
+export CELERY_PASSWORD="celery_pass"           # Celery service password
+```
+### 1.1 Configure Package Repositories
+# Import security keys
+```
+sudo rpm --import https://github.com/rabbitmq/signing-keys/releases/download/2.0/rabbitmq-release-signing-key.asc
+sudo rpm --import https://packagecloud.io/rabbitmq/erlang/gpgkey
+sudo rpm --import https://packagecloud.io/rabbitmq/rabbitmq-server/gpgkey
+```
+# Erlang Repository (AlmaLinux 8)
+```
+sudo tee /etc/yum.repos.d/rabbitmq_erlang.repo <<'EOL'
+[rabbitmq_erlang]
+name=RabbitMQ Erlang
+baseurl=https://packagecloud.io/rabbitmq/erlang/el/8/$basearch
+gpgcheck=1
+gpgkey=https://packagecloud.io/rabbitmq/erlang/gpgkey
+       https://github.com/rabbitmq/signing-keys/releases/download/2.0/rabbitmq-release-signing-key.asc
+enabled=1
+EOL
+```
+# RabbitMQ Server Repository
+```
+sudo tee /etc/yum.repos.d/rabbitmq_server.repo <<'EOL'
+[rabbitmq_server]
+name=RabbitMQ Server
+baseurl=https://packagecloud.io/rabbitmq/rabbitmq-server/el/8/$basearch
+gpgcheck=1
+gpgkey=https://packagecloud.io/rabbitmq/rabbitmq-server/gpgkey
+enabled=1
+EOL
+```
+### 1.2 Install Packages
+```
+sudo yum install -y socat logrotate
+sudo yum install -y erlang-25.3.2 rabbitmq-server-3.12.6
+sudo systemctl enable --now rabbitmq-server
+```
+## 2. RabbitMQ Configuration
+### 2.1 Create Admin User & Permissions
+```
+sudo rabbitmqctl add_user $RABBITMQ_USER $RABBITMQ_PASSWORD
+sudo rabbitmqctl set_user_tags $RABBITMQ_USER administrator
+sudo rabbitmqctl set_permissions -p / $RABBITMQ_USER ".*" ".*" ".*"
+```
+### 2.2 Configure Celery Environment
+```
+sudo rabbitmqctl add_vhost $CELERY_VHOST
+sudo rabbitmqctl add_user $CELERY_USER $CELERY_PASSWORD
+sudo rabbitmqctl set_permissions -p $CELERY_VHOST $CELERY_USER ".*" ".*" ".*"
+```
+## 3. Agent Service Setup
+### 3.1 Deploy Agent Codebase
 ```
-# import the class
+AGENT_DIR="/cm/shared/rabbitmq_agents"
-from rc_rmq import RCRMQ
+sudo git clone https://gitlab.rc.uab.edu/rc/rabbitmq_agents.git $AGENT_DIR
+cd $AGENT_DIR
+sudo git checkout main  # Or specific commit/tag
+```
+### 3.2 Configure Python Environment
+```
+sudo python3 -m venv $AGENT_DIR/venv
+sudo $AGENT_DIR/venv/bin/pip install -r $AGENT_DIR/requirements.txt
+```
-# instantiate an instance
+### 3.3 Create Agent Configuration
-rc_rmq = RCRMQ({'exchange': 'RegUsr'})
-# publish a message to message queue
+Create  `$AGENT_DIR/rabbit_config.py`  with:
-rc_rmq.publish_msg({
-  'routing_key': 'your_key',
-  'msg': {
-    'type': 'warning',
-    'content': 'this is warning'
-  }
-})
-# to consume message from a queue
+```
-# you have to first define callback function
+# rabbit_config.py
-# with parameters: channel, method, properties, body
+host = 'localhost'           # Or your cluster host
-def callback_function(ch, method, properties, body):
+port = 5672
-  msg = json.loads(body)
+vhost = '$CELERY_VHOST'      # Use actual vhost name
-  print("get msg: {}".format(msg['username')
+user = '$CELERY_USER'        # Celery username
+password = '$CELERY_PASSWORD'
+```
-  # this will stop the consumer
+### 3.4 Systemd Service Setup for All Agents
-  rc_rmq.stop_consumer()
+OHPC Account Agent:
-# start consume messagre from queue with callback function
+```
-rc_rmq.start_consume({
+sudo tee /etc/systemd/system/ohpc_account_agent.service <<EOL
-  'queue': 'queue_name',
+[Unit]
-  'routing_key: 'your_key',
+After=rabbitmq-server.service
-  'cb': callback_function
-})
-# don't forget to close connection
+[Service]
-rc_rmq.disconnect()
+Type=simple
+StartLimitInterval=0
+Restart=on-failure
+User=root
+WorkingDirectory=/cm/shared/rabbitmq_agents/dev_rmq_agents
+Environment="PYTHONPATH=/cm/shared/rabbitmq_agents/"
+Environment="PATH=/cm/shared/rabbitmq_agents/dev_rmq_agents/venv/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin"
+ExecStart=/cm/shared/rabbitmq_agents/venv/bin/python -u ohpc_account_create.py
+[Install]
+WantedBy=multi-user.target
+EOL
 ```
-### Account creation flowchart
+OOD Account Agent:
-![Account creation flowchart](./account-creation-flow.png)
\ No newline at end of file
+```
+sudo tee /etc/systemd/system/ood_account_agent.service <<EOL
+[Unit]
+After=rabbitmq-server.service
+[Service]
+Type=simple
+StartLimitInterval=0
+Restart=on-failure
+User=root
+WorkingDirectory=/cm/shared/rabbitmq_agents/dev_rmq_agents
+Environment="PYTHONPATH=/cm/shared/rabbitmq_agents/"
+Environment="PATH=/cm/shared/rabbitmq_agents/dev_rmq_agents/venv/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin"
+ExecStart=/cm/shared/rabbitmq_agents/venv/bin/python -u ood_account_create.py
+[Install]
+WantedBy=multi-user.target
+EOL
+```
+Slurm Agent:
+```
+sudo tee /etc/systemd/system/slurm_agent.service <<EOL
+[Unit]
+After=rabbitmq-server.service
+[Service]
+Type=simple
+StartLimitInterval=0
+Restart=on-failure
+User=root
+WorkingDirectory=/cm/shared/rabbitmq_agents/dev_rmq_agents
+Environment="PYTHONPATH=/cm/shared/rabbitmq_agents/"
+Environment="PATH=/cm/shared/rabbitmq_agents/dev_rmq_agents/venv/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin"
+ExecStart=/cm/shared/rabbitmq_agents/venv/bin/python -u slurm_agent.py
+[Install]
+WantedBy=multi-user.target
+EOL
+```
+## 4. Network Configuration
+### Disable Firewall (Testing Only)
+```
+sudo systemctl stop firewalld
+sudo systemctl disable firewalld
+```
+## 5. Validation & Testing
+```
+# Verify Services 
+systemctl is-active rabbitmq-server ood_account_agent 
+# Check RabbitMQ Users  
+sudo rabbitmqctl list_users 
+# Inspect Agent Logs 
+journalctl -u ood_account_agent -f  --since  "5m ago"
+```
+## Error Handling
+-   Failures (e.g., duplicate email) are reported in the  `completed<queuename>`  message’s  `errmsg`  field.
+-   The  `user reg event logger`  tracks all registration attempts. Check logs at  `/var/log/user_reg.log`.
No results found