diff --git a/README.md b/README.md index a58d15fb4cf0e1324ebeb9d272abb11361e7c1f7..3be9839d4cc3b27a10a97e0e459c2f319a3bb6dc 100644 --- a/README.md +++ b/README.md @@ -1,27 +1,45 @@ +# Introduction +This is a flask application that leverages [RabbitMQ](https://www.rabbitmq.com/) and [Celery](https://docs.celeryproject.org/en/latest/getting-started/introduction.html) +to asynchronously create a Cheaha user account. Currently the project is being developed on an Openstack cluster. # Project Setup To clone this repo use the command: ``` $ git clone https://gitlab.rc.uab.edu/mmoo97/flask_user_reg.git +$ cd flask_user_reg ``` ## Prerequisites -### Clone Repository -- Ensure `pip` is installed (see: https://packaging.python.org/guides/installing-using-pip-and-virtualenv/ ). - - Check if installed by typing `$ pip` for Mac/Linux or `$ py` for Windows. - - Mac/Linux: Install pip using `$ python -m pip install --user --upgrade pip`. - - Windows: Install pip using `$ py -m pip install --upgrade pip` -- Ensure you have created a [virtual environment](https://packaging.python.org/guides/installing-using-pip-and-virtual-environments) -called `venv` setup within the cloned project. - - Note, this project requires a virtual environment running python2 (2.7.x) -- Ensure Flask and other dependencies are installed using the following commands: +### Setup a Virtual Environment +- Ensure you have created a [virtual environment](https://docs.python.org/3/library/venv.html) +called `venv` setup running python3. + - Note, this project requires a virtual environment running __python3__ (__3.4.10__ in this case). + - Create this by navigating to you home directory via typing `$ cd` and entering the following commands:<br> + ``` + $ python3 -m venv ~/venv + $ source ~/venv + ``` + - Upon Activation, you should see the prompt update accordingly: + ``` + [centos@ood ~]$ <------Old Prompt + (venv) [centos@ood ~]$ <------New Prompt + ``` + In this case, the env name is displayed as `venv` but would change to reflect whatever name you initialized it with + in the previous step. Additionally, this example is running on the `ood node` provisioned + via __OpenStack__. +- Ensure [pip](https://docs.python.org/3/installing/index.html) is installed. + - #### Mac/Linux + - Check if installed by typing `$ pip` + - Install pip using `$ python -m pip install --user --upgrade pip`. + - #### Windows + - Check if installed using `$ py` + - Install pip using `$ py -m pip install --upgrade pip`<br><br> +- Ensure Flask and other dependencies are installed to you virtual environment using the following commands: ``` $ cd ~/your/repo/path/flask_user_reg -$ git checkout version-1b-openstack-rabbitmq -$ source venv/bin/activate $ pip install -r requirements.txt ``` -- Note, to install flask in your own `$HOME` use `pip install --user Flask`. +- Note, to install flask in your own `$HOME` use `pip install --user Flask`. ### Install RabbitMQ (Reference: [here](https://www.rabbitmq.com/tutorials/tutorial-four-python.html)) - Install RabbitMQ server on the host machine. ([Installation Guide](https://www.rabbitmq.com/download.html)) @@ -47,13 +65,53 @@ $ pip install -r requirements.txt `$ rabbitmqctl change_password reggie <new_password>`. - Note that rabbitmqctl may require sudo permissions and that changing the password will require a password change in the credentials within `test_producer.py` and `base_consumer.py` as well. +# Run Project + +## 1. Connect to OHPC/OOD +1. Network Setup: <br> + Assuming proper steps have been run to provision your openstack cluster, your network + topology should look like the following:<br> +  + +2. Locate Floating Ip for each instance: <br> + <br> + According to this diagram of the network infrastructure, the IP Addresses that you will need to + connect to in order to ssh into each machine will be `164.111.161.xxx` where + `.xxx` is the last octet of your assigned Floating IP Addresses shown below.<br> + <br> + Note that these numbers will typically vary depending on factors in the provisioning + process. -## Test RabbitMQ +3. SSH into each machine: <br> + Now that you have the IP Addresses of the OHPC and OOD nodes, you can connect + to them via the following command in two separate shell windows/tabs: <br> + ``` + $ ssh centos@164.111.161.xxx + ``` + +4. Drop Firewall: <br> + Currently, the firewall prevents us from connecting to our soon to be running flask app. + To check if the firewall is active, type: <br> + `$ sudo systemctl status firewalld` <br> + In the case the firewall is still active, type: <br> + `$ sudo systemctl stop firewalld` <br> + In the case you need to turn the firewall back on, type: <br> + `sudo systemctl start firewalld` + +5. Edit Security Rules: <br> + Lastly, we want to make sure that our network it open to run on our flask application + which runs on `localhost:5000` by default. Modify your __Default__ security group to + reflect the following rules: <br> +  <br> + The main takeaway/modification in these rules is that traffic is allowed in and out of port 5000. + + +## 2. Initialize RabbitMQ +Make sure that you are running rabbitmq via the command `sudo rabbitmq server`. +## 3. Test RabbitMQ For a simple example on the functionality of RabbitMQ, do the following: - Open up a new ssh terminal and ssh into your ohpc node and, in a separate window, ssh into your ood node. -- Once in, ensure your rabbitmq server is running using the command ` ` - Additionally, ensure you have a rabbitmq user configured with the username and password as `reggie`. -- ``` # Run consumer on ohpc node @@ -66,6 +124,38 @@ $ python test_producer.py ohpc ``` You should now see that the message has been sent and displayed on the ohpc node. -- **Note,** that the `test_producer.py` script is identical to the code within the `ingest_data()` function in `run.py`. - - \ No newline at end of file + +## 4. Celery Worker +In order to execute our tasks asychronously, we create a celery worker using the fllowing: <br> +`$ celery -A tasks worker --loglevel=info --concurrency=4 1> ~/celery.out 2> ~/celery.err &` <br><br> +In this case, the `celery -A tasks worker --loglevel=info --concurrency=4` portion of the command +is what is initiating the worker and the remainder serves to write the stdout and stderr to two separate files +located in the home directory and to run the process in the background. <br> + + +## 5. Initialize the Flask App +Simply type `$ python run.py` <br> +Alternatively, type `$ python run.py > ~/flask.out 2> ~/flask.err &` to run the process in the background +and output to files in the home directory. + +## 6. Check Job status + +If all goes well, you should have your processes running in the background. Check the status of these by typing +`$ jobs`. You should now see the something similar to the following: <br> +``` +(venv) [centos@ood flask_user_reg]$ jobs +[1]- Running celery -A tasks worker --loglevel=info --concurrency=4 > ~/celery.out 2> ~/celery.err & +[2]+ Running python run.py > ~/server.out 2> ~/server.err & +(venv) [centos@ood flask_user_reg]$ +``` + +## 7. Connect to Server +Open a new browser window and connect to the __OOD__ node by typing `http://164.111.161.xxx:5000` in the +address bar replacing `.xxx` with the final octet of the __OOD__ node. You should see something like this: <br> + <br> +## 8. Key Info/Observe +- Fill out the form and hit "Submit" +- You should see an overlay signifying the account is being created. After 5 seconds it should disappear and signify the account creation has been successful. +- You can now enter the command `kill %1 %2` to terminate the celery worker and the flask server. +- You can view the output/errors of either the worker and flask server by opening `flask.out`, `flask.err`, +`celery.out`, or `celery.err`. \ No newline at end of file diff --git a/docs/images/floating_ips.png b/docs/images/floating_ips.png new file mode 100644 index 0000000000000000000000000000000000000000..dc6c24604ffeb5e914075437d26babdf5b038d65 Binary files /dev/null and b/docs/images/floating_ips.png differ diff --git a/docs/images/network_topology.png b/docs/images/network_topology.png new file mode 100644 index 0000000000000000000000000000000000000000..192e173a0635c64f739ae2c9de801a4bf81d81c2 Binary files /dev/null and b/docs/images/network_topology.png differ diff --git a/docs/images/rc-network .png b/docs/images/rc-network .png new file mode 100644 index 0000000000000000000000000000000000000000..42e3244c98cc41516805f0bc39ac94f0acc274dd Binary files /dev/null and b/docs/images/rc-network .png differ diff --git a/docs/images/security_rules.png b/docs/images/security_rules.png new file mode 100644 index 0000000000000000000000000000000000000000..0081fdf31891ff85aacbde479a855cc62370e3d5 Binary files /dev/null and b/docs/images/security_rules.png differ diff --git a/docs/images/site_example.png b/docs/images/site_example.png new file mode 100644 index 0000000000000000000000000000000000000000..116abb2b0c053c460a229c7c96b26d12313486bc Binary files /dev/null and b/docs/images/site_example.png differ