RELION interactive app in OOD

Background and justification

In the near future, we are retiring sinteractive. Researchers with CryoEM data who use RELION have expressed a need that exposes a gap in our services. Currently, the only way CryoEM researchers can interact with the RELION GUI with (up to) four GPUs is using sinteractive.

The need has been demonstrated directly in tickets and email discussions by three researchers and, from discussions with the CryoEM facility director (Terje Dokland), there are at least 6 other regular users who have not communicated with us. When the CryoEM facility is fully operational again (expected May-June 2023), we will see an increase in RELION use on Cheaha.

An additional detail is the need to know when requested jobs have started. The typical workflow is to request an entire pascalnodes node at once. It can take substantial time for the job to start, so a functional email feedback system is needed.

Update 2023-04-12: Reviewing with James Kizziah reveals the software has multiple windows that must be open simultaneously. They must all be discoverable and interactive as needed.

Proposed action items:

• Add RELION as first-class interactive app in OOD.
• RELION job submission form allows selection of number of GPUs.
• Each RELION job allocation should be within a single node.
• Ensure module load RELION/4.0.0 works as expected with multiple GPUs. (See rc/cluster-software#76)
• Ensure module load RELION/4.0.0 has all functionality described here
• Functional "I would like to receive an email when the session starts" checkbox.
• Interactive environment with multiple window support.

Related material

Related tickets:

• RITM0616403
• RITM0616558

Related issues:

• rc/cheaha#15
• rc/cluster-software#76 (thanks @prema!)

Related projects:

• rc-data-science/facilitation-projects/cryoem-needs>

added Backlog label

changed the description

I had forgotten the detail for a working email notification checkbox, and have added it to the issue.

changed the description

Adding this link for future reference related to Relion issues dealing with CUDA and MPI errors: https://gitlab.rc.uab.edu/rc/cluster-software/-/issues/76.

changed the description

changed the description

Script provided by James Kizziah that "just works". This uses a self-installed copy of RELION.

#!/bin/bash

# load required modules
module load FLTK/1.3.5-GCC-8.3.0
module load LibTIFF/4.1.0-GCCcore-10.2.0
module load CMake/3.20.1-GCCcore-10.3.0
module load CUDA/11.3.1
module load GCC/10.3.0
module load GCCcore/10.2.0
module load OpenMPI/4.0.5-GCC-10.2.0
module load cuda11.3/toolkit/11.3.0
module load libjpeg-turbo/2.0.5-GCCcore-10.2.0
module load Ghostscript/9.53.3-GCCcore-10.2.0
module load Anaconda3/2020.11

# set environment
export PATH="$USER_DATA/relion/build/bin:$PATH"
export LD_LIBRARY_PATH="$USER_DATA/relion/build/lib:$LD_LIBRARY_PATH"
export RELION_QSUB_TEMPLATE="$USER_DATA/relion/scripts/qsub.csh"
export RELION_PDFVIEWER_EXECUTABLE="/usr/bin/evince"
# MotionCor2 does not work in current configuration due to CUDA incompatibilty
export RELION_MOTIONCOR2_EXECUTABLE="/data/rc/apps/rc/software/MotionCor2/1.0.5/
MotionCor2_Cuda8.0_v1.0.5"
export RELION_CTFFIND_EXECUTABLE="/share/apps/rc/software/CTFFIND/4.1.8-foss-2016b/bin/ctffind"
export RELION_GCTF_EXECUTABLE="/share/apps/rc/software/Gctf/1.06_devel/bin/Gctf-v1.06_sm_30_cu8.0_x86_64"
export RELION_RESMAP_EXECUTABLE="/share/apps/rc/software/ResMap/1.1.4/ResMap-1.1.4-linux64"

# enter topaz conda env
export PATH="$USER_DATA/.conda/envs/topaz/bin:$PATH"
export RELION_TOPAZ_EXECUTABLE="/data/user/kizziah4/.conda/envs/topaz/bin/topaz"
export RELION_PYTHON_EXECUTABLE="$USER_DATA/.conda/envs/topaz/bin/python"
conda activate topaz

One of James's requests was to have the various software components outlined in the script above be integrated into the interactive app, so users of the software won't need to modify or customize how they use the software. One issue James and I have both encountered is that different users have slightly different workflows, and information on best practices is decentralized, fragmented and hard to keep everyone up-to-date on. Enforcing as much of the best practices as we can, by taking away these choices, should help.

The components needed for a fully-functioning RELION module are:

1. A PDF viewer
   • James has been using evince but we can probably use another one
   • https://wiki.gnome.org/Apps/Evince
2. module load MotionCor2
   • latest is 1.5.0 on EasyBuild
   • license may be more restrictive than in the past, there is a Google form in the way
   • https://emcore.ucsf.edu/ucsf-software
3. module load ctffind
   • latest is 4.1.14 with CUDA support on EasyBuild
   • https://grigoriefflab.umassmed.edu/ctffind4
4. module load gctf
   • we have the latest (and I can't find the origin webpage, it seems to have vanished!)
5. module load resmap
   • this one is 10 years old! It's dead, doesn't support Python 3.x, but still in use so...
6. Topaz
   • Python based, but you can supply both Python executable and Topaz source
   • We don't have to build a Conda env in this case, at least
   • https://github.com/tbepler/topaz

changed the description

Yale's RC group has already done forms like this. These may be helpful when forming the app submit form:

• https://github.com/ycrc/ycrc-ood/blob/main/apps/ycrc_desktop/form.yml.erb
• https://github.com/ycrc/ycrc-ood/blob/main/apps/ycrc_desktop/submit.yml.erb

Development is blocked by rc/cheaha#20 and rc/cheaha#18 and by TurboVNC version (can't crosslink yet).

added Sprint 23-09 label

removed Backlog label

assigned to @ravi89

@ravi89 I believe this is in Knightly. If we don't intent to add to prod, then we could close this one.

FYI, CryoEM is creating their own custom app based on the existing one (from Knightly) that will use a new way of interacting with Relion and optimize cluster resource usage.

unassigned @ravi89