Develop a production branch
All of the scripts for the pipeline are stored in src[/subdir]
right now, but many of them that run inside jobs reference out
and err
directories in their parent dir. This means job outputs and source code are mixing which needs to be solved.
Need to think about how a production branch should be organized, what should be kept when a new version of the production branch is pulled, possibly how to distribute this pipeline in the first place. Should it be a clone of the repository? Or should it be packaged as a library instead? There are issues with any solution particularly concerning the sudo
permissions currently being granted to the src/run-policy/run-submit-pol-job.py
script.
Need to do some more research on how other projects have handled this since it's not a unique problem to us