GitLab

* User explicitly creates balsam DB directory with "balsam init"
* User launches DB server with "balsam dbserver"
* user_settings is vastly simplified...moved all configuration logic
into django_config module
* Database is auto-configured on django startup; will connect to DB
server automatically if client

* user kill works even when a kill save() takes a long time in flight
  and some other processing happens first
* updated functional tests
* all tests passing on Theta, Cooley

Added test cases

concurrently; they block by queueing up at the DB-writer, which is a ZMQ
proxy to the Sqlite database

* dag.spawn_child now clones jobs in a more sane way; fixed bug related
to job_id being sent over the wire as None

* improved logging

* Fixed deserialize bug when job_id is None (for new jobs)

* added test case for concurrent BalsamJob insertions to database from a
single mpi4py applicaiton, where all ranks are concurrently calling
save() with new jobs

cli_commands: fixed small user error message bug
model saving: more robust retry: wait and check if save succeeded in
spite of the exception

An mpi4py test program with the Python sqlite3 wrapper has no trouble
with concurrent DB insertions from several tens of processes.  The issue
appears to be in the Django-sqlite3 wrapper.

This patch simply wraps save() in a loop that intercepts
OperationalError until the save is successful.

This reverts commit 74ce64c5

than 1.  Added Functional Test cases for running several hybrid
MPI/OpenMP jobs back-to-back from the launcher; exercising various
combinations of ranks-per-node, threads-per-rank, threads-per-core
(-d and -j options of aprun)

These tests will only run on Theta right now; can extend by recompiling
c_apps/omp.c on different platforms and referring to the appropriate
binary in the test case.

the job.

BalsamJob Model:
If a RecordModifiedError arises while killing, the job
state is always overwritten as USER_KILLED

If RecordModifiedError arises when trying to update the job to
some other state, it is ignored and the job is left in USER_KILLED
state.

end-state.

The get_envs() method of BalsamJobs only produces environment variables
that contain a BALSAM or DJANGO substring now.

substrings not containing dashes.  It does not work
for substrings containing a dash, but this is not an expected usage.

(e.g. balsam ls --state JOB_FINISHED)

Fixed bug (inconsistent variable naming) which caused ranks_per_node
not to be assigned correctly by "balsam qsub" command line