Cluster node state change accounting log

See merge request aig/cobalt!125

Merge remote-tracking branch 'refs/remotes/origin/cluster_node_state_change' into cluster_node_state_change

removing location from the email subject.  The subject line is too long on...

See merge request aig/cobalt!124

Make cluster systems respect --attrs location

See merge request aig/cobalt!123

Enhance reservation record information

See merge request aig/cobalt!122

removing location from the email subject.  The subject line is too long on large systems, otherwise.

fix for a display issue with cluster node names that have additional device information.

See merge request aig/cobalt!121

Addressed comments from code review and found a further corner
case while working on those with one-off nodenames that sometimes
get added to clusters.  Tests added and behavior fixed for those.

Resolve "Cray system draining flap"

Closes #158

See merge request aig/cobalt!120

A node was getting added to the candidate list twice when looking over
currently running jobs. resulting in a duplicate node and an inflated
count which resulted in an early break.  When this list is later
converted to a set, the duplicates are removed and results in a set
of nodes too small for the job to use for draining, so the job doesn't
get drained for.

Test case added for the drain flap now works correctly now that a check
has been added so a duplicate node doesn't get added.

Lock doesn't need to be held around this state fetch

Closes #178

See merge request aig/cobalt!119

Resolve "schedule_jobs_interval not respected"

Closes #177

See merge request aig/cobalt!118

This is to improve Cobalt responsiveness during state update.  Combined
with the fix for aig/cobalt#177, this should allow for much faster
scheduling cadences as well and should significantly reduce the delay in
a number of commands.

There is an additional fix here where the check for a starting job
during the cleanup update wasn't working.  This was due to a hidden type
mismatch.  This was exposed by both speedups put together.

This allows the site to configure the base sleeptime between automatic
checks on a per-component basis.

This removes the argument and relies on the config file.  Nothing was
setting the option prior to this, and this is a more consistient way to
handle this configuration.

Preventing a doubleing of cycle time if deferring an active reservation.

Closes #176

See merge request aig/cobalt!117