busy-spin latency on cooly really high
This command takes longer than 20 minutes to run on Cooley:
mpirun -f $COBALT_NODEFILE -n 2 numactl -N 1 -m 1 ./margo-p2p-latency -i 100000 -n "ofi+verbs;ofi_rxm://mlx5_0:3339" -t 0,0The corresponding test in the no-spin case is shorter:
mpirun -f $COBALT_NODEFILE -n 2 numactl -N 1 -m 1 ./margo-p2p-latency -i 100000 -n "verbs://"
# <op> <iterations> <warmup_iterations> <size> <min> <q1> <med> <avg> <q3> <max>
noop 100000 100 0 0.000017643 0.000020266 0.000020742 0.000023233 0.000026464 0.000243187Not sure when that first started happening, but has been going on since the nightly tests got going again in November