Commit fc11c8b6 authored by Justs Zarins's avatar Justs Zarins
Browse files

Clean up readme and add original dataset and some useful info

parent 5c6d9b9a
1) Use
bash get_code.sh
to get code and apply modifications.
## Replicate the GPCNeT plot in the paper
2) cd gpcnet
* `python3 analyse_data.py 256probes_256congestors_bw_ad0_ad3.pickle`
3) Modify Makefile for your compilation environment and run
make all
* If you encounter issues, e.g. with Latex, you can do `cp stub_Common_lib.py Common_lib.py`. This will only affect figure aesthetics and can be undone using `git checkout Common_lib.py`.
4) Modify submit.cobalt to suit your submission system and change "benchmark" variable to point to "network_load_test" in your GPCNeT directory.
## Generate the same kind of data used in the GPCNeT plot in the paper
5) qsub submit.cobalt
1. `bash get_code.sh` to get the code used in the GPCNeT experiments.
2. `cd gpcnet`
3. Modify the `Makefile` for your compilation environment.
4. `make all`
5. Modify `submit.cobalt` to suit your submission system and change "benchmark" variable to point to "network_load_test" executable in your GPCNeT directory.
6. `qsub test_job_theta.cobalt` or similar to launch the job on your system.
6. `python3 gather_data.py ./Probe_ADAPTIVE_*__nnodes_512/summary.out` to process the raw data
7. `python3 analyse_data.py dataset.pickle` to generate the kind of plot used in the paper
6) python3 gather_data.py ./Probe_ADAPTIVE_*__nnodes_512/summary.out
7) To generate plot:
python3 analyse_data.py dataset.pickle
......@@ -250,7 +250,6 @@ def plot_zdata_data_probes_and_congs_individual(df, num_nodes):
def get_resampled_kde(binv, binc):
num_samples = 100000
print("Drawing", str(num_samples), "samples for plot. For final version this should be 100'000.")
np.random.seed(20201337)
resamples = np.random.choice(binv, size=num_samples, p=binc/binc.sum())
return resamples
......@@ -262,7 +261,7 @@ def set_more_printout_pandas():
pd.set_option('display.width', 1000)
def pickout_data(df):
def pick_paper_data(df):
df2 = df.loc[(df.benchmark_name == "RR Two-sided BW") & (df.cong == "yes")]
return df2.loc[(df2.p_ar == "ADAPTIVE_0") | (df2.p_ar == "ADAPTIVE_3")]
......@@ -270,7 +269,7 @@ def pickout_data(df):
# Main entry point
if __name__ == "__main__":
df = load_dataframe(sys.argv[1])
df = pickout_data(df)
df = pick_paper_data(df)
if df.isnull().values.any():
print("There are NULL values in the dataframe. Aborting.")
......
Loaded modules when compiling the code for the GPCNeT experiments presented in the paper:
1) modules/3.2.11.3
2) intel/19.0.5.281
3) craype-network-aries
4) craype/2.6.1
5) cray-libsci/19.06.1
6) udreg/2.3.2-6.0.7.1_5.15__g5196236.ari
7) ugni/6.0.14.0-6.0.7.1_3.15__gea11d3d.ari
8) pmi/5.0.14
9) dmapp/7.1.1-6.0.7.1_6.6__g45d1b37.ari
10) gni-headers/5.0.12.0-6.0.7.1_3.13__g3b1768f.ari
11) xpmem/2.2.15-6.0.7.1_5.13__g7549d06.ari
12) job/2.2.3-6.0.7.1_5.48__g6c4e934.ari
13) dvs/2.7_2.2.120-6.0.7.1_12.5__g74cb2cc4
14) alps/6.6.43-6.0.7.1_5.51__ga796da32.ari
15) rca/2.2.18-6.0.7.1_5.52__g2aa4f39.ari
16) atp/2.1.3
17) PrgEnv-intel/6.0.5
18) craype-mic-knl
19) cray-mpich/7.7.10
20) nompirun/nompirun
21) darshan/3.1.5
22) trackdeps
23) xalt
24) papi/5.7.0.2
def init_plotting():
pass
def hide_spines():
pass
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment