README.md 5.27 KB
Newer Older
Valentin Reis's avatar
Valentin Reis committed
1
#### integration testing
Valentin Reis's avatar
Valentin Reis committed
2

Valentin Reis's avatar
Valentin Reis committed
3
This repository contains integration tests that validate the argo stack.
Valentin Reis's avatar
Valentin Reis committed
4
5
6
7

It leverages the argopkgs repo, but overrides some sources in it with
their master branch counterparts. see file `default.nix` for details.

Valentin Reis's avatar
Valentin Reis committed
8
The intended usage is to override (some of) the source(s) with WIP
9
version(s), as part of development or continuous integration. This
Valentin Reis's avatar
Valentin Reis committed
10
11
gitlab CI snippets shows how to do this on a nix-enabled runner. Mind
that this setup tracks argotest master.
Valentin Reis's avatar
Valentin Reis committed
12

13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
### Usage

-   Get Nix: `curl https://nixos.org/nix/install | sh`

-   Use the 'argotest' nix function to provide an environment, among
    which the `argotk.hs` tool, which runs tests. For example, we can
    run default tests using pinned `argopkgs` sources like so:

``` {.nix}
nix-shell -E '{ argotest ? (builtins.fetchGit {
  url = https://xgitlab.cels.anl.gov/argo/argotest.git;
  ref="master";})
  }: (import argotest {}).test' --run 'argotk.hs helloworld'
```

Alternatively, one can get rid of the `--run` option in order to be
provided with an environment in which all the argo components should be
in the `PATH`, along with the `argotk.hs` tool.

The arguments to the argotest function are defined in the default.nix
file in this repository. They all have default values.

For a more involved example, let's run a test in a custom environment.
Here, we'll run the 'helloworld' test in an environment that uses a
local `nrm` source, the master `libnrm` branch and a specific revision
Valentin Reis's avatar
Valentin Reis committed
38
39
of the `containers` branch. We'l'l use the master `argotest` branch for
that.
40
41
42
43

``` {.nix}
nix-shell -E '{ argotest ? (builtins.fetchGit {
    url = https://xgitlab.cels.anl.gov/argo/argotest.git;
Valentin Reis's avatar
Valentin Reis committed
44
    ref="master";})
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
}:
(import argotest {
  nrm-src = /path/to/nrm;
  libnrm-src = builtins.fetchGit {
    url = https://xgitlab.cels.anl.gov/argo/libnrm.git;
    ref="master"; };
  containers-src = builtins.fetchGit  {
    url = https://xgitlab.cels.anl.gov/argo/containers.git;
    ref="fancy-branch-name"; 
    rev="commit-revisions-string";
    };
}).test' --run 'argotk.hs helloworld'
```

*WARNING* There are a few things one has to be aware of using this
workflow:

-   Builds may fail if the local source repositories are dirty with old
    build files.

-   Builds may fail if `PWD` is in a `nosuid`-enabled filesystem.

-   Without using the `rev` argument, the `builtins.fetchGit` nix
    command prefetches and buffers its output, with an expiration time
    that ranges ten minutes by default. Use a local checkout if you need
    to modify some of these sources on the fly:

``` {.nix}
nix-shell -E '{ argotest ? /path/to/argotest }:
(import argotest {
  nrm-src = /path/to/nrm;
  # libnrm-src left at default argument value (inherits from argopkgs)
  # containers-src left at default argument value (inherits from argopkgs)
}).test' --run 'argotk.hs helloworld'
```

### Example CI setup

For example, we could setup the CI for the `containers` repo like:

85
``` {.yml}
Valentin Reis's avatar
Valentin Reis committed
86
87
88
integration.test:
  stage: test
  script:
89
90
91
92
93
94
    - nix-shell -E '{ argotest ? (builtins.fetchGit {
          url = https://xgitlab.cels.anl.gov/argo/argotest.git;
          ref="master";})
      }:
      (import argotest { containers-src = ./. ; }).test' 
      --run 'argotk.hs helloworld'
Valentin Reis's avatar
Valentin Reis committed
95
96
  artifacts:
    paths:
97
98
99
100
101
102
      - argotest/_output/cmd_err.log
      - argotest/_output/cmd_out.log
      - argotest/_output/daemon_out.log
      - argotest/_output/daemon_out.log
      - argotest/_output/nrm.log
      - argotest/_output/time.log
Valentin Reis's avatar
Valentin Reis committed
103
    expire_in: 1 week
Valentin Reis's avatar
Valentin Reis committed
104
105
106
107
108
109
  except:
    - /^wip\/.*/
    - /^WIP\/.*/
  tags:
    - integration
```
110

111
### Argotk.hs usage:
112
113

``` {.bash}
114
argotk.hs --help
115
116
117
118
119
120
121
122
```

Output:

``` {.txt}
Usage: argotk.hs COMMAND

Available options:
123
  COMMAND                  Type of action to run
124
125
126
  -h,--help                Show this help text

Available commands:
127
128
129
130
131
  clean                    Clean sockets, logfiles.
  daemon                   Set up and launch the daemon in synchronous mode,
                           with properly cleaned sockets, logfiles.
  application              Setup stack and run an arbitrary command in a
                           container.
Valentin Reis's avatar
Valentin Reis committed
132
  helloworld               Test 1: Setup stack and check that a hello world app
133
                           sends message back to cmd.
Valentin Reis's avatar
Valentin Reis committed
134
135
  perfwrapper              Test 2: Setup stack and check that a hello world app
                           sends message back to cmd. Uses argo-perf-wrapper.
136
137
138
```

``` {.bash}
139
argotk.hs helloworld --help
140
141
142
143
```

``` {.txt}
Usage: argotk.hs helloworld 
Valentin Reis's avatar
Valentin Reis committed
144
145
  Test 1: Setup stack and check that a hello world app sends message back to
  cmd.
146
147
148

Available options:
  -h,--help                Show this help text
149
```
150
151

``` {.bash}
152
argotk.hs application --help
153
154
155
```

``` {.txt}
Valentin Reis's avatar
Valentin Reis committed
156
Usage: argotk.hs application COMMAND
157
158
159
160
161
162
163
  Setup stack and run an arbitrary command in a container.

Available options:
  COMMAND                  Application to run inside the container
  -h,--help                Show this help text
```

164
165
166
167
168
``` {.bash}
argotk.hs daemon --help
```

``` {.txt}
Valentin Reis's avatar
Valentin Reis committed
169
Usage: argotk.hs daemon 
170
171
172
173
174
175
176
177
  Set up and launch the daemon in synchronous mode, with properly cleaned
  sockets, logfiles.

Available options:
  -h,--help                Show this help text
```

### Hacking
178
179
180
181
182

-   edit `.README.md` in place of README.md.

-   the ./shake.hs build file takes care of a few things for the
    development workflow (readme and completion generation).