README.md 5.27 KB
Newer Older
Valentin Reis's avatar
Valentin Reis committed
1
#### integration testing
Valentin Reis's avatar
Valentin Reis committed
2

Valentin Reis's avatar
Valentin Reis committed
3
This repository contains integration tests that validate the argo stack.
Valentin Reis's avatar
Valentin Reis committed
4 5 6 7

It leverages the argopkgs repo, but overrides some sources in it with
their master branch counterparts. see file `default.nix` for details.

8
The intended usage is to override (some of) the source(s) with WIP
9
version(s), as part of development or continuous integration. This
10 11
gitlab CI snippets shows how to do this on a nix-enabled runner. Mind
that this setup tracks argotest master.
Valentin Reis's avatar
Valentin Reis committed
12

13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37
### Usage

-   Get Nix: `curl https://nixos.org/nix/install | sh`

-   Use the 'argotest' nix function to provide an environment, among
    which the `argotk.hs` tool, which runs tests. For example, we can
    run default tests using pinned `argopkgs` sources like so:

``` {.nix}
nix-shell -E '{ argotest ? (builtins.fetchGit {
  url = https://xgitlab.cels.anl.gov/argo/argotest.git;
  ref="master";})
  }: (import argotest {}).test' --run 'argotk.hs helloworld'
```

Alternatively, one can get rid of the `--run` option in order to be
provided with an environment in which all the argo components should be
in the `PATH`, along with the `argotk.hs` tool.

The arguments to the argotest function are defined in the default.nix
file in this repository. They all have default values.

For a more involved example, let's run a test in a custom environment.
Here, we'll run the 'helloworld' test in an environment that uses a
local `nrm` source, the master `libnrm` branch and a specific revision
Valentin Reis's avatar
Valentin Reis committed
38 39
of the `containers` branch. We'l'l use the master `argotest` branch for
that.
40 41 42 43

``` {.nix}
nix-shell -E '{ argotest ? (builtins.fetchGit {
    url = https://xgitlab.cels.anl.gov/argo/argotest.git;
Valentin Reis's avatar
Valentin Reis committed
44
    ref="master";})
45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84
}:
(import argotest {
  nrm-src = /path/to/nrm;
  libnrm-src = builtins.fetchGit {
    url = https://xgitlab.cels.anl.gov/argo/libnrm.git;
    ref="master"; };
  containers-src = builtins.fetchGit  {
    url = https://xgitlab.cels.anl.gov/argo/containers.git;
    ref="fancy-branch-name"; 
    rev="commit-revisions-string";
    };
}).test' --run 'argotk.hs helloworld'
```

*WARNING* There are a few things one has to be aware of using this
workflow:

-   Builds may fail if the local source repositories are dirty with old
    build files.

-   Builds may fail if `PWD` is in a `nosuid`-enabled filesystem.

-   Without using the `rev` argument, the `builtins.fetchGit` nix
    command prefetches and buffers its output, with an expiration time
    that ranges ten minutes by default. Use a local checkout if you need
    to modify some of these sources on the fly:

``` {.nix}
nix-shell -E '{ argotest ? /path/to/argotest }:
(import argotest {
  nrm-src = /path/to/nrm;
  # libnrm-src left at default argument value (inherits from argopkgs)
  # containers-src left at default argument value (inherits from argopkgs)
}).test' --run 'argotk.hs helloworld'
```

### Example CI setup

For example, we could setup the CI for the `containers` repo like:

85
``` {.yml}
Valentin Reis's avatar
Valentin Reis committed
86 87 88
integration.test:
  stage: test
  script:
89 90 91 92 93 94
    - nix-shell -E '{ argotest ? (builtins.fetchGit {
          url = https://xgitlab.cels.anl.gov/argo/argotest.git;
          ref="master";})
      }:
      (import argotest { containers-src = ./. ; }).test' 
      --run 'argotk.hs helloworld'
Valentin Reis's avatar
Valentin Reis committed
95 96
  artifacts:
    paths:
97 98 99 100 101 102
      - argotest/_output/cmd_err.log
      - argotest/_output/cmd_out.log
      - argotest/_output/daemon_out.log
      - argotest/_output/daemon_out.log
      - argotest/_output/nrm.log
      - argotest/_output/time.log
Valentin Reis's avatar
Valentin Reis committed
103
    expire_in: 1 week
Valentin Reis's avatar
Valentin Reis committed
104 105 106 107 108 109
  except:
    - /^wip\/.*/
    - /^WIP\/.*/
  tags:
    - integration
```
110

111
### Argotk.hs usage:
112 113

``` {.bash}
114
argotk.hs --help
115 116 117 118 119 120 121 122
```

Output:

``` {.txt}
Usage: argotk.hs COMMAND

Available options:
123
  COMMAND                  Type of action to run
124 125 126
  -h,--help                Show this help text

Available commands:
127 128 129 130 131
  clean                    Clean sockets, logfiles.
  daemon                   Set up and launch the daemon in synchronous mode,
                           with properly cleaned sockets, logfiles.
  application              Setup stack and run an arbitrary command in a
                           container.
Valentin Reis's avatar
Valentin Reis committed
132
  helloworld               Test 1: Setup stack and check that a hello world app
133
                           sends message back to cmd.
Valentin Reis's avatar
Valentin Reis committed
134 135
  perfwrapper              Test 2: Setup stack and check that a hello world app
                           sends message back to cmd. Uses argo-perf-wrapper.
136 137 138
```

``` {.bash}
139
argotk.hs helloworld --help
140 141 142 143
```

``` {.txt}
Usage: argotk.hs helloworld 
Valentin Reis's avatar
Valentin Reis committed
144 145
  Test 1: Setup stack and check that a hello world app sends message back to
  cmd.
146 147 148

Available options:
  -h,--help                Show this help text
149
```
150 151

``` {.bash}
152
argotk.hs application --help
153 154 155
```

``` {.txt}
Valentin Reis's avatar
Valentin Reis committed
156
Usage: argotk.hs application COMMAND
157 158 159 160 161 162 163
  Setup stack and run an arbitrary command in a container.

Available options:
  COMMAND                  Application to run inside the container
  -h,--help                Show this help text
```

164 165 166 167 168
``` {.bash}
argotk.hs daemon --help
```

``` {.txt}
Valentin Reis's avatar
Valentin Reis committed
169
Usage: argotk.hs daemon 
170 171 172 173 174 175 176 177
  Set up and launch the daemon in synchronous mode, with properly cleaned
  sockets, logfiles.

Available options:
  -h,--help                Show this help text
```

### Hacking
178 179 180 181 182

-   edit `.README.md` in place of README.md.

-   the ./shake.hs build file takes care of a few things for the
    development workflow (readme and completion generation).