Commit 7477ba06 authored by Valentin Reis's avatar Valentin Reis
Browse files

Initial commit

parents
Pipeline #5023 failed with stage
in 0 seconds
#### integration testing
Integration tests that validate the argo stack, leveraging the 'argopkgs'
repository. The intended usage is to override (some of) the source(s) with WIP
version(s), as part of development or continuous integration.
### Usage (tl;dr, I already have nix on my machine.)
```bash
nix-shell -E '
(import( builtins.fetchGit {
url = https://xgitlab.cels.anl.gov/argo/argotest.git;
ref="master";
}) {
#nrm-src = /path/to/nrm
#libnrm-src = /path/to/nrm
#containers-src = /path/to/nrm
}).test' --run "argotk.hs TestHello"
```
### Usage (in three parts)
- [**1**] Get Nix: `curl https://nixos.org/nix/install | sh`
- [**2**] Use the `test` attribute of the argotest' nix attribute set to enter a
test environment. For example, we can run default tests on the
"argopkgs-pinned" version of argo components using:
```bash
nix-shell -E '
let
argotest-src =
builtins.fetchGit {
url = https://xgitlab.cels.anl.gov/argo/argotest.git;
ref="master";
};
argotest = import argotest-src {};
in
argotest.test'
```
This environment has all the necessary Argo components in its PATH. The
argotest function has various arguments, defined in the default.nix file at the
rooto of this repository. They all have default values. For a more involved
example, let's get a custom test environment.
Here, we'll use an environment that uses a local `nrm` source, the master
`libnrm` branch and a specific revision of the `containers` branch. We'l'l use
the master `argotest` branch for that:
```nix
nix-shell -E '{ argotest ? (builtins.fetchGit {
url = https://xgitlab.cels.anl.gov/argo/argotest.git;
ref="master";})
}:
(import argotest {
nrm-src = /path/to/nrm;
libnrm-src = builtins.fetchGit {
url = https://xgitlab.cels.anl.gov/argo/libnrm.git;
ref="master"; };
containers-src = builtins.fetchGit {
url = https://xgitlab.cels.anl.gov/argo/containers.git;
ref="fancy-branch-name";
rev="commit-revisions-string";
};
}).test' --run 'argotk.hs TestHello'
```
- [**3**] The `test`environment contains the `argotk.hs` tool, which runs various
operations on the argo stack:
Commands list:
```{.bash}
argotk.hs --help
```
Output:
```{.txt pipe="sh"}
root/argotk/argotk.hs --help
```
Detailed help:
```{.bash}
argotk.hs TestHello --help
```
Output:
```{.txt pipe="sh"}
root/argotk/argotk.hs TestHello --help
```
#### Misc
Alternatively, one can use the `--run` option to run a test directly:
```bash
nix-shell -E '
(import( builtins.fetchGit {
url = https://xgitlab.cels.anl.gov/argo/argotest.git;
ref="master";
}) {}).test' --run "argotk.hs TestHello"
```
#### WARNINGS
There are a few things one has to be aware of using this workflow:
- Builds may fail if the local source repositories are dirty with old build files.
- Without using the `rev` argument, the `builtins.fetchGit` nix command
prefetches and buffers its output, with an expiration time that ranges ten
minutes by default. Use a local checkout if you need to modify some of these
sources on the fly.
### Hacking
- edit `.README.md` in place of README.md.
- the ./shake.hs build file takes care of a few things for the development
workflow (readme and completion generation).
.argo_nodeos_config_exit_message
_output
result
.shake
*.log
*/build
*/new-build
*/dist
*/new-dist
*/result
_output
*/_output
stages:
- test
locally-sourced-tests.test:
stage: test
script:
- nix-shell -A test --run "argotk.hs tests"
except:
- /^wip\/.*/
- /^WIP\/.*/
artifacts:
when: always
paths:
- _output/*.log
- _output/.argo_nodeos_config_exit_message
tags:
- integration
all-test-provenances.test:
stage: test
script:
- nix-build -A test-list
except:
- /^wip\/.*/
- /^WIP\/.*/
artifacts:
when: always
paths:
- result
tags:
- integration
# Revision history for argo
## 0.1.0.0 -- YYYY-mm-dd
* First version. Released on an unsuspecting world.
Copyright (c) 2018 Valentin Reis
Permission is hereby granted, free of charge, to any person obtaining
a copy of this software and associated documentation files (the
"Software"), to deal in the Software without restriction, including
without limitation the rights to use, copy, modify, merge, publish,
distribute, sublicense, and/or sell copies of the Software, and to
permit persons to whom the Software is furnished to do so, subject to
the following conditions:
The above copyright notice and this permission notice shall be included
in all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.
IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY
CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT,
TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE
SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
#### integration testing
Integration tests that validate the argo stack, leveraging the
'argopkgs' repository. The intended usage is to override (some of) the
source(s) with WIP version(s), as part of development or continuous
integration.
### Usage (tl;dr, I already have nix on my machine.)
``` {.bash}
nix-shell -E '
(import( builtins.fetchGit {
url = https://xgitlab.cels.anl.gov/argo/argotest.git;
ref="master";
}) {
#nrm-src = /path/to/nrm
#libnrm-src = /path/to/nrm
#containers-src = /path/to/nrm
}).test' --run "argotk.hs TestHello"
```
### Usage (in three parts)
- \[**1**\] Get Nix: `curl https://nixos.org/nix/install | sh`
- \[**2**\] Use the `test` attribute of the argotest' nix attribute
set to enter a test environment. For example, we can run default
tests on the "argopkgs-pinned" version of argo components using:
``` {.bash}
nix-shell -E '
let
argotest-src =
builtins.fetchGit {
url = https://xgitlab.cels.anl.gov/argo/argotest.git;
ref="master";
};
argotest = import argotest-src {};
in
argotest.test'
```
This environment has all the necessary Argo components in its PATH. The
argotest function has various arguments, defined in the default.nix file
at the rooto of this repository. They all have default values. For a
more involved example, let's get a custom test environment.
Here, we'll use an environment that uses a local `nrm` source, the
master `libnrm` branch and a specific revision of the `containers`
branch. We'l'l use the master `argotest` branch for that:
``` {.nix}
nix-shell -E '{ argotest ? (builtins.fetchGit {
url = https://xgitlab.cels.anl.gov/argo/argotest.git;
ref="master";})
}:
(import argotest {
nrm-src = /path/to/nrm;
libnrm-src = builtins.fetchGit {
url = https://xgitlab.cels.anl.gov/argo/libnrm.git;
ref="master"; };
containers-src = builtins.fetchGit {
url = https://xgitlab.cels.anl.gov/argo/containers.git;
ref="fancy-branch-name";
rev="commit-revisions-string";
};
}).test' --run 'argotk.hs TestHello'
```
- \[**3**\] The `test`environment contains the `argotk.hs` tool, which
runs various operations on the argo stack:
Commands list:
``` {.bash}
argotk.hs --help
```
Output:
``` {.txt}
Usage: argotk.hs COMMAND
Available options:
COMMAND Type of test to run. There are extensive options
under each action, but be careful, these do not all
have the same defaults. The default values are
printed when you call --help on these actions.
-h,--help Show this help text
Available commands:
clean Clean sockets, logfiles.
daemon-only Set up and launch the daemon in synchronous mode,
with properly cleaned sockets, logfiles.
full-stack Setup stack and run a command in a container.
TestHello test1: Setup stack and check that a hello world app
sends message back to cmd.
TestListen test2: Setup stack and check that argo-perf-wrapper
sends at least one message to the daemon.
TestPerfwrapper test3: Setup stack and check that argo-perf-wrapper
sends at least one *performance* message to cmd
listen through the daemon.
TestPower test4: Setup stack and check that argo-perf-wrapper
sends at least one *power* message to cmd listen
through the daemon.
tests Run all tests
```
Detailed help:
``` {.bash}
argotk.hs TestHello --help
```
Output:
``` {.txt}
Usage: argotk.hs TestHello [--application APP]
[--container_name ARGO_CONTAINER_UUID]
[--output_dir DIR] [--manifest_directory DIR]
[--manifest_name FILENAME] [--cmd_run_out FILENAME]
[--cmd_run_err FILENAME] [--cmd_listen_out FILENAME]
[--cmd_listen_err FILENAME] [--daemon_out FILENAME]
[--daemon_err FILENAME] [--nrm_log FILENAME]
[--message_daemon_stdout STRING]
[--message_daemon_stderr STRING]
[--message_cmd_run_stdout STRING]
[--message_cmd_run_stderr STRING]
[--message_cmd_listen_stdout STRING]
[--message_cmd_listen_stderr STRING]
test1: Setup stack and check that a hello world app sends message back to cmd.
Available options:
--application APP Target application executable name. PATH is
inherited. (default: "echo")
--container_name ARGO_CONTAINER_UUID
Container name (default: "testContainer")
--output_dir DIR Working directory. (default: FilePath "_output")
--manifest_directory DIR Manifest lookup
directory (default: FilePath "/nix/store/9hbj77178wrp2pv8qs6d063afhdymgk2-manifests")
--manifest_name FILENAME Manifest file basename (relative to
--manifest_directory) (default: FilePath "basic.json")
--cmd_run_out FILENAME Output file (relative to --output_dir), "cmd run"
stdout (default: FilePath "cmd_run_out.log")
--cmd_run_err FILENAME Output file (relative to --output_dir), "cmd run"
stderr (default: FilePath "cmd_run_err.log")
--cmd_listen_out FILENAME
Output file (relative to --output_dir), "cmd listen"
stdout (default: FilePath "cmd_listen_out.log")
--cmd_listen_err FILENAME
Output file (relative to --output_dir), "cmd listen"
stderr (default: FilePath "cmd_listen_err.log")
--daemon_out FILENAME Output file (relative to --output_dir), daemon
stdout (default: FilePath "daemon_out.log")
--daemon_err FILENAME Output file (relative to --output_dir), daemon
stderr (default: FilePath "daemon_err.log")
--nrm_log FILENAME Output file (relative to --output_dir), daemon
log (default: FilePath "nrm.log")
--message_daemon_stdout STRING
The appearance of this character string in the daemon
stdout will be monitored during execution. When
observed, the stack will be killed and a return code
of 0 will be returned.
--message_daemon_stderr STRING
The appearance of this character string in the daemon
stdout will be monitored during execution. When
observed, the stack will be killed and a return code
of 0 will be returned.
--message_cmd_run_stdout STRING
The appearance of this character string in the cmd
run stdout will be monitored during execution. When
observed, the stack will be killed and a return code
of 0 will be
returned. (default: "someComplicatedMessage")
--message_cmd_run_stderr STRING
The appearance of this character string in the cmd
run stdout will be monitored during execution. When
observed, the stack will be killed and a return code
of 0 will be
returned. (default: "someComplicatedMessage")
--message_cmd_listen_stdout STRING
The appearance of this character string in the cmd
listen stdout will be monitored during execution.
When observed, the stack will be killed and a return
code of 0 will be returned.
--message_cmd_listen_stderr STRING
The appearance of this character string in the cmd
listen stdout will be monitored during execution.
When observed, the stack will be killed and a return
code of 0 will be returned.
-h,--help Show this help text
```
#### Misc
Alternatively, one can use the `--run` option to run a test directly:
``` {.bash}
nix-shell -E '
(import( builtins.fetchGit {
url = https://xgitlab.cels.anl.gov/argo/argotest.git;
ref="master";
}) {}).test' --run "argotk.hs TestHello"
```
#### WARNINGS
There are a few things one has to be aware of using this workflow:
- Builds may fail if the local source repositories are dirty with old
build files.
- Without using the `rev` argument, the `builtins.fetchGit` nix
command prefetches and buffers its output, with an expiration time
that ranges ten minutes by default. Use a local checkout if you need
to modify some of these sources on the fly.
### Example CI setup
``` {.yml}
integration.test:
stage: test
script:
- nix-shell -E '{ argotest ? (builtins.fetchGit {
url = https://xgitlab.cels.anl.gov/argo/argotest.git;
ref="master";})
}:
(import argotest { containers-src = ./. ; }).test' \
--run 'argotk.hs TestHello'
artifacts:
paths:
- argotest/_output/cmd_err.log
- argotest/_output/cmd_out.log
- argotest/_output/daemon_out.log
- argotest/_output/daemon_out.log
- argotest/_output/nrm.log
- argotest/_output/time.log
expire_in: 1 week
except:
- /^wip\/.*/
- /^WIP\/.*/
tags:
- integration
```
### Hacking
- edit `.README.md` in place of README.md.
- the ./shake.hs build file takes care of a few things for the
development workflow (readme and completion generation).
{
"fetch": {
"args": {
"fetchSubmodules": false,
"rev": "a7e6a1136bf87d4e3ba51749c7f142f310536796",
"sha256": "1l6i4lq2zh1hj7zwywvckskfp3lrw06bn8x2h9ip8rq4lby88s5x",
"url": "https://xgitlab.cels.anl.gov/argo/argopkgs.git"
},
"fn": "fetchgit"
},
"rev": "refs/heads/master",
"type": "fetchgit",
"url": "https://xgitlab.cels.anl.gov/argo/argopkgs.git"
}
\ No newline at end of file
name: argotk
version: 0.1.0.0
license: MIT
license-file: LICENSE
author: Valentin Reis
maintainer: fre@freux.fr
build-type: Simple
extra-source-files: ChangeLog.md
cabal-version: >=1.10
executable argotk
main-is: argotk.hs
-- other-modules:
-- other-extensions:
build-depends: base, shake, turtle, data-default, async, unix, text, optparse-applicative, foldl, ansi-terminal, conduit, process, unliftio-core, conduit-extra, bytestring, system-filepath, pretty-show
hs-source-dirs: src
default-language: Haskell2010
ghc-options:
-Wall
-Wincomplete-uni-patterns
-Wincomplete-record-updates
-Wmissing-home-modules
-Widentities
-Wredundant-constraints
-Wcpp-undef
-fwarn-tabs
-fwarn-unused-imports
-fwarn-missing-signatures
-fwarn-name-shadowing
-fprint-potential-instances
-Wmissing-export-list
-fwarn-incomplete-patterns
_argotk.hs()
{
local CMDLINE
local IFS=$'\n'
CMDLINE=(--bash-completion-index $COMP_CWORD)
for arg in ${COMP_WORDS[@]}; do
CMDLINE=(${CMDLINE[@]} --bash-completion-word $arg)
done
COMPREPLY=( $(argotk.hs "${CMDLINE[@]}") )
}
complete -o filenames -F _argotk.hs argotk.hs
{
hostPkgs ? import <nixpkgs> {},
pkgs ? import (hostPkgs.nix-update-source.fetch ./argopkgs.json).src {},
}:
let
filterHdevTools = builtins.filterSource (path: type: baseNameOf path != ".hdevtools.sock");
hpkgs = pkgs.haskellPackages.override {
overrides = self: super: rec {
argotk = self.callCabal2nix "argotk" (filterHdevTools ./.) {};
};
};
devInputs = with hpkgs; with pkgs; [
sysstat
];
devHPackages = with hpkgs; [
cabal-install
apply-refact
hdevtools
hindent
hlint
shake
brittany
ghcid
];
in rec
{
inherit(hpkgs) argotk;
hack = hpkgs.shellFor {
packages = p: with p; [ argotk ];
withHoogle = true;
buildInputs = devInputs ++ devHPackages;
};
}
#! /usr/bin/env runhaskell
import Development.Shake
import Development.Shake.FilePath
import Control.Monad
main = shakeArgs shakeOptions $ do
phony "clean" $ do
removeFilesAfter "completion" ["//*"]
removeFilesAfter "." ["README.md"]
want ["README.md", "completion/argotk.sh"]
"completion/*.sh"
%> \out -> mkCompletionRule out "bash" $ takeFileName out -<.> "hs"
"README.md" %> \out -> do
let template = ".README.md"
need [template , "argotk/argotk.hs" , "argo/src/Argo/Stack.hs"]
(Stdout panpipe) <- cmd "which panpipe"
cmd_ "pandoc --filter"
[take (length panpipe - 1) panpipe, template, "-o", out]
where
mkCompletionAction str sn =
(sn, cmd ("argotk/" ++ sn) ["--" ++ str ++ "-completion-script", sn])
mkCompletionRule out str fn = do
let (needed, cplA) = mkCompletionAction str fn
(Stdout cplScript) <- cplA
liftIO $ writeFile out cplScript
{-|
Module : Argo
Description : The holt core package
Copyright : (c) Valentin Reis, 2018
License : MIT
Maintainer : fre@freux.fr
-}
module Argo
( module Argo.Stack
, module Argo.Args
, module Argo.Utils
)
where
import Argo.Stack
import Argo.Utils
import Argo.Args
{-# LANGUAGE OverloadedStrings, ApplicativeDo, GeneralizedNewtypeDeriving, RecordWildCards #-}
module Argo.Args where
import Options.Applicative as OA
import Options.Applicative.Types
import Options.Applicative.Builder ( option )
import Data.Default
import Data.Text as T
hiding ( empty )
import Turtle hiding ( option )
import Prelude hiding ( FilePath )
data StackArgs = StackArgs
{ verbosity :: Verbosity
, app :: AppName
, args :: [AppArg]
, containerName :: ContainerName
, workingDirectory :: WorkingDirectory
, manifestDir :: ManifestDir
, manifestName :: ManifestName
, daemon :: ProcessBehavior
, cmdrun :: ProcessBehavior
, cmdlisten :: ProcessBehavior
, cmdlistenprogress :: ProcessBehavior
, cmdlistenperformance :: ProcessBehavior
, cmdlistenpower :: ProcessBehavior
} deriving (Show)
{-data OutputFiles = OutputFiles FilePath FilePath-}
data Verbosity = Normal | Verbose deriving (Show,Read,Eq)
newtype AppArg = AppArg Text deriving (IsString, Show, Read)
newtype WorkingDirectory = WorkingDirectory FilePath deriving (IsString, Show)
newtype AppName = AppName Text deriving (IsString, Show, Read)
newtype ContainerName = ContainerName Text deriving (IsString, Show, Read)
newtype ManifestDir = ManifestDir FilePath deriving (IsString, Show)
newtype ManifestName = ManifestName FilePath deriving (IsString, Show)
data ProcessBehavior =
Test TestText StdOutLog StdErrLog
| JustRun StdOutLog StdErrLog
| DontRun deriving (Show,Read)