1. 07 Dec, 2018 1 commit
  2. 05 Dec, 2018 4 commits
  3. 03 Dec, 2018 2 commits
  4. 28 Nov, 2018 1 commit
  5. 27 Nov, 2018 3 commits
    • Swann Perarnau's avatar
      [fix] ensure container has single owner · 31fcc32f
      Swann Perarnau authored
      Ensure that the client that created the container is considered as the
      one owning it, with the consequence that if its command exits, the
      container is destroyed. Also deals with the race issue we had on the cmd
      side.
      31fcc32f
    • Swann Perarnau's avatar
      [refactor/fix] always send process events for run · bb7af5d7
      Swann Perarnau authored
      Current code sends start/exit events when a container is created and
      process_start/process_exit when its already there. Instead, have the
      container start/exit only care about container stuff, and always sends
      the process start/exit events around. That makes the cmd run fsm easier
      to work out.
      
      Changes the message format a tiny bit.
      Fixes some missing stdout/stderr issues we had before.
      bb7af5d7
    • Valentin Reis's avatar
      [Typo] operator whitespace fixes. · f2638563
      Valentin Reis authored
      f2638563
  6. 21 Nov, 2018 1 commit
  7. 24 Oct, 2018 2 commits
    • Swann Perarnau's avatar
      [fix] ensure container has single owner · 926c8302
      Swann Perarnau authored
      Ensure that the client that created the container is considered as the
      one owning it, with the consequence that if its command exits, the
      container is destroyed. Also deals with the race issue we had on the cmd
      side.
      926c8302
    • Swann Perarnau's avatar
      [refactor/fix] always send process events for run · 11d667db
      Swann Perarnau authored
      Current code sends start/exit events when a container is created and
      process_start/process_exit when its already there. Instead, have the
      container start/exit only care about container stuff, and always sends
      the process start/exit events around. That makes the cmd run fsm easier
      to work out.
      
      Changes the message format a tiny bit.
      Fixes some missing stdout/stderr issues we had before.
      11d667db
  8. 23 Oct, 2018 1 commit
  9. 21 Oct, 2018 1 commit
    • Swann Perarnau's avatar
      [refactor] replace upstream comms with msg layer · 0b0ab966
      Swann Perarnau authored
      Replace the fragile upstream communications with the new messaging
      layer, improving the stability and performance of this API.
      
      NOTE: this breaks previous clients
      NOTE: this patch is missing client tracking, to handle children signals.
      0b0ab966
  10. 17 Oct, 2018 1 commit
    • Sridutt Bhalachandra's avatar
      [Feature] Multi- node and process support · 5a41baba
      Sridutt Bhalachandra authored
      Added multi- node and process support that will allow launching of
      multiple processes within a container. This is important for enabling
      use of NRM with MPI applications with multiple processes in a container
      and thus enabling multi-node executions
      
      See Issue #17
      5a41baba
  11. 15 Aug, 2018 1 commit
  12. 14 Aug, 2018 1 commit
  13. 10 Aug, 2018 4 commits
  14. 16 Jul, 2018 1 commit
  15. 21 Dec, 2017 1 commit
  16. 20 Dec, 2017 1 commit
    • Swann Perarnau's avatar
      [feature] Add PowerActuator and update control · 26e9c239
      Swann Perarnau authored
      This patch adds a poweractuator based on rapl settings available through
      the sensor manager. Adding this actuator forces us to use a list of
      actuators in the controller, changing a bit the structure of the code.
      26e9c239
  17. 19 Dec, 2017 3 commits
    • Swann Perarnau's avatar
      [refactor] Move control scheme to its own module · 246edb75
      Swann Perarnau authored
      The "control" part of the NRM is bound to change and become more complex
      in the near future, so move it in its own module.
      
      This refactor also introduce some controller logic. Control is split
      into 3 steps: planning, execution and updates. The goal is to use this
      new code organization as a way to abstract different control policies
      that could be implemented later.
      
      Note that we might at some point move into a "control manager" and a
      bunch of "policies" and "actuators", as a way of matching typical
      control theory vocabulary.
      246edb75
    • Swann Perarnau's avatar
      [fix] Wrong streaming_callback on stderr · dec31967
      Swann Perarnau authored
      Fixes a copy/paste mistake on the name of the callback to trigger on
      stderr events.
      dec31967
    • Swann Perarnau's avatar
      [feature] Implement Application Manager · f43a38d3
      Swann Perarnau authored
      This patch moves the tracking of applications clients of the downstream
      API into a ApplicationManager, that is able to track progress and thread
      management.
      
      This change is necessary in the long term to build a comprehensive
      downstream API and centralize the management of application tracking.
      
      Note that this tracking is currently independent of the container and
      pid tracking, and that might be a problem in the long term.
      f43a38d3
  18. 18 Dec, 2017 2 commits
    • Swann Perarnau's avatar
      [feature] Implement skeleton downstream API · 19c9eb54
      Swann Perarnau authored
      This patch refactors the downstream API to use pub/sub socket pair, like
      the upstream API. This is part of the effort to improve the downstream
      API. See #2.
      
      This patch doesn't touch the client module, which will be adapted in
      future commits.
      19c9eb54
    • Swann Perarnau's avatar
      [refactor] daemon should always bind on sockets · 1391a197
      Swann Perarnau authored
      The way 0MQ works on PUB/SUB sockets, publishers might drop
      messages if subscribers are not detected faster enough. One way to fix
      it is to have the "server" always bind sockets, and the "client" use
      connect. This way, the handshake is initiated properly, and the client
      can publish as soon as the connection is done.
      
      This patch makes the daemon bind on the upstream API and the CLI connect,
      fixing in the process the message dropping we were experiencing before.
      
      Long term, we might have a think of using 2 types of sockets for the
      upstream API: pub/sub for actual events published from the daemon, and
      a REQ/REP or ROUTER/DEALER pair for "commands".
      1391a197
  19. 15 Dec, 2017 2 commits
    • Swann Perarnau's avatar
      [feature] Properly handle run events in order · 957deb8d
      Swann Perarnau authored
      This patch implements a small finite state machine on the cmd side to be
      able to run a command, wait for all of its output, and then exit.
      
      As the daemon can send those message in any order, we need to wait them
      properly, in particular the closing of stdout/stderr before exiting.
      
      This patch also fixes the read_until_close callback creation to ensure
      that the stream EOF is handled as a distinct message.
      957deb8d
    • Swann Perarnau's avatar
      [refactor] Only track container inside the CM · f2bc8b80
      Swann Perarnau authored
      The daemon code was maintaining its own container tracker using pids,
      instead of using the one in the container manager. This patch removes
      this additional tracking, and let the daemon side deal with an actual
      namedtuple.
      f2bc8b80
  20. 14 Dec, 2017 4 commits
    • Swann Perarnau's avatar
      [feature] Add stdout/stderr streaming · 78f63cd4
      Swann Perarnau authored
      This patch adds stdout/stderr streaming capabilities, based on partial
      evaluation of a tornado.iostream callback. The bin/cmd CLI is updated to
      wait until an exit message, although that doesn't guaranty anything on
      message ordering...
      
      The next step is obviously to figure out a message flow that allows the
      CLI to send and receive the command IO properly, in order...
      78f63cd4
    • Swann Perarnau's avatar
      [refactor] Fix container namedtuple · 9afe59c7
      Swann Perarnau authored
      This patch propagates the process object into the container namedtuple,
      fix a couple of bad function calls and adapt the run command handler to
      use that process object instead of just the pid of it.
      9afe59c7
    • Swann Perarnau's avatar
      [fix] Fix missing logging changes · ed33ef6d
      Swann Perarnau authored
      The logging improvement patch missed a few calls.
      ed33ef6d
    • Swann Perarnau's avatar
      [refactor] Use globally configured logger · b66c88ec
      Swann Perarnau authored
      The logging module allow us to configure logging facilities once per
      process using basicConfig, and then to use globally defined, named,
      logger objects. This simplifies access to logger objects, their
      configuration and remove pointers from all objects.
      
      This patch refactor all the logging calls to use a single 'nrm' logger
      object, using those facilities.
      b66c88ec
  21. 13 Dec, 2017 3 commits
    • Swann Perarnau's avatar
      [feature] Add kill command · 63c2dea8
      Swann Perarnau authored
      This patch adds a command to kill the parent process of a container
      based on the container uuid, triggering the death of the container.
      
      The os.kill command interacts pretty badly with the custom built
      children handling, causing us to catch unwanted exceptions in an effort
      to keep the code running. The waitpid code was also missing a bit about
      catching children exiting because of signals, so we fixed that.
      
      At this point, two things should be paid attention to:
        - we don't distinguish properly between a container and a command.
        This will probably cause issues later, as it should be possible to
        launch multiple programs in the same container, and for partitions to
        survive the death of the parent process.
        - the message format is growing more complex, but without any
        component having strong ownership over it. This will probably cause
        stability issues in the long term, as the format complexifies and we
        lose track of the fields expected from everyone.
      63c2dea8
    • Swann Perarnau's avatar
      [feature] Add command to list containers · 2f470afb
      Swann Perarnau authored
      This patch adds a very simple command to list the containers currently
      known by the NRM. There's no history or state tracking on the NRM, so
      the code is pretty simple.
      
      We expect that some of the container tracking doesn't need to be sent
      for such a command, so the listing also filters some of the fields.
      
      This patch also adds an 'event' field to container messages, as it would
      probably be needed further for other kind of operations.
      2f470afb
    • Swann Perarnau's avatar
      [feature] Implement simple RM for containers · 1c4645cc
      Swann Perarnau authored
      This patch refactor the resource management and hwloc code into a
      working, albeit very simple scheduling policy. Indeed, the previous code
      contained strong assumptions about the output of hwloc matching an Argo
      NodeOS configuration used during the previous phase of the project, that
      always contained enough CPUs and Mems to perform exclusive scheduling.
      
      The current version is simpler, but should work on more regular systems.
      The patch also improves code organization so that introducing more
      complex scheduling algorithms will be simpler.
      
      The testing of this code resulted in the discovery of simple bugs in the
      daemon children handling code, which should work now.
      1c4645cc