1.62 KB
Newer Older
Matthieu Dorier's avatar
Matthieu Dorier committed
1 2 3 4 5 6 7 8
REsource Migration Interface

REMI is a Mochi microservice designed to handle the migration of sets of files
from a node to another. It uses RDMA and memory mapping to efficiently transfer
potentially large groups of files at once.

### Installing
Matthieu Dorier's avatar
Matthieu Dorier committed
9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41

Just like all Mochi services, REMI can be installed using Spack. Once you have
clone the [sds-repo]( package repository
and added it to your spack installation, you can install REMI using the following

spack install remi

REMI depends on [Thallium](, which
Spack will install (if needed) along with Thallium's own dependencies.

### Overview

REMI works with _filesets_. A fileset consists of a root directory and
a set of file paths relative to this root directory. A fileset is also characterized
by the name of its _migration class_.

REMI clients create filesets to group files corresponding to a particular resource
(e.g. a database's files). They can then request the migration of fileset to
a target provider.

Uppon receiving a request for migration, a provider will recreate the tree of
directories required to receive the files of the fileset, create the files,
mmap them into memory, and issue an RDMA pull operation from the client's files
(themselves mmap-ed into the client's memory).

Following successful migration, the provider will call a user-supplied callback
corresponding to the particular fileset's migration class.

For an example of code, please see the [examples](
folder in the source tree.