Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in
Toggle navigation
Menu
Open sidebar
codes
codes
Commits
f1a8ef05
Commit
f1a8ef05
authored
Nov 03, 2014
by
Jonathan Jenkins
Browse files
Initial IO kernel language documentation
parent
805cdf3c
Changes
1
Hide whitespace changes
Inline
Side-by-side
GETTING_STARTED
View file @
f1a8ef05
...
...
@@ -92,9 +92,54 @@ IO kernel language, and in-development ScalaTrace
(
http
://
moss
.
csc
.
ncsu
.
edu
/~
mueller
/
ScalaTrace
/)
and
IO
Recorder
(
https
://
github
.
com
/
babakbehzad
/
Recorder
)
traces
.
====
Synthetic
IO
language
(
TODO
)
===
Network
(
TODO
)
====
Synthetic
IO
language
The
synthetic
IO
language
is
a
simple
,
interpreted
set
of
IO
and
basic
arithmetic
commands
meant
to
simplify
the
specification
and
running
of
application
workloads
.
In
the
code
it
's currently called the "bgp" workload
generator but that is just a historical artifact - in the future it will be
refactored/renamed.
The input for the workload generator consists of an IO kernel metadata file and
a number of IO kernel files. The former specifies a set of kernel files to run
and logical client IDs to participate in the workload, while the latter
describes the IO to be performed.
The format of the metadata file is a set of lines containing:
<group ID> <start ID> <end ID inclusive> <kernel file>
where:
* <group ID> is the ID of this group (see restrictions)
* <start ID> and <end ID> form the range of logical client IDs that will
perform the given workload. Note that the end ID is inclusive, so a start,
end pair of 0, 3 will include IDs 0, 1, 2, and 3. An <end ID> of -1 indicates
to use the remaining number of clients as specified by the user.
* <kernel file> is the path to the IO kernel workload. It may either be an
absolute or relative path.
The IO kernel file contains a set of commands performed on a per-client
basis. Like the workload generator interface, files are represented by integer
IDs, and the standard set of "POSIX-ish" operations can be applied (e.g., open,
close, sync, write, read) and have a similar argument list (file ID, [length],
[offset] where applicable). pread/pwrite equivalents are given by
readat/writeat.
More detailed documentation on the language is ongoing, but for now a general
example can be seen at doc/workload, which shows a simple out-of-core data
shuffle. Braver souls may wish to visit the implementation at src/iokerellang
and src/workload/codes-bgp-io-wrkld.c.
The following restrictions currently apply to the IO language:
* all user-defined variables must be a single, lower-case letter (the symbol
table from the code we inherited is an array of 26 chars)
* the implementation of "groups" is currently broken. We have gotten around
this by hard-coding in the group size and client ID into the parser when a
kernel file is loaded (parsing currently occurs on a per-client basis).
Hence, getgroupid should be completely ignored and getgrouprank and
getgroupsize ignore the group ID parameter passed in.
=== Network
Documentation will be provided as this feature is further developed.
== LP-IO
...
...
@@ -116,7 +161,7 @@ in the parameter sweep is defined by a python source file with well-defined
field
names
,
to
maximize
flexibility
and
enable
some
essential
features
for
flexible
parameter
sweeps
(
disabling
certain
combinations
of
parameters
,
deriving
parameters
from
other
parameters
in
the
sweep
).
The
actual
replacement
is driven by token replacement defined by the values in the configura
i
ton file.
is
driven
by
token
replacement
defined
by
the
values
in
the
configurat
i
on
file
.
An
exhaustive
example
can
be
found
at
scripts
/
example
.
The
scripts
themselves
are
codes_configurator
.
py
,
codes_filter_configs
.
py
,
and
...
...
@@ -125,6 +170,12 @@ heavily-overlapping functionality, so in the future these may be merged.
==
Miscellaneous
utilities
===
Workload
display
utility
For
debugging
and
experimentation
purposes
,
a
plain
-
text
"dump"
of
an
IO
workload
can
be
seen
using
the
utility
src
/
workload
/
codes
-
workload
-
dump
(
it
gets
installed
into
$
bindir
).
===
LP
template
(
src
/
util
/
templates
)
As
writing
ROSS
/
CODES
models
currently
entail
a
not
-
insignificant
amount
of
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment