[doc] Add doxygen documentation to the project

.gitignore

@@ -3,6 +3,7 @@
 
 # unignore files with extensions
 !*.*
+
 # unignore special files
 !AUTHORS
 !LICENSE
@@ -50,7 +51,7 @@ stamp-h1
 /install-sh
 /missing
 /stamp-h1
-version.h
+/version.h
 /m4/libtool.m4
 /m4/ltmain.sh
 /m4/ltoptions.m4

.gitlab-ci.yml

@@ -4,6 +4,7 @@ variables:
 stages:
   - style
   - build
+  - release
 
 make:generic:
   stage: build
@@ -53,3 +54,12 @@ checkpatch:
   stage: style
   script:
     - nix run -f "$ARGOPKGS" checkpatch --command checkpatch.pl
+
+readthedocs:
+  stage: release
+  when: on_success
+  only:
+    - master
+    - /v\.[0-9]+\.[0-9]+\.[0-9]+/
+  script:
+    - nix run nixpkgs.curl -c curl -X POST -d "branch=$CI_COMMIT_REF_NAME" -d "token=$READTHEDOCS_TOKEN" https://readthedocs.org/api/v2/webhook/argo-aml/83161/

CONTRIBUTING.markdown

-Contributing to The Project
-===========================
+AML Contribution Guide {#contributing_page}
+============
 
-The following is a set of guidelines for contributing to this project. These
-are guidelines, not rules, so use your best judgment and feel free to propose
-changes. For information about the overall design of the library and general
-coding guidelines, see the `DESIGN.markdown` file.
+Welcome to AML contribution guide.
+This page walk through the main library expectations.
+When contributing to AML, contributors should write
+features aligned with AML goal and AML spirit.
+Additionnally, several good practices are requested, some beeing inforced
+through the continuous integration (CI) pipeline.
+For readability and maintainability, coding conventions need to be respected,
+code needs to be tested, documented and integrated with the main AML branch
+and its build tool-chain.
 
-## Contribution Workflow
+## General Guidelines
+
+At its core, AML is able to provide access to established, well known
+abstractions inside a common toolbox, with the more flexibility possible.
+Abstractions are made available through __building blocks__, each representing
+a single abstraction. Each building block is separated into a __generic__ API
+for operations that the abstraction provides, and __implementations__ of those
+operations for a specific implementation of the abstraction.
+__generic__ API, i.e the first layer exposed to users
+through a generic interface declares a table of operations common to the
+abstraction, the main abstraction structure containing a such a table
+and an opaque handle to store implementation specific data.
+
+```
+struct aml_building_block_ops{
+       aml_building_block_op0();
+       ...
+};
+
+struct aml_building_block_data;
+
+struct aml_building_block{
+       struct aml_building_block_ops *ops;
+       struct aml_building_block_data *data;
+};
+```
+
+A set of generic functions taking the abstraction as first argument
+is supposed to be exposed while it will de-reference underlying
+implementation function table to implement expected behavior.
+Such generic functions are supposed to return an AML error code as defined
+in [error header](@ref error.h).
+
+```
+int aml_building_block_feature(struct aml_building_block *, ...);
+```
+
+Building blocks should interact with each other through the generic API, but
+specific implementations are allowed to break this rule for
+performance-oriented reasons. A generic version should still be available.
+It is expected that these will be transparent and composable,
+because low-level optimizations require control.
+Therefore, most implementation details (structures, functions) will be exposed
+to the user in a dedicated header, and use of opaque structures, static functions
+and variables there should be avoided.
+Moreover, customization of a building block should happen at creation time: the goal
+is to make it easy for users to evaluate different configurations options with
+minimal impact on the inner code of the application.
+Therefore, building blocks specialized implementation, must declare a dynamic allocator, free,
+initialization and finish functions, and stack declaration.
+
+```
+int  aml_building_block_create (struct aml_building_block **, ...);
+void aml_building_block_destroy(struct aml_building_block *);
+int  aml_building_block_init   (struct aml_building_block *, ...);
+void aml_building_block_fini   (struct aml_building_block *);
+
+#define AML_BUILDING_BLOCK_DECL(name) \
+	struct aml_building_block_data __ ##name## _inner_data; \
+	struct aml_building_block name = { \
+		&aml_building_block_ops, \
+		(struct aml_building_block_data *)&__ ## name ## _inner_data, \
+	}
+```
+
+Usually a building blocks implementation would declare an exportable (`extern`)
+`struct aml_building_block_ops`, and `struct aml_building_block`, to provide
+users with a default static implementation of the block.
+
+Finally, building blocks headers, sources and tests go to a dedicated folder
+located respectively in `include`, `src` and `tests`.
+
+## Coding Convention
+
+AML code respects several coding conventions. Some of them are enforced through
+the CI pipeline.
+Prior to start coding AML, you might want to set your editor to minimize
+the process of code formatting.
+Most of the formatting can also be scripted with `indent` program.
+Code formatting compliance with AML requirements can be checked by running
+[linux kernel linter](https://github.com/torvalds/linux/blob/master/scripts/checkpatch.pl)
+at the root of AML project.
+
+The list of coding convention is the following:
+* use tabulation of 8 characters,
+* remove trailing white spaces,
+* lines may not exceed 80 characters,
+* pointers star have to stick to the variable name: `void *var;`,
+* function return type is on the same line as function name,
+* curly braces are on the same line as the statement or function,
+* A new line must appear after curly braces.
+* macros are UPPERCASE_WITH_UNDERSCORES, function lowercase_with_underscores
+* All functions/macros should start with aml/AML followed by the name of the building block involved.
+
+## Versioning and Submission Workflow
+
+AML code and CI runners is hosted on Argonne National Laboratory
+[gitlab](https://xgitlab.cels.anl.gov/argo/aml) infrastructure under git version control.
+New user may create new branches upon request to host and test their code.
+New branched must be created from the master branch and merging is done
+via git or gitlab pull request interface.
 
 We basically follow the [Gitlab Flow](https://docs.gitlab.com/ee/workflow/gitlab_flow.html)
 model:
  - `master` is the primary branch for up-to-date development
- - any new feature/development should go into a `feature-branch` based of
+ - any new feature/development should go into a `feature-branch` based on
    `master` and merge into `master` after.
  - we follow a `semi-linear history with merge commits` strategy: each merge
    request should include clean commits, but only the HEAD and merge commit
@@ -21,7 +126,8 @@ model:
    change should appear in its own commit.
  - to help with in-development work, CI will not activate on branches with
    names started with wip/
-## Commit Messages Styleguide
+
+### Commit Messages Styleguide
 
 - use present tense, imperative mood
 - reference issues and merge requests
@@ -59,7 +165,7 @@ commit.template my-config-template.txt` to use it automatically.
 # --------------------
 ```
 
-## Signoff on Contributions:
+### Signoff on Contributions:
 
 The project uses the [Developer Certificate of
 Origin](https://developercertificate.org/) for copyright and license
@@ -69,3 +175,98 @@ the `LICENSE` file) and that you agree to the DCO.
 
 To signoff commits, use: `git commit --signoff`.
 To signoff a branch after the fact: `git rebase --signoff`
+
+## Testing
+
+AML project includes a set of unit and integration tests.
+Each AML building block has its own set of tests in a separate directory.
+Tests are launched together with automake test suite with the `make check` command.
+Building blocks should be tested as thoroughly as possible and whenever
+relevant, integration tests with other components should be available.
+Tests compilation and launch are implemented together in `tests/Makefile.am`.
+Whenever a branch is pushed to AML repository, it automatically triggers
+the CI pipeline, unless branch name starts with `wip/`.
+The CI pipeline must be successful for a merge request to be accepted.
+
+## Documentation
+
+For obvious reasons, the code and building block must be documented.
+The whole documentatation is based doxygen tool.
+All the public code (data structures, functions, macro, variables) must be
+documented as precisely as possible. Additionnaly, new buiding blocks implementation
+must create a group (with `\@defgroup`) and provide a detailed description of
+the features it provides.  Syntax for code documentation can be found
+[here](http://www.doxygen.nl/manual/docblocks.html),
+and most useful elements can be found in the example below.
+
+```
+/**
+ * \@defgroup aml_buidling_block_implementation "AML buidling block implemation"
+ *
+ * \@brief This is a short description of this group.
+ *
+ * The documentation encapsulated below
+ * including this long description of the building_block
+ * implementation will be create a section dedicated to this buidling block
+ * implementation. It will be possible to link to it in the main header
+ * documentation with "@ref" or "@see".
+ * \@{
+ **/
+
+/**
+ * This is a default operation table implementing
+ * table defined in main header.
+ **/
+extern struct aml_building_block_ops aml_building_block_ops;
+
+/**
+ * This is the bb inner data declaration, to
+ * stuff with needed attributes.
+ **/
+struct aml_building_block_data {};
+
+/**
+ * This is the bb declaration
+ **/
+struct aml_building_block{
+       struct aml_building_block_ops *ops;
+       struct aml_building_block_data *data;
+};
+
+/** This is the static buidling block declaration macro **/
+#define AML_BUILDING_BLOCK_DECL(name) \
+	struct aml_building_block_data __ ##name## _inner_data; \
+	struct aml_building_block name = { \
+		&aml_building_block_ops, \
+		(struct aml_building_block_data *)&__ ## name ## _inner_data, \
+	}
+
+/** Static declaration of the building block size **/
+#define AML_BUILDING_BLOCK_ALLOCSIZE (sizeof(struct aml_building_block_data) + \
+				 sizeof(struct aml_building_block))
+
+
+/**
+ * Allocates and initializes a new building_block.
+ *
+ * \@param bb[out]: A pointer to store the new building block.
+ * \@param other[in]: Other parameters used as input. 
+ * \@return AML_SUCCESS if successful; an error code otherwise.
+ **/
+int aml_building_block_create(struct aml_building_block **bb, ...);
+
+...
+
+/**
+ * This is the end of the inclusion into building_block documentation group.
+ * \@}
+ **/
+```
+
+If the contribution is an implementation of a building block and header has
+been placed in `include/aml/building_block` then its documentation will automatically added to
+the project documentation. However, it is necessary to point to this particular group
+with `@ref` or `@see` command in the [main aml header](\ref aml.h).
+If the contribution creates new include directories, the latter have to be included
+in `doc/aml.doxy` after the field `INPUT`.
+

Makefile.am

@@ -5,7 +5,12 @@ if ADD_BENCHMARKS
 SUBDIRS += benchmarks
 endif
 
+if AML_DOXYGEN
+SUBDIRS += doc
+endif
+
 pkgconfigdir = $(libdir)/pkgconfig
 pkgconfig_DATA = aml.pc
 
 EXTRA_DIST = autogen.sh aml.pc README.markdown
+

README.markdown

@@ -7,6 +7,30 @@ memory management policies for allocation and placement of data across devices.
 
 This library is still in the prototyping phase. APIs might break often.
 
+## Documentation
+
+The latest documentation can be found
+[online](https://argo-aml.readthedocs.io/en/latest/),
+can be installed when installing a release or generated
+from git clone before installing with
+```
+make -C doc doxygen
+```
+
+## Requirements:
+
+* autoconf
+* automake
+* libnuma
+
+## Installation
+
+```
+sh autogen.sh
+./configure
+make -j install
+```
+
 # General Architecture
 
 The architecture of the library relies on two principles: type-specific

configure.ac

@@ -69,6 +69,12 @@ AC_CHECK_HEADERS([numa.h],,[AC_MSG_ERROR([AML requires libnuma headers.])])
 AC_CHECK_HEADERS([numaif.h],,[AC_MSG_ERROR([AML requires libnuma headers.])])
 AC_CHECK_LIB(numa, mbind,,[AC_MSG_ERROR([AML requires libnuma.])])
 
+# Doxygen
+# See m4/ax_doxygen.m4
+AC_CHECK_PROG(doxygen_happy, [doxygen], [yes], [no])
+
+AM_CONDITIONAL([AML_DOXYGEN],[ test "x$doxygen_happy" = "xyes" ])
+
 AC_CONFIG_HEADERS([include/config.h])
 
 AC_CONFIG_FILES([Makefile
@@ -80,6 +86,7 @@ AC_CONFIG_FILES([Makefile
 		 src/utils/Makefile
 		 include/Makefile
 		 tests/Makefile
+		 doc/Makefile
 		 benchmarks/Makefile
 		 aml.pc
 		 include/aml/utils/version.h])

doc/Makefile.am

+DATA_INSTALL_DIR=$(datadir)/aml
+DOXYGEN_BUILD_DIR=./build-doxygen
+SPHINX_BUILD_DIR=./build-sphinx
+
+doxygen:
+	doxygen aml.doxy
+
+create-doc-dir:
+	mkdir -p $(DATA_INSTALL_DIR)
+
+install-doc-html:
+	if [ -d $(DOXYGEN_BUILD_DIR)/html ]; then \
+	cp -r $(DOXYGEN_BUILD_DIR)/html $(DATA_INSTALL_DIR); \
+	fi
+
+install-data-local: create-doc-dir install-doc-html
+
+uninstall-local:
+	rm -rf $(DATA_INSTALL_DIR)
+
+clean-local:
+	rm -rf $(DOXYGEN_BUILD_DIR) $(SPHINX_BUILD_DIR)
+
+dist-hook: doxygen
+	cp -r $(DOXYGEN_BUILD_DIR) $(distdir)
+
+

doc/conf.py

+################################################################################
+# Copyright 2019 UChicago Argonne, LLC.
+# (c.f. AUTHORS, LICENSE)
+# 
+# This file is part of the AML project.
+# For more info, see https://xgitlab.cels.anl.gov/argo/aml
+# 
+# SPDX-License-Identifier: BSD-3-Clause
+################################################################################
+
+# Configuration file for the Sphinx documentation builder.
+#
+# This file only contains a selection of the most common options. For a full
+# list see the documentation:
+# http://www.sphinx-doc.org/en/master/config
+
+# -- Path setup --------------------------------------------------------------
+
+# If extensions (or modules to document with autodoc) are in another directory,
+# add these directories to sys.path here. If the directory is relative to the
+# documentation root, use os.path.abspath to make it absolute, like shown here.
+#
+import os
+import sys
+import subprocess
+
+# Generate documentation
+subprocess.call('doxygen aml.doxy', shell=True)
+ 
+# -- General configuration ---------------------------------------------------
+
+# The language for content autogenerated by Sphinx.
+language = 'C'
+
+# Add any Sphinx extension module names here, as strings. They can be
+# extensions coming with Sphinx (named 'sphinx.ext.*') or your custom
+# ones.
+extensions = []
+
+# Add any paths that contain templates here, relative to this directory.
+templates_path = []
+
+# List of patterns, relative to source directory, that match files and
+# directories to ignore when looking for source files.
+# This pattern also affects html_static_path and html_extra_path.
+exclude_patterns = []
+
+# -- Project information -----------------------------------------------------
+
+project   = 'AML'
+copyright = 'Copyright 2019 UChicago Argonne, LLC'
+author    = 'Swann Perarnau, Kamil Iskra, Brian Suchy, Valentin Reis, Nicolas Denoyelle'
+release   = 'X.X.X'
+
+# -- Options for HTML output -------------------------------------------------
+
+# The theme to use for HTML and HTML Help pages.  See the documentation for
+# a list of builtin themes.
+#
+html_theme = 'alabaster'
+
+# Add any paths that contain custom static files (such as style sheets) here,
+# relative to this directory. They are copied after the builtin static files,
+# so a file named "default.css" will overwrite the builtin "default.css".
+html_static_path = []
+
+#Overwrite sphinx documentation with doxygen documentation.
+html_extra_path = ['build-doxygen/html']
+

doc/contents.rst

+This is an empty file required by shpinx when
+compiling documentation on readthedocs.org.

doc/index.rst

+This is an empty landing page overwritten with doxygen documentation.
+

doc/md/main.md

+Welcome to AML Documentation Page {#index}
+============
+
+AML is a memory management library designed for highly optimized
+codes to delegate complex memory operations.
+
+## Overview
+
+AML goals are the following:
+1. to provide a convenient interface for addressing all heterogeneous
+addressable memories and moving data around.
+2. to provide an interface for expressing and translating data
+structures between several representations and physical memories.
+
+Toward achieving these goals, AML implements the following abstractions:
+
+\image html building-blocks-diagram.png width=400cm
+
+* [Areas](@ref aml_area), a set of addressable physical memories,
+* [Tilings](@ref aml_tiling), a description of tiling data structures,
+* [DMA](@ref aml_dma), an engine to asynchronously move data structures between areas,
+* [Scratchpad](@ref aml_scratch), a stage-in, stage-out abstraction for prefetching.
+
+Each of these abstractions have several implementations. For instance,
+areas may refer to usual DDR of a subset of them, GPU memory or non-volatile memory.
+Tilings are implemented to reflect either 1D or 2D structures etc.
+
+## Problems Solved with AML
+
+AML library is designed for highly optimized codes to delegate complex memory
+operations. AML is currently used in a
+[proof of concept](https://xgitlab.cels.anl.gov/argo/nauts/tree/master/papers/mchpc18)
+of efficient matrix multiplication on architectures with software managed memory side
+cache. It is also currently being extended toward specialized memory allocations
+for latency sensitive applications.
+
+## Quick Start Guide
+
+### Download
+
+* [Github version](https://xgitlab.cels.anl.gov/argo/aml):
+```
+git clone git@xgitlab.cels.anl.gov:argo/aml.git
+```
+* Release versions: 
+TODO after release.
+
+### Requirements:
+
+* autoconf
+* automake
+* libnuma
+
+### Installation
+
+```
+sh autogen.sh
+./configure
+make -j install
+```
+
+### Workflow
+
+* Include aml header:
+```
+#include <aml.h>
+...
+int main(int argc, char **argv){
+```
+
+* Check AML version:
+```
+if(aml_version_major != AML_VERSION_MAJOR){
+    printf("AML ABI mismatch!");
+    return 1;
+}
+```
+
+* Initialize and Cleanup AML:
+```
+if(aml_init(&argc, &argv) != 0){
+    printf("AML library init failure!");
+    return 1;
+}
+...
+aml_finalize();
+```
+
+* Link your program with `-laml`
+
+See above building blocks specific pages for further examples and information on
+library features.
+
+## Support
+
+AML interface with users is mainly done through
+[gitlab issue interface](https://xgitlab.cels.anl.gov/argo/aml/issues) and
+direct contact with developers/maintainers:
+* Swann Perarnau (swann@anl.gov)
+* Nicolas Denoyelle (ndenoyelle@anl.gov)
+
+AML is looking forward to new users. If you think your problem can be solved with AML
+eventually with some additional non-existing features which are relevant to the library,
+please let us know. Tell us about your topic of interests and your scientific/technical
+problems. We are also pleased to improve the software, thus any suggestion, bug
+declaration, contribution is welcomed.
+
+## Contributing
+
+AML is looking forward to new collaborations and contributions.
+If you wish to contribute to AML project rendez-vous [here](@ref contributing_page)
+

include/aml/area/linux.h

@@ -15,81 +15,86 @@
 #include <numa.h>
 #include <numaif.h>
 
-/* allowed binding flags */
+/**
+ * @defgroup aml_area_linux "AML Linux Areas"
+ * @brief Linux Implementation of Areas.
+ *
+ * Linux implementation of AML areas.
+ * This building block relies on libnuma implementation and
+ * linux mmap/munmap to provide mmap/munmap on NUMA host
+ * host processor memory. New areas may be created
+ * to allocate a specific subset of memories.
+ * This building block also include a static declaration of
+ * a default initialized area that can be used out of the box with
+ * abstract area API.
+ *
+ * #include <aml/area/linux.h>
+ * @{
+ **/
+
+/**
+ * Allowed binding flag for area creation.
+ * This flag will apply strict binding to the selected bitmask.
+ * If subsequent allocation will failt if they cannot enforce binding
+ * on bitmask.
+ **/
 #define AML_AREA_LINUX_BINDING_FLAG_BIND       (MPOL_BIND)
+
+/**
+ * Allowed binding flag for area creation.
+ * This flag will make subsequent allocations to interleave
+ * pages on memories of the bitmask.
+ **/
 #define AML_AREA_LINUX_BINDING_FLAG_INTERLEAVE (MPOL_INTERLEAVE)
+
+/**
+ * Allowed binding flag for area creation.
+ * This flag will make subsequent allocations to bound to the
+ * nodes of bitmask if possible, else to some other node.
+ **/
 #define AML_AREA_LINUX_BINDING_FLAG_PREFERRED  (MPOL_PREFERRED)
 
-/* allowed mmap flags to pass */
+/**
+ * Allowed mapping flag for area creation.
+ * This flag will make subsequent allocations to be private
+ * to the process making them.
+ **/
 #define AML_AREA_LINUX_MMAP_FLAG_PRIVATE (MAP_PRIVATE | MAP_ANONYMOUS)
-#define AML_AREA_LINUX_MMAP_FLAG_SHARED  (MAP_SHARED | MAP_ANONYMOUS)
-
-extern struct aml_area_ops aml_area_linux_ops;
-
-/* User data stored inside area */
-struct aml_area_linux_data {
-	/** numanodes to use **/
-	struct bitmask *nodeset;
-	/** numaif.h mbind policy or AML_AREA_LINUX_FLAG_* **/
-	int             binding_flags;
-	/** mmap flags **/
-	int             mmap_flags;
-};
-
-/* Default linux area with private mapping and no binding. */