Module development
Introduction
System/hardware modules are responsible for analysing a workflow SDFG in the context of a specific component or its part in a system graph. They describe how the component should be modelled/profiled and can provide optional information of interest to a user such as profiling results.
SDFG documentation
SDFGs are explained in this page of the DaCe documentation.
Currently, all outputs produced by modules should be saved to disk using the Adaptyst module API explained in the following section. These outputs can be loaded later in Adaptyst Analyser using its module API described in the Adaptyst Analyser implementation section. The same API allows for developing a custom UI for inspecting the outputs.
In the future Adaptyst versions, as indicated in the roadmap, the modules will also play a significant role in the software-hardware co-design goal of the tool, i.e. determining how well an SDFG or part of it matches a specific system component or its part (e.g. whether a specific code segment is more suitable for a CPU or a GPU). The documentation will be updated accordingly as the R&D work on this progresses.
Compatibility of changes to modules due to R&D work
The changes planned to be made to modules as part of the R&D work explained above will be backward compatible.
Adaptyst implementation

As shown in the figure above, all modules are implemented
in form of a shared library linking against libadaptyst.so containing
the Adaptyst module API. The C interface is used. In turn, libadaptyst.so
loads modules dynamically through dlopen(), using the
lib<MODULE NAME>.so file located inside the <MODULE NAME> folder in
the /opt/adaptyst/modules module path (e.g. in case of linuxperf,
the full shared library path is /opt/adaptyst/modules/linuxperf/liblinuxperf.so
by default).
Defining installation of modules for Adaptyst
There is no pre-defined standard for installing modules for Adaptyst, so you are expected to provide users with installation instructions of your module.
Note that this is not the case for Adaptyst Analyser, see the Adaptyst Analyser implementation section.
Module path changes
The /opt/adaptyst/modules path can be changed by a user, either during Adaptyst compilation
or through the ADAPTYST_MODULE_DIR runtime environment variable.
Therefore, it is crucial to:
- allow the user to change the installation directory of your module or at least not
hardcode that directory (you can e.g. use the
ADAPTYST_MODULE_PATHvariable in CMake when importing the Adaptyst CMake target, see below), - check the value of
ADAPTYST_MODULE_DIRat all points where you refer to the module path or make all variables dependent on the module path changeable by the user through module options.
When Adaptyst is installed, libadaptyst.so and the header files are copied to the usual
library and include directories unless specified otherwise. At the same time,
the adaptyst::adaptyst CMake target for libadaptyst.so and
the header files is created automatically. To use it, put e.g. these two lines in your
CMakeLists.txt file:
find_package(adaptyst REQUIRED)
target_link_libraries(example PUBLIC adaptyst::adaptyst)The CMake target also makes available the ADAPTYST_MODULE_PATH variable storing the
module path (i.e. /opt/adaptyst/modules by default). One example of its usage is
the following:
find_package(adaptyst REQUIRED)
set(INSTALL_DIR "${ADAPTYST_MODULE_PATH}/my_module")The shared library must implement the following C functions:
// These three lines are REQUIRED.
#define ADAPTYST_MODULE_ENTRYPOINT
#include <adaptyst/hw.h>
amod_t module_id = 0;
// Called once when a module instance is initialised by
// Adaptyst (there may be several module instances within
// an entity up to the limit defined in max_count_per_entity).
//
// The return value indicates whether the initialisation
// has succeeded.
bool adaptyst_module_init() {
return true;
}
// Called when a module instance is expected to process
// a workflow SDFG stored in a serialised format in the
// "sdfg" parameter. It is up to the module to decide
// what parts of the SDFG to process. Adaptyst module
// API methods should be used for saving module outputs.
//
// This function is guaranteed to be called after
// adaptyst_module_init().
//
// The return value indicates whether the module
// has successfully processed the SDFG.
bool adaptyst_module_process(const char *sdfg) {
return true;
}
// Called once when a module instance is finalised/closed
// by Adaptyst.
//
// This function is guaranteed to be called last.
void adaptyst_module_close() {
}Serialisation of SDFGs
Serialised SDFGs are in JSON and the serialisation is described in this page of the DaCe documentation. You are welcome to make your own experiments with DaCe to see what the JSON format of SDFGs is like.
Additionally, it must define the following constants/variables in the global scope:
// The human-friendly name of a module.
volatile const char *name = "MyModule";
// The human-friendly version of a module.
volatile const char *version = "1.0.0";
// The non-negative version numbers of a module, with
// the last element being negative (-1 is standard here).
//
// Note that the last element is always ignored, it is used only
// for indicating the end of the array.
//
// At least one non-negative number must be defined.
//
// Version comparisons are done lexicographically,
// e.g. { 3, <anything> } > { 2, <anything> },
// { 3, 2, <anything> } > { 3, 1, <anything> },
// { 11, 2, -1 } > { 11, -1 },
// { 12, 5, 3, -1 } > { 12, 4, 88, -1 } etc.
//
// Please bear in mind that {<xyz>, 0, ..., 0, -1} > {<xyz>, -1},
// where <xyz> is any combination of non-negative numbers and
// the number of zeroes on the left-hand side is anything greater
// than or equal to 1.
volatile const int version_nums[] = { 1, 0, 0, -1 };
// The maximum number of instances of a module that can be
// spawned in an entity. 0 means unlimited.
//
// This constant is optional, the default value is 0.
volatile const unsigned int max_count_per_entity = 0;
volatile const char *options[] = {
// The names of all user-settable options provided by
// a module, with the last element being NULL.
NULL
};
volatile const char *tags[] = {
// The module tags that will be attached
// automatically to a node, with the last element
// being NULL.
//
// Tags can be used for a variety of reasons,
// e.g. to verify whether a GPU module is attached
// to a node that is connected with a CPU node.
//
// All tags used by a module are public-facing
// and thus must be visibly documented to the outside
// world, especially if a module is closed-source.
NULL
};
volatile const char *log_types[] = {
// The names of all types of logs that a module
// can produce, with the last element being NULL.
NULL
};
// Each option XYZ can accept either a single or an array value, NOT both.
// *** For each option XYZ in the "options" array with a single value: ***
// Help message, required.
volatile const char *XYZ_help = "XYZ help message";
// Option value type, required.
volatile const option_type XYZ_type = UNSIGNED_INT;
// Default option value, optional (the option becomes
// required to be set by a user if this is not provided).
//
// The type of this constant is determined by the value
// of XYZ_type. See the Doxygen documentation of
// option_type for matchings between option_type values
// and C types.
volatile const unsigned int XYZ_default = 1;
// *** For each option XYZ in the "options" array with an array value: ***
// Help message, required.
volatile const char *XYZ_help = "XYZ help message";
// Value type of every option array element, required.
volatile const option_type XYZ_array_type = UNSIGNED_INT;
// Default option value, optional (the option becomes
// required to be set by a user if this is not provided).
//
// The type of this array constant is determined by the value
// of XYZ_array_type. See the Doxygen documentation of
// option_type for matchings between option_type values
// and C types.
volatile const unsigned int XYZ_array_default[] = {};
// Number of elements in the default option value, required
// if XYZ_array_default is set.
volatile const unsigned int XYZ_array_default_size = 0;You are very likely to use at least some Adaptyst module API functions in your
module: please consult the Doxygen documentation of the API here.
All essential Adaptyst module API functions compatible with C can be found in the
adaptyst/*.h files. Similarly, extra Adaptyst module API features
compatible with C++20 (not C) can be found in the adaptyst/*.hpp files.
Technically speaking, modules are used in the Adaptyst flow in one of the two ways:
- When workflow execution is Adaptyst-handled and a module indicates that
it will profile a workflow:
- Adaptyst calls
adaptyst_module_init()in the module. Then, in the module, a call to theadaptyst_set_will_profile()API method is made to indicate that the module will profile the workflow. - Adaptyst compiles an SDFG to an executable.
- Adaptyst starts the executable and makes it wait for all profiling modules in all entities to send a “ready” notification.
- Adaptyst calls
adaptyst_module_process()in the module in a separate thread. - The module notifies Adaptyst through the
adaptyst_profile_notify()API call that it is ready for profiling. - The executable resumes its work as soon as it receives a “ready” notification from all profiling modules in all entities.
- The executable finishes its work and the module finishes its processing.
- Adaptyst calls
adaptyst_module_close()in the module.
- Adaptyst calls
- In all other cases:
- Adaptyst calls
adaptyst_module_init()in the module. - Adaptyst calls
adaptyst_module_process()in the module in a separate thread. - The module finishes its processing.
- Adaptyst calls
adaptyst_module_close()in the module.
- Adaptyst calls
adaptyst_module_process() is always called in all modules within an entity in separate
threads, each with its own copy of an SDFG.
Adaptyst Analyser implementation
In contrary to the Adaptyst part of a module, the Adaptyst Analyser part has a pre-defined
directory structure and is installable by adaptyst-analyser:
| <root directory>
| python
| <module name>
| __init__.py
| ...
| (Python files)
| ...
| web
| <module name>
| backend.js
| backend.css
| settings.html
| settings.css
| deps
| ...
| (extra .js/.cjs/.css files to be loaded by Adaptyst Analyser)
| ...
| metadata.ymlBecause Adaptyst Analyser is a Flask app, the Adaptyst
Analyser part of a module consists of two components: a server-side one in Python and a
client-side one in HTML/CSS/JavaScript/jQuery. The server-side files are stored in
python/<module name> and the client-side files are stored in
web/<module name> (the module name must be the same in both cases).
The general information about a module is stored in
metadata.yml.
Adaptyst Analyser JavaScript documentation
The remainder of this section assumes that you have looked at the Adaptyst Analyser JavaScript documentation.
The client-side component must have these files:
backend.js: a JavaScript file implementing at least the following (you can use jQuery, elements fromsettings.htmlare available and can be used):
// Called when a user double-clicks a node in a system graph
// and asks to open a specific module. This function should
// create a new window representing the starting point
// for exploring outputs produced by the module.
//
// Parameters:
// * entity_id (String): the ID of an entity of a node double-clicked
// by a user.
// * node_id (String): the ID of a node double-clicked by a user.
// * session (Object): a Session object corresponding to the
// currently-selected performance analysis session.
function createRootWindow(entity_id, node_id, session) {
}
export { createRootWindow };backend.css: a CSS stylesheet of HTML elements produced bybackend.js. If you want to apply CSS to parts of a window created by your module, you may want to use one of the class names indicated by the diagram below, where<type>is the return value ofgetType()in your JavaScriptWindow-inheriting class.

settings.html: an HTML file implementing a settings window of a module. JavaScript functions frombackend.jscannot be used. Settings widgets to be used bybackend.jsmust have IDs. It is highly recommended that the ID of every element is prefixed by the name of a module, e.g.linuxperf_option123. Otherwise, there may be conflicts between the IDs of elements from two different modules, leading to an unexpected behaviour.settings.css: a CSS stylesheet of the settings window. If you want to apply CSS to the content of the window, you may want to use#<module name>_settings_block.
Other files can be added if used by the above.
Important notes
- When defining HTML in
backend.js, do not use IDs: use classes instead to distinguish elements of your layout. This is because multiple windows with the same layout can be opened at the same time by a user. - When defining CSS in
backend.css, nest your classes in.<type>_contentor.<type>_window, do not put them in the global scope. Otherwise, there may be conflicts between the class names of elements from two different modules or from a module and the Adaptyst Analyser core, leading to an unexpected behaviour.
The client-side component may also have the “deps” folder containing any extra .js, .cjs, and .css files to be loaded by Adaptyst Analyser (this is meant for JavaScript dependencies along with their CSS stylesheets if any). If you use it, please define all filenames expected in the directory in metadata.yml: see the metadata.yml part of the documentation later. Also, if you don’t want to store the files locally in “deps”, you can specify their HTTP(S) URLs in metadata.yml so that Adaptyst Analyser downloads them automatically when installing a module: again, see the metadata.yml documentation for more details.
Dependency conflicts
Both files stored in “deps” and files downloaded from metadata.yml URLs are installed into the same directory in Adaptyst Analyser.
If there are any name conflicts, a user will be given a choice of file version to keep when installing a module. It is not possible to have two different versions of the same file stored at the same time because loading them simultaneously in Adaptyst Analyser may lead to an undefined behaviour.
When writing an HTML code, you may find Adaptyst-Analyser-defined classes
useful for formatting purposes: check the main stylesheet in
the Adaptyst Analyser code. Similarly, you may find the locally-stored
selection of Google Material icons useful:
you can embed them in your HTML in getContentCode() in backend.js
in the following way:
<svg xmlns="http://www.w3.org/2000/svg" data-icon="<icon type>"
other-attributes>
Your extra SVG tags, e.g. <title>
</svg>where <icon type> can be one of:
| Icon type | Icon |
|---|---|
general | |
warning | |
replace | |
download | |
delete | |
copy |
If you want more Google Material icons to be embeddable, don’t hesitate to contact us!
For communication between the client-side and server-side components,
POST requests sent by a client-side part through the sendRequest() JavaScript
function in a Window-inheriting class or Session are used. These are transmitted
to process() in the Python code of your module. Therefore, the server-side
component must export the following Python functions through __init__.py:
def process(storage, identifier, entity, node, data):
"""
Process a request sent by the client-side of a module.
The return value is a (code, d) tuple, where "code" is
an HTTP code to send back to the client and "d" is
data to send back to the client (e.g. a JSON string).
If your HTTP code for the client is 200 (OK),
the return value can also be just data to send back to
the client rather than the tuple above.
:param str storage: The path to the *parent* directory of
results of a performance analysis
session the request corresponds to.
:param str identifier: The ID of a performance analysis
session the request corresponds to.
:param str entity: The name of an entity of a node in a
system graph the request corresponds to.
:param str node: The name of a node in a system graph
the request corresponds to.
:param dict data: JSON dictionary sent by the client.
"""metadata.yml is a YAML file and its structure is as follows:
name: ModuleName
version: 1.0.0
short_desc: Short module description.
# The minimum supported version of the corresponding
# Adaptyst module. If a user tries to inspect data produced
# by an older version of the module, they will get an error.
#
# Use the same versioning scheme as in version_nums in an
# Adaptyst module shared library (negative numbers shouldn't be
# included).
min_module_version: [1, 0, 0]
# Python dependencies. Use the pip requirements.txt format.
python_dependencies:
- package1
- package2
- package3
# JavaScript dependencies expected to be found in "deps" folder
# along with CSS stylesheets if any, in form of full filenames including
# extensions. Only .js, .cjs, and .css files are accepted.
js_dependencies:
- script1.js
- script2.js
- script2.css
# JavaScript dependencies to be downloaded along with CSS
# stylesheets if any, in form of HTTP(S) URLs (e.g. pointing to CDNs).
# Only .js, .cjs, and .css files are accepted.
js_url_dependencies:
- https://example.com/example.js
- https://example.com/example.cssOnce you create the first version of the above files, you can use adaptyst-analyser
to install the Adaptyst Analyser part of your module in development mode, i.e. with all
changes to your code being immediately propagated to the Adaptyst Analyser installation
directory so that they can be tested directly in a real environment (similarly to how
pip install -e works in Python):
adaptyst-analyser -d <path to the directory with your module>