Skip to content

Mat-O-Lab DataStack

DataStack turns your research data files into FAIR-compliant, publishable datasets — without writing any code.

Upload a CSV, a microscopy export, or a structured data file. The pipeline enriches your data with structured metadata, links it to shared scientific vocabularies, and publishes it to a data portal where it is searchable, downloadable, and citable.

Create complete, consistent metadata for a non-semantic resource — then use semantic technologies to transform that further.

The result: data that meets FAIR principles (Findable, Accessible, Interoperable, Reusable) by design, not as an afterthought.

Where do you want to start?

→ What the pipeline does

Learn what happens to your data, what outputs you receive, and how to interpret the results. No technical background needed.

For a deeper look at how semantic enrichment works, see Semantic Foundation.

→ Quickstart

Get the stack running with Docker Compose. Configure data sources and connect your storage backend.

→ Author a Mapping

Learn how to describe your data structure so the pipeline can enrich it automatically.

→ Standalone APIs

Run individual pipeline components without deploying the full stack.

Three domains — one pipeline

The same pipeline handles different scientific and industrial data formats:

Domain Source What you get Live example
Lab / materials science CSV measurement file (tensile test, spectroscopy) A structured, standards-aligned metadata record — searchable and citable IOFMaterialsTutorial
Microscopy imaging OMERO image archive Image metadata linked to instrument, acquisition parameters, and sample context BAMresearch DF-TEM-PAW
Automotive supply chain SAMM / Catena-X product data Machine-readable records linked to shared automotive industry vocabularies futurecarproduction.materialsdata.space

The pipeline does not need to be told which row applies to a given upload. When data arrives, CKAN tests it against its library of mapping rule files. The file that matches identifies what the data is about — and simultaneously provides the enrichment rules. No manual classification step, no format-specific pipeline branch.

For example: upload image acquisition metadata and the pipeline selects a microscopy mapping that knows about instrument parameters and acquisition context. Upload a CSV whose column names and unit annotations match a tensile test rule file exactly, and that mapping is selected instead.

Matching is based on the names and annotations in the metadata — not on column order or row order in the raw data. A rule applies if the named items it references are present; column sequence does not matter. But if the names themselves differ (different machine output, different export format), the rules do not match and the file is left unenriched rather than partially processed. This means one machine model may need its own mapping file. The key insight is that multiple mapping files can all point to the same prototype graph: an INSTRON tensile tester and a Zwick tensile tester produce differently named columns, but both get mapped to the same tensile test semantic structure. The mapping handles the format difference; the prototype graph defines what the data means.

In production today

DataStack runs at public data portals:

The pipeline approach is documented in peer-reviewed publications:

Nasrabadi, Hanke et al. (2023). Toward a digital materials mechanical testing lab. Computers in Industry, 153, 104016. doi:10.1016/j.compind.2023.104016

Hanke et al. (2023). FAIR microscopy data via the Mat-O-Lab pipeline. Scientific Data (Nature). doi:10.1038/s41597-023-02244-6

Ready to explore?