Structures Data

Research organizations β€” institutions, laboratories, teams β€” are a key part of the research information managed by CRISalid. Getting them into the graph is often the first step, and it can involve some work: mapping your local identifiers, encoding the hierarchy, handling supervision relationships between units and institutions. This page is here to help you through it.

Two options to connect to institutional directories

CRISalid directory bridge offers two ways to feed structure data into the knowledge graph. Both produce the same AMQP message format consumed by the IKG middleware.

Option Source DAG Best suited for
1 β€” LDAP + spreadsheet LDAP directory (SUPANN 2021) + CSV complement load_ldap_structures Institutions with a SUPANN-compliant directory
2 β€” Spreadsheet CSV spreadsheet load_spreadsheet_structure Manual curation, institutions without LDAP, graduate schools, cross-institutional units

Option 1 β€” LDAP + spreadsheet

The load_ldap_structures DAG queries an LDAP directory filtered on SUPANN object classes and converts entries into IKG-compliant structure records.

Key LDAP attributes used: supannCodeEntite (identifier), supannCodeEntiteParent (hierarchy), supannTypeEntite, supannRefId (external identifiers such as ROR, NNS), eduOrgLegalName, ou, description, businessCategory, labeledURI.

CSV complement

When OVERRIDE_LDAP_STRUCTURE_DATA_FROM_SPREADSHEET=True, the DAG also reads a CSV file and merges it into the LDAP results: any non-empty CSV value overwrites the corresponding LDAP field, while LDAP values are kept where the CSV is blank. Use it to supply fields that LDAP does not carry (national identifiers, tutelle relationships, HAL collections, research classifications) or to add structures not listed in LDAP. A CSV-only structure (absent from LDAP) is added to the result set; a structure with generic_type=ignore is dropped.

Generating a CSV template from live LDAP

Rather than building the CSV from scratch, use the provided script to pre-fill it from your live directory:

APP_ENV=DEV python scripts/generate_structure_template.py --output data/structures.csv

The script connects to LDAP using the credentials defined in the environment file matching APP_ENV (.env.dev, .env.staging, .env.prod, …), fetches all structures, and writes a CSV with one row per entry. It maps the fields it can derive from LDAP (local_id, short_labels, long_labels, descriptions, web, main_mission, inclusions) and leaves the rest blank for you to fill in. Redirect to stdout by omitting --output.

Once you have the template you can open it in a spreadsheet editor or edit it directly, then use the tools below to validate and sort it before ingestion.

Option 2 β€” Spreadsheet

The load_spreadsheet_structure DAG (in crisalid-directory-bridge) reads a UTF-8 CSV file whose columns follow the CRISalid structures format. This is the recommended path for structures that are not in the institutional LDAP, or for institutions that prefer to maintain a fully curated file.

File format reference

The full column specification and annotation syntax (labels with language tags, relationship entries with supervision codes and date ranges) is documented on the wiki:

πŸ‘‰ Import des donnΓ©es structures on the CRISalid Esup Portail wiki

Tooling for CSV files

Three standalone Python scripts are available in the repository to prepare and inspect a structures CSV β€” whether you use the LDAP+CSV combined path or the pure spreadsheet path. They require only the standard library (no Airflow, no virtualenv).

Fixing row ordering

The CSV is processed row by row, so any structure referenced in inclusions or participations must appear in an earlier row than the structure that references it. If you have ordering violations β€” for example after inserting rows manually β€” the sort script rewrites the file in the correct topological order:

python scripts/sort_structures.py path/to/structures.csv

The script uses Kahn’s topological sort and preserves the original relative order of rows as much as possible. If a circular dependency is detected it prints a warning and leaves the file unchanged.

Validating the CSV

python .claude/commands/check_structures.py path/to/structures.csv

Prints all errors and warnings with row numbers, then exits with a non-zero status if errors are found.

What it checks:

  • Required columns and fields
  • local_id format (max 10 chars, alphanumeric / - / _ / :)
  • Duplicate local_id values; generic_type=ignore rows are only checked for duplicates
  • National type compatibility with generic_type (fetched live from the IKG model)
  • Mission fields (main_mission required and valid for units; warns if set on other types)
  • Conditional identifier requirements (UAI for institutions; NNS or ROR for research units)
  • Identifier format (bare ROR/ISNI IDs, RNSR pattern for NNS)
  • Reference integrity β€” every local-* target must exist in the file
  • Ordering β€” a referenced structure must appear in an earlier row
  • Isolation β€” warns on teams and unit subdivisions with no parent
  • Date annotation format (YYYYMMDD)
  • Label language tags

Visualising the hierarchy

python .claude/commands/visualize_structures.py path/to/structures.csv [output.html] \
  [--root-institution LOCAL_ID]

Generates a self-contained HTML file (no server required, opens in any browser). --root-institution sets the root node used for the initial viewport centering and the default 1-level expanded view; omit it to start fully expanded. If output.html is not specified, the file is written next to the CSV with the same basename.

As Claude Code slash commands

If you use Claude Code inside the crisalid-directory-bridge repository, the same tools are available as slash commands:

/check-structures path/to/structures.csv
/visualize-structures path/to/structures.csv [output.html]