Docker Compose deployment

⚠️ Warning: Do not deploy this setup to production. This configuration is meant solely for local development and testing.

🛡️ Ensure your firewall is enabled on the machine running this setup (e.g., ufw on Linux). Running this deployment without a firewall can lead to data leaks or insecure access to internal institutional directories (especially if LDAP is enabled).

🔍 Choosing the Components

Before you start, review the components available in map/components.qmd and decide which ones you need.

The main Compose file (docker/docker-compose.yaml) is modular. It uses the include directive and profiles to enable only selected components.

Main docker-compose.yaml

name: crisalid

include:
  - path: ./neo4j/neo4j.yaml
    env_file: ./neo4j/.env
    project_directory: ./neo4j
  - path: ./apollo/apollo.yaml
    env_file: ./apollo/.env
    project_directory: ./apollo
  - path: ./crisalid-bus/crisalid-bus.yaml
    env_file: ./crisalid-bus/.env
    project_directory: ./crisalid-bus
  - path: ./harvester/harvester.yaml
    env_file: ./harvester/.env
    project_directory: ./harvester
  - path: ./ikg/ikg.yaml
    env_file: ./ikg/.env
    project_directory: ./ikg
  - path: ./cdb/cdb.yaml
    env_file: ./cdb/.env
    project_directory: ./cdb
  - path: ./sovisuplus/sovisuplus.yaml
    env_file: ./sovisuplus/.env
    project_directory: ./sovisuplus
  - path: ./keycloak/keycloak.yaml
    env_file: ./keycloak/.env
    project_directory: ./keycloak
  - path: ./ofelia/ofelia.yaml
    env_file: ./ofelia/.env
    project_directory: ./ofelia

For example:

docker compose \
  --profile neo4j \
  --profile apollo \
  --profile crisalid-bus \
  --profile harvester \
  --profile ikg \
  --profile cdb \
  --profile keycloak \
  --profile sovisuplus \
  --profile ofelia \
  up

🧰 Preparation Steps

1. 🧾 `.env` Files

Each directory under docker/ (e.g. apollo, crisalid-bus, ikg, neo4j, cdb, harvester, …) has its own .env.sample file.

Copy each .env.sample to .env
Fill in appropriate values (hostnames, ports, secrets, etc.)
The main docker/.env.sample includes values used by multiple components (like RabbitMQ or Neo4j credentials)

If you plan to connect the CRISalid Directory Bridge (cdb) to your institutional LDAP, make sure to set:
LDAP_HOST=
LDAP_BIND_DN=
LDAP_BIND_PASSWORD=

2. 🔧 Configure CRISalid Bus

This script reads the .env values and generates the RabbitMQ definitions.json file (exchanges, queues, admin user, etc.).

./docker/configure_crisalid_bus.sh

3. 🔧 Configure CRISalid Directory Bridge (CDB)

This script clones the DAGs and runs the Airflow initialization:

./docker/configure_cdb.sh

ℹ️ In dev, Airflow GUI admin credentials are set to admin:admin.

After running the script, if you intend to use the CSV mode for structures and people (instead of LDAP), place your data files in:

docker/cdb/data/
├── structure.csv
└── people.csv

Sample CSVs:

docker/cdb/dags/data

Full documentation (in French):

4. 🔑 Configure Keycloak

Keycloak is handling authentication within the system. Multiple client applications (such as Sovisu+) can share the same authentication realm. To set up Keycloak in this environment, follow these steps:

Global .env Configuration

In the global .env file, you will find the shared Keycloak configuration variables, such as the realm name ( KEYCLOAK_REALM) and the client secrets (SOVISUPLUS_KEYCLOAK_CLIENT_SECRET). The KEYCLOAK_REALM can be customized ( e.g., crisalid-my-university) for readability.

Example:

KEYCLOAK_REALM=crisalid-inst
SOVISUPLUS_KEYCLOAK_CLIENT_SECRET=MY-SECRET-VALUE

Keycloak Configuration Script

Run the ./configure_keycloak.sh script. This will create the required configuration file from the template ( docker/keycloak/config/crisalid-inst.json.template).

Customizing Keycloak .env Settings

The docker/keycloak/.env.sample file provides the environment settings for Keycloak, such as the admin credentials and database configurations. Copy the sample file to .env and modify the settings as needed.

KEYCLOAK_ADMIN=admin
KEYCLOAK_ADMIN_PASSWORD=admin
KEYCLOAK_DB_VENDOR=postgres
KEYCLOAK_DB_HOST=keycloak-db
KEYCLOAK_DB_PORT=5432
KEYCLOAK_DB_NAME=keycloak
KEYCLOAK_DB_USER=keycloak
KEYCLOAK_DB_PASSWORD=keycloak

Define specific URIs in /etc/hosts

To ensure that Sovisu+ and Keycloak can be accessed correctly, you need to define specific URIs in your /etc/hosts file. This is necessary because Sovisu+ uses OAuth2 with ORCID, which requires a specific hostname even to deliver “sandbox” keys.

# Add these lines to your /etc/hosts file
127.0.0.1 sovisuplus.local
127.0.0.1 keycloak.local

5. SoVisu+ custom themes

SoVisu+ allows you to customize its appearance using themes. You can create your own theme by following these steps: 1. Copy the sample theme directory:

cp -r sovisuplus/theme-sample sovisuplus/theme

Edit the theme files in sovisuplus/theme to customize text and images according to your institution’s branding.

6. SoVisu+ RBAC Roles File

SoVisu+ comes with a sample RBAC configuration you can customize. Start by copying the sample file and editing it:

cp sovisuplus/config/rbac.roles.sample.yaml sovisuplus/config/rbac.roles.yaml

Edit sovisuplus/config/rbac.roles.yaml to define your roles by grouping permissions (but dont create new permissions, i.e new actions/subjects/fields as they are used in the code).
After any change, the docker startup script will copy the file to the container and re-seed the roles and permissions in database.

7. ⏰ Configure Ofelia (system-wide scheduler)

Ofelia is the scheduler of the whole CRISalid stack. It runs as its own container and periodically triggers tasks for other containers (by running a CLI command or calling an API endpoint).

Ofelia reads a config file and uses the Docker API to:

job-exec: run a command inside an existing container (via docker exec)
job-local: run a command directly inside the Ofelia container itself (typically, a curl command to call an API)
(and job-run: run a one-shot container if needed)

You can find full documentation here:

Docker Hub: https://hub.docker.com/r/mcuadros/ofelia
GitHub repository: https://github.com/mcuadros/ofelia/tree/master

In this deployment, Ofelia is included as its own Docker Compose profile (ofelia) with:

a compose file: docker/ofelia/ofelia.yaml
a scheduler config file template: docker/ofelia/config.ini
a configuration template: docker/ofelia/.env.sample

7.1. Create Ofelia `.env`

In docker/ofelia/, copy the sample file:

cp docker/ofelia/.env.sample docker/ofelia/.env

The sample content is:

CONFIG_FILE_NAME=config.ini

This variable is used only by Docker Compose to choose which config file to mount inside Ofelia. You can later duplicate config.ini under another name (for example config.dev.ini, config.prod.ini, etc.) and switch by changing CONFIG_FILE_NAME in .env without editing the Compose file:

CONFIG_FILE_NAME=config.dev.ini
# or:
# CONFIG_FILE_NAME=config.prod.ini

7.2. Define the scheduled jobs in `config.ini`

The default scheduler configuration file is:

docker/ofelia/config.ini

Example content:

; file: docker/ofelia/config.ini

[job-exec "ikg-fetch-pubs"]
schedule = @every 2m
container = crisalid-ikg
command = python -m app.cli people fetch-publication-random

This declares a job named ikg-fetch-pubs that:

runs every 2 minutes (@every 2m)
uses job-exec: it executes the command inside the running crisalid-ikg container
calls the internal CLI: python -m app.cli people fetch-publication-random

7.3. Run Ofelia with Docker Compose

To start the scheduler along with the rest of the stack, include the ofelia profile, for example:

docker compose \
...
  --profile ofelia \
  up -d

🔌 Communication with Host Machine

If you want to connect external tools (on your host) to the containers, open the necessary ports.

For example, to expose RabbitMQ’s AMQP port on the host machine, edit docker/crisalid-bus/crisalid-bus.yaml and uncomment the 2nd ports line:

ports:
  - "${CRISALID_BUS_HTTP_PORT}:15672"
#  - "${CRISALID_BUS_AMQP_PORT}:5672"
expose:
  - "${CRISALID_BUS_AMQP_PORT}"

♻️ Resetting Containers

To stop and delete containers + volumes for one profile:

docker compose --profile cdb down --volumes

To also delete images:

docker compose --profile cdb down --volumes --rmi all

To ensure volumes are removed, you can also run:

docker volume rm postgres-db-volume redis-db-volume data-versioning-redis-volume
docker volume rm keycloak_postgres_data
docker volume rm svp-db-volume
docker volume rm crisalid-bus-volume
docker volume rm neo4j-data-volume neo4j-logs-volume neo4j-import-volume neo4j-plugins-volume neo4j-backups-volume

🚀 Starting the Services

To start the services, run:

docker compose --profile neo4j --profile apollo --profile crisalid-bus --profile harvester --profile ikg --profile cdb --profile keycloak --profile sovisuplus --profile ofelia up -d

This command will start the selected components in the background. You can add or remove profiles as needed.

✅ Next Steps

Once your services are up, follow the component-specific instructions in each section of the documentation. You can now:

Access the CRISalid Directory Bridge (CDB) UI at http://localhost:8081 (Airflow) and trigger DAGs to import structures and people
Access the Neo4j UI at http://localhost:7474 and explore the graph database
Access the RabbitMQ UI at http://localhost:15672 with credentials from docker/.env and monitor messages
Access SVP Harvester at http://localhost:8000 to monitor publication harvesting
Access the Apollo GraphQL UI at http://localhost:4000/graphql to explore the API through Apollo GUI
Access Keycloak at http://keycloak.local:8080 to manage users and roles
Access SoVisu+ at http://sovisuplus.local:3000 to visualize your data
Start connecting other CRISalid modules from the host machine

🧭 Back to Development Index