Quick Start

This guide assumes you have already installed IOWarp. It walks you through activating your environment, the default configuration, starting the CLIO Runtime, and running a Context Exploration Engine (CEE) example.

1. Activate Your Environment

Pip

No activation needed -- the clio_run CLI and Python modules are available once the package is installed in your Python environment.

Conda

conda activate iowarp

Docker

If you are using Docker, the environment is already set up inside the container. Exec into a running container:

docker exec -it iowarp bash

Spack

spack load iowarp

Verify the environment is active:

clio_run --help

2. Default Configuration

During installation a default ~/.clio/clio.yaml is seeded for you. You can edit it directly or point the runtime at a custom file with CLIO_SERVER_CONF=/path/to/config.yaml. Existing files are never overwritten on reinstall.

The default configuration starts 4 worker threads on port 9413 and composes these modules automatically:

Module	Pool	Purpose
clio_bdev	301.0	DRAM block device (0g = 80% of total system DRAM)
clio_cae_core	400.0	Context Assimilation Engine — forwards its data path to CTE
clio_cte_core	512.0	Context Transfer Engine: a DRAM tier plus a 10 GB persistent disk tier at ~/.clio/cte_disk_tier.dat, with metadata logging to ~/.clio/cte_metadata_log
clio_cte_replication	561.0	One persistent replica of every blob on the disk tier
clio_cte_indexer	564.0	BM25 index that serves semantic search
clio_cte_cache	563.0	Node-local raw copies (the zero-IPC read fast path)
clio_cte_filesystem	560.0	POSIX-style filesystem the FUSE / POSIX adapters drive

The last four form the interposition chain — each speaks the CTE core's own task interface and stacks over the next via next_pool_id:

cache(563.0) → indexer(564.0) → replication(561.0) → core(512.0)

Because they share the core's vocabulary, none of this changes how you call CTE. Every layer is optional: delete its compose entry and re-point the entry above it. See Cache / Replication / Indexing ChiMods.

3. Start the Runtime

# Start in the background
clio_run start &

# Verify it is running
clio_run monitor --once

# When done
clio_run stop

4. Context Exploration Engine Example

The Context Exploration Engine (CEE) lets you assimilate data into IOWarp, query for it by name or regex, retrieve it, and clean up -- all from Python.

Save the following as cee_quickstart.py:

#!/usr/bin/env python3
"""IOWarp CEE Quickstart -- assimilate, query, retrieve, destroy."""

import os
import tempfile

import clio_cee as cee

# -- 1. Create a sample file -------------------------------------------
data = b"Hello from IOWarp! " * 50_000          # ~950 KB
tmp = tempfile.NamedTemporaryFile(delete=False, suffix=".bin")
tmp.write(data)
tmp.close()
print(f"Created test file: {tmp.name} ({len(data):,} bytes)")

# -- 2. Initialise the CEE interface -----------------------------------
#    ContextInterface connects to the running CLIO Runtime.
iface = cee.ContextInterface()

# -- 3. Bundle (assimilate) the file -----------------------------------
tag = "quickstart_demo"
ctx = cee.AssimilationCtx(
    src=f"file::{tmp.name}",       # source: local file  (note :: not ://)
    dst=f"iowarp::{tag}",          # destination tag in IOWarp
    format="binary",               # raw binary ingest
)
rc = iface.context_bundle([ctx])
assert rc == 0, f"context_bundle failed (rc={rc})"
print(f"Assimilated file into tag '{tag}'")

# -- 4. Query for blobs in the tag -------------------------------------
blobs = iface.context_query(tag, ".*", 0)   # regex ".*" matches all blobs
print(f"Found {len(blobs)} blob(s): {blobs}")

# -- 5. Retrieve the data back -----------------------------------------
packed = iface.context_retrieve(tag, ".*", 0)
if packed:
    print(f"Retrieved {len(packed[0]):,} bytes")

# -- 6. Destroy the tag ------------------------------------------------
iface.context_destroy([tag])
print(f"Destroyed tag '{tag}'")

# -- Cleanup ------------------------------------------------------------
os.unlink(tmp.name)
print("Done!")

Run it (make sure the runtime is still running in the background):

python3 cee_quickstart.py

Expected output:

Created test file: /tmp/tmpXXXXXXXX.bin (950,000 bytes)
Assimilated file into tag 'quickstart_demo'
Found 2 blob(s): ['chunk_0', 'description']
Retrieved 950,029 bytes
Destroyed tag 'quickstart_demo'
Done!

5. Key Environment Variables

Variable	Description
CLIO_SERVER_CONF	Path to YAML configuration file (highest priority)
CLIO_IPC_MODE	Force a transport: SHM, IPC (Unix socket), or TCP. Unset (the default) auto-probes for the fastest usable one in that order.
CLIO_PORT	Override the RPC port (default: 9413). Takes priority over YAML config.
CLIO_SERVER_ADDR	Address clients dial to reach the runtime (default: 127.0.0.1).
CLIO_BIND_ADDR	Address the runtime's listeners bind to (default: 0.0.0.0).
CLIO_EPHEMERAL	1 starts the runtime bare — the compose section is skipped.
CLIO_CTE_POOL	Bind the CTE client to an interposing pool, e.g. 563.0 for the chain top.
CTP_LOG_LEVEL	Logging verbosity: debug, info, success, warning, error, fatal
CTP_LOG_OUT	Also write log output to this file.

See the Configuration Reference for the full list.

Next Steps

• Edit ~/.clio/clio.yaml to tune thread counts, storage tiers, or add file-backed block devices
• Configuration Reference -- full parameter documentation
• HPC Deployment -- multi-node cluster setup
• CLIO Kit -- MCP servers for AI-powered scientific computing
• SDK Reference -- component APIs and development guides
• Deprecation Notes -- legacy CLI / env / config names that still work as aliases

Support

• Open an issue on the GitHub repository
• Join the Zulip Chat
• Visit the IOWarp website
• Email: grc@illinoistech.edu