Skip to main content

Quick Start

This guide assumes you have already installed IOWarp. In a few minutes you will start the CLIO Runtime, mount the IOWarp filesystem on your machine, write files to it with ordinary tools, and watch what happens in the runtime's dashboard.

1. Activate Your Environment​

No activation needed -- the clio_run CLI and Python modules are available once the package is installed in your Python environment.

Verify the environment is active:

clio_run --help

2. Start the Runtime​

clio_run start &

That is the whole deployment. The default ~/.clio/clio.yaml seeded at install time brings up everything the filesystem needs on this node:

  • a DRAM block device (80% of system memory by default) and a 10 GB persistent disk tier at ~/.clio/cte_disk_tier.dat
  • the Context Transfer Engine core with its interposition chain (cache → indexer → replication → core), plus the Context Assimilation Engine
  • the filesystem pool (clio_cte_filesystem) that the FUSE mount in the next step drives — paths become CTE tags, file contents become 1 MiB page blobs, and every page rides the chain
  • the web dashboard on http://127.0.0.1:8080

Confirm it is up:

clio_run monitor --once

or open the dashboard in a browser. Nothing here needs to be composed by hand; every layer is optional and can be trimmed later — see the shipped default in the Configuration Reference.

3. Mount the Filesystem​

The FUSE adapter (clio_cte_fuse) presents the runtime's filesystem pool as an ordinary directory. It attaches as a client to the runtime you just started, so set CLIO_WITH_RUNTIME=0 — without it the adapter would spin up a private runtime of its own and nothing you write would be visible to anyone else.

Needs the distro's FUSE 3 runtime (sudo apt install fuse3 / sudo dnf install fuse3). In a second terminal:

mkdir -p ~/cte-mnt
CLIO_WITH_RUNTIME=0 clio_cte_fuse ~/cte-mnt -f

-f keeps the adapter in the foreground so you can see what it is doing; omit it to run it as a daemon. Everything about the mount — other mountpoints, the FSKit backend on macOS, mounting at a directory on Windows, libfuse options — is in the FUSE Adapter guide.

4. Use It​

Any program can now read and write files on the mount with standard tools; nothing is IOWarp-specific.

echo "Hello, IOWarp!" > ~/cte-mnt/greeting.txt
mkdir -p ~/cte-mnt/data
cp dataset.csv ~/cte-mnt/data/

cat ~/cte-mnt/greeting.txt
ls -l ~/cte-mnt/

5. Watch It in the Dashboard​

Open http://127.0.0.1:8080. While files land on the mount:

  • Cluster shows this node's CPU and memory and the workers' processed task count climbing.
  • Pools lists every pool the runtime composed. Click clio_cte_core to see the storage targets your writes are landing on, or the clio_bdev card to watch the DRAM device's capacity and statistics.
  • Config shows the settings the daemon actually came up with.

Take the Dashboard tour for the rest, including creating pools from the browser.

6. Stop​

Unmount, then stop the runtime:

fusermount3 -u ~/cte-mnt      # or Ctrl-C in the clio_cte_fuse terminal
clio_run stop

Files on the DRAM tier are gone when the runtime stops; the persistent disk tier and its metadata log under ~/.clio/ survive a restart.

7. Optional: Assimilate Data from Python​

The filesystem is one door into IOWarp. The Context Exploration Engine (CEE) is another: it lets you assimilate data into IOWarp, query for it by name or regex, retrieve it, and clean up -- all from Python.

Save the following as cee_quickstart.py:

#!/usr/bin/env python3
"""IOWarp CEE Quickstart -- assimilate, query, retrieve, destroy."""

import os
import tempfile

import clio_cee as cee

# -- 1. Create a sample file -------------------------------------------
data = b"Hello from IOWarp! " * 50_000          # ~950 KB
tmp = tempfile.NamedTemporaryFile(delete=False, suffix=".bin")
tmp.write(data)
tmp.close()
print(f"Created test file: {tmp.name} ({len(data):,} bytes)")

# -- 2. Initialise the CEE interface -----------------------------------
#    ContextInterface connects to the running CLIO Runtime.
iface = cee.ContextInterface()

# -- 3. Bundle (assimilate) the file -----------------------------------
tag = "quickstart_demo"
ctx = cee.AssimilationCtx(
    src=f"file::{tmp.name}",       # source: local file  (note :: not ://)
    dst=f"iowarp::{tag}",          # destination tag in IOWarp
    format="binary",               # raw binary ingest
)
rc = iface.context_bundle([ctx])
assert rc == 0, f"context_bundle failed (rc={rc})"
print(f"Assimilated file into tag '{tag}'")

# -- 4. Query for blobs in the tag -------------------------------------
blobs = iface.context_query(tag, ".*", 0)   # regex ".*" matches all blobs
print(f"Found {len(blobs)} blob(s): {blobs}")

# -- 5. Retrieve the data back -----------------------------------------
packed = iface.context_retrieve(tag, ".*", 0)
if packed:
    print(f"Retrieved {len(packed[0]):,} bytes")

# -- 6. Destroy the tag ------------------------------------------------
iface.context_destroy([tag])
print(f"Destroyed tag '{tag}'")

# -- Cleanup ------------------------------------------------------------
os.unlink(tmp.name)
print("Done!")

Run it against a running runtime (clio_run start & again if you stopped it above):

python3 cee_quickstart.py

Expected output:

Created test file: /tmp/tmpXXXXXXXX.bin (950,000 bytes)
Assimilated file into tag 'quickstart_demo'
Found 2 blob(s): ['chunk_0', 'description']
Retrieved 950,029 bytes
Destroyed tag 'quickstart_demo'
Done!

8. Key Environment Variables​

VariableDescription
CLIO_SERVER_CONFPath to YAML configuration file (highest priority)
CLIO_IPC_MODEForce a transport: SHM, IPC (Unix socket), or TCP. Unset (the default) auto-probes for the fastest usable one in that order.
CLIO_PORTOverride the RPC port (default: 9413). Takes priority over YAML config.
CLIO_SERVER_ADDRAddress clients dial to reach the runtime (default: 127.0.0.1).
CLIO_BIND_ADDRAddress the runtime's listeners bind to (default: 0.0.0.0).
CLIO_EPHEMERAL1 starts the runtime bare — the compose section is skipped.
CLIO_WITH_RUNTIME0 makes a client (such as clio_cte_fuse) attach to the running daemon instead of embedding its own runtime.
CLIO_VIZ_PORT / CLIO_VIZ_BIND / CLIO_VIZ_ENABLEDashboard port (default 8080), bind address (default 127.0.0.1), and on/off.
CLIO_CTE_POOLBind the CTE client to an interposing pool, e.g. 563.0 for the chain top.
CTP_LOG_LEVELLogging verbosity: debug, info, success, warning, error, fatal
CTP_LOG_OUTAlso write log output to this file.

See the Configuration Reference for the full list.

Next Steps​

  • Dashboard -- tour the runtime's web dashboard and create pools from the browser
  • FUSE Adapter -- every mount option, platform notes, and troubleshooting
  • Edit ~/.clio/clio.yaml to tune thread counts, storage tiers, or add file-backed block devices
  • Configuration Reference -- full parameter documentation
  • HPC Deployment -- multi-node cluster setup
  • CLIO Kit -- MCP servers for AI-powered scientific computing
  • SDK Reference -- component APIs and development guides
  • Deprecation Notes -- legacy CLI / env / config names that still work as aliases

Support​