Data Vault

Installation

CODEBLOCK0

A persistent data store using the Lance columnar format for fast ML data access.

Quick Start

CODEBLOCK1

Note: list-datasets-info shows dataset metadata (schema, field types, record count) — it does not return the actual data rows. Use read-dataset to retrieve records.

Storage Location

DataSets are created and stored on the current path '.'

Critical Behavior: Data Type Strictness

⚠️ Lance is strict about data types — they CANNOT change after the first record

When you append the first record to a dataset, Lance infers the data type for each field. All subsequent records MUST use the same types.

Example — this FAILS:

CODEBLOCK2

Correct approach — maintain consistent types:

CODEBLOCK3

Why This Matters

Unlike traditional databases that may coerce types, Lance rejects type mismatches. If you store numbers as strings initially, you must always pass strings. Plan your schema carefully.

Initialization Workflow

When starting a session, always initialize by listing existing datasets first:

CODEBLOCK4

Example output:

CODEBLOCK5

Understanding field_types

Important: If field_types is empty, the first append will define types. Be deliberate about the first record's types.

Commands Reference

Create Dataset

CODEBLOCK6

Creates a metadata entry. Fields have no types until first append.

Append Record

CODEBLOCK7

Appends one record. Types are inferred from first record.

Batch Append

CODEBLOCK8

Example: INLINECODE5

Update Record

CODEBLOCK9

Updates fields for a specific record by ID.

Delete Record

CODEBLOCK10

List All Datasets

CODEBLOCK11

Get Dataset Info

CODEBLOCK12

Returns schema, field types (if data exists), and record count.

List All Datasets with Full Info

CODEBLOCK13

Recommended for initialization. Returns all datasets with complete metadata.

Get Dataset Path

CODEBLOCK14

Backup Dataset

CODEBLOCK15

Count Records

CODEBLOCK16

Read All Records

Returns all records from the dataset as a list of objects.

CODEBLOCK17

Drop Dataset

Requires confirmation if have not created a backup beforehand.

Delete the entire dataset and its metadata.

CODEBLOCK18

Internal fields available in every dataset:

List Records (Paginated)

CODEBLOCK19

Returns records with optional pagination.

Get Single Record

CODEBLOCK20

Retrieves a specific record by its UUID.

Get Dataset Info

CODEBLOCK21

Returns schema, field types (if data exists), and record count.

Response Format

All commands return JSON:

CODEBLOCK22

Internal Fields

Every dataset automatically includes:

• - _id — UUID for each record
• INLINECODE9 — timestamp of last insert/update

These are managed automatically — when appending, only provide your defined fields.

Data Type Inference

Lance infers types from the first record:

CLI caveat: When passing via command line, all values are strings. To ensure integer types, initialize with actual integers in a script rather than CLI.

Tips

1. 1. Initialize at session start: Run list-datasets-info to understand what data already exists
2. Plan your schema: First record determines types for the entire dataset
3. Use batch append when adding multiple records: More efficient than individual appends

Requirements

Dependencies are declared in frontmatter (metadata.openclaw.install) and handled by the OpenClaw install system via uv. The Python packages required are:

• - pylance — The Lance columnar format library.

⚠️ Naming note: Despite the PyPI package being named pylance, the library is imported as import lance in Python code. This is the official Lance project naming convention — it is NOT the VS Code "pylance" language server. See lance.org for details.

• - pandas — Data manipulation