Spice is a SQL query, search, and LLM-inference engine, written in Rust, for data apps and agents.
Spice provides four industry standard APIs in a lightweight, portable runtime (single binary/container):
- SQL Query & Search: HTTP, Arrow Flight, Arrow Flight SQL, ODBC, JDBC, and ADBC APIs;
vector_search and text_search UDTFs.
- OpenAI-Compatible APIs: HTTP APIs for OpenAI SDK compatibility, local model serving (CUDA/Metal accelerated), and hosted model gateway.
- Iceberg Catalog REST APIs: A unified Iceberg REST Catalog API.
- MCP HTTP+SSE APIs: Integration with external tools via Model Context Protocol (MCP) using HTTP and Server-Sent Events (SSE).
🎯 Goal: Developers can focus on building data apps and AI agents confidently, knowing they are grounded in data.
Spice's primary features include:
- Data Federation: SQL query across any database, data warehouse, or data lake. Deploy with single-node or distributed multi-node query execution. Learn More (opens in a new tab).
- Data Materialization and Acceleration: Materialize, accelerate, and cache database queries with Arrow, DuckDB, SQLite, PostgreSQL, or Cayenne (Vortex+SQLite) for simplified multi-file acceleration. Read the MaterializedView interview - Building a CDN for Databases (opens in a new tab)
- Hybrid Search: Keyword, vector, and full-text search with Tantivy-powered BM25 and petabyte-scale vector similarity search via Amazon S3 Vectors or pgvector for structured and unstructured data.
- SQL LLM Inference: Call LLMs directly from SQL. Generate, summarize, and enrich data using the Spice SQL AI function or Text-to-SQL. Use Spice as an AI-database powering retrieval-augmented generation (RAG) and intelligent agents with OpenAI-compatible APIs and MCP integration. Learn More (opens in a new tab).
If you want to build with DataFusion, DuckDB, or Vortex, Spice provides a simple, flexible, and production-ready engine you can just use.
📣 Read the Spice.ai 1.0-stable announcement (opens in a new tab).
Spice is built-on industry leading technologies including Apache DataFusion (opens in a new tab), Apache Arrow, Arrow Flight, SQLite, and DuckDB.
🎥 Watch the CMU Databases Accelerating Data and AI with Spice.ai Open-Source (opens in a new tab)
🎥 Watch How to Query Data using Spice, OpenAI, and MCP (opens in a new tab)
🎥 Watch How to search with Amazon S3 Vectors (opens in a new tab)
Why Spice?
Spice simplifies building data-driven AI applications and agents by making it fast and easy to query, federate, and accelerate data from one or more sources using SQL, while grounding AI in real-time, reliable data. Co-locate datasets with apps and AI models to power AI feedback loops, enable RAG and search, and deliver fast, low-latency data-query and AI-inference with full control over cost and performance.
Latest Capabilities
- Spice Cayenne Data Accelerator (Beta): Simplified multi-file acceleration using the Vortex columnar format (opens in a new tab) + SQLite metadata. Delivers DuckDB-comparable performance without single-file scaling limitations.
- Multi-Node Distributed Query (Preview): Scale query execution across multiple nodes with Apache Ballista integration for improved performance on large datasets.
- Acceleration Snapshots: Bootstrap accelerations from S3 for fast cold starts (seconds vs. minutes). Supports ephemeral storage with persistent recovery.
- Iceberg Table Writes: Write to Iceberg tables using standard SQL
INSERT INTO for data ingestion and transformation—no Spark required.
- Petabyte-Scale Vector Search: Native Amazon S3 Vectors integration manages the full vector lifecycle from ingestion to embedding to querying. SQL-integrated hybrid search with RRF.
How is Spice different?
-
AI-Native Runtime: Spice combines data query and AI inference in a single engine, for data-grounded AI and accurate AI.
-
Application-Focused: Designed to run distributed at the application and agent level, often as a 1:1 or 1:N mapping between app and Spice instance, unlike traditional data systems built for many apps on one centralized database. It’s common to spin up multiple Spice instances—even one per tenant or customer.
-
Dual-Engine Acceleration: Supports both OLAP (Arrow/DuckDB) and OLTP (SQLite/PostgreSQL) engines at the dataset level, providing flexible performance across analytical and transactional workloads.
-
Disaggregated Storage: Separation of compute from disaggregated storage, co-locating local, materialized working sets of data with applications, dashboards, or ML pipelines while accessing source data in its original storage.
-
Edge to Cloud Native: Deploy as a standalone instance, Kubernetes sidecar, microservice, or cluster—across edge/POP, on-prem, and public clouds. Chain multiple Spice instances for tier-optimized, distributed deployments.
How does Spice compare?
Data Query and Analytics
| Feature | Spice | Trino / Presto | Dremio | ClickHouse | Materialize |
|---|
| Primary Use-Case | Data & AI apps/agents | Big data analytics | Interactive analytics | Real-time analytics | Real-time analytics |
| Primary deployment model | Sidecar | Cluster | Cluster | Cluster | Cluster |
| Federated Query Support | ✅ | ✅ | ✅ | ― | ― |
| Acceleration/Materialization | ✅ (Arrow, SQLite, DuckDB, PostgreSQL) | Intermediate storage | Reflections (Iceberg) | Materialized views | ✅ (Real-time views) |
| Catalog Support | ✅ (Iceberg, Unity Catalog, AWS Glue) | ✅ | ✅ | ― | ― |
| Query Result Caching | ✅ | ✅ | ✅ | ✅ | Limited |
| Multi-Modal Acceleration | ✅ (OLAP + OLTP) | ― | ― | ― | ― |
| Change Data Capture (CDC) | ✅ (Debezium) | ― | ― | ― | ✅ (Debezium) |
AI Apps and Agents
| Feature | Spice | LangChain | LlamaIndex | AgentOps.ai | Ollama |
|---|
| Primary Use-Case | Data & AI apps | Agentic workflows | RAG apps | Agent operations | LLM apps |
| Programming Language | Any language (HTTP interface) | JavaScript, Python | Python | Python | Any language (HTTP interface) |
| Unified Data + AI Runtime | ✅ | ― | ― | ― | ― |
| Federated Data Query | ✅ | ― | ― | ― | ― |
| Accelerated Data Access | ✅ | ― | ― | ― | ― |
| Tools/Functions | ✅ (MCP HTTP+SSE) | ✅ | ✅ | Limited | Limited |
| LLM Memory | ✅ | ✅ | ― | ✅ | ― |
| Evaluations (Evals) | ✅ | Limited | ― | Limited | ― |
| Hybrid Search | ✅ (Keyword, Vector, & Full-Text-Search) | ✅ | ✅ | Limited | Limited |
| Caching | ✅ (Query and results caching) | Limited | ― | ― | ― |
| Embeddings | ✅ (Built-in & pluggable models/DBs) | ✅ | ✅ | Limited | ― |
✅ = Fully supported
❌ = Not supported
Limited = Partial or restricted support
Example Use-Cases
Data-grounded Agentic AI Applications
Database CDN and Query Mesh
Retrieval-Augmented Generation (RAG)
- Unified Search with Vector Similarity: Perform efficient vector similarity search across structured and unstructured data sources with native Amazon S3 Vectors integration for petabyte-scale vector storage and querying. The Spice runtime manages the vector lifecycle: ingesting data, embedding it using AWS Bedrock (Amazon Titan, Cohere), HuggingFace models, or Model2Vec (500x faster static embeddings), and storing in S3 Vector buckets or pgvector. Supports cosine similarity, Euclidean distance, or dot product. SQL-integrated search via
vector_search and text_search UDTFs with hybrid search using reciprocal rank fusion (RRF). Example: SELECT * FROM vector_search(my_table, 'search query', 10) WHERE condition ORDER BY score;. Amazon S3 Vectors Cookbook Recipe (opens in a new tab)
- Semantic Knowledge Layer: Define a semantic context model to enrich data for AI. Semantic Model Feature Documentation (opens in a new tab)
- Text-to-SQL: Convert natural language queries into SQL using built-in NSQL and sampling tools for accurate query. Text-to-SQL Recipe (opens in a new tab)
- Model and Data Evaluations: Assess model performance and data quality with integrated evaluation tools. Language Model Evaluations Recipe (opens in a new tab)
FAQ
-
Is Spice a cache? No specifically; you can think of Spice data acceleration as an active cache, materialization, or data prefetcher. A cache would fetch data on a cache-miss while Spice prefetches and materializes filtered data on an interval, trigger, or as data changes using CDC. In addition to acceleration Spice supports results caching (opens in a new tab).
-
Is Spice a CDN for databases? Yes, a common use-case for Spice is as a CDN for different data sources. Using CDN concepts, Spice enables you to ship (load) a working set of your database (or data lake, or data warehouse) where it's most frequently accessed, like from a data-intensive application or for AI context.
➡️ Docs FAQ (opens in a new tab)
Watch a 30-sec BI dashboard acceleration demo
https://github.com/spiceai/spiceai/assets/80174/7735ee94-3f4a-4983-a98e-fe766e79e03a (opens in a new tab)
See more demos on YouTube (opens in a new tab).
Supported Data Connectors
| Name | Description | Status | Protocol/Format |
|---|
databricks (mode: delta_lake) | Databricks (opens in a new tab) | Stable | S3/Delta Lake |
delta_lake | Delta Lake | Stable | Delta Lake |
dremio | Dremio (opens in a new tab) | Stable | Arrow Flight |
duckdb | DuckDB | Stable | Embedded |
file | File | Stable | Parquet, CSV |
github | GitHub | Stable | GitHub API |
postgres | PostgreSQL | Stable | |
s3 | S3 (opens in a new tab) | Stable | Parquet, CSV |
mysql | MySQL | Stable | |
spice.ai | Spice.ai (opens in a new tab) | Stable | Arrow Flight |
graphql | GraphQL | Release Candidate | JSON |
dynamodb | Amazon DynamoDB | Release Candidate | |
databricks (mode: spark_connect) | Databricks (opens in a new tab) | Beta | Spark Connect (opens in a new tab) |
flightsql | FlightSQL | Beta | Arrow Flight SQL |
iceberg | Apache Iceberg (opens in a new tab) | Beta | Parquet |
mssql | Microsoft SQL Server | Beta | Tabular Data Stream (TDS) |
odbc | ODBC | Beta | ODBC |
snowflake | Snowflake | Beta | Arrow |
spark | Spark | Beta | Spark Connect (opens in a new tab) |
oracle | Oracle | Alpha | Oracle ODPI-C (opens in a new tab) |
abfs | Azure BlobFS | Alpha | Parquet, CSV |
clickhouse | Clickhouse | Alpha | |
debezium | Debezium CDC | Alpha | Kafka + JSON |
kafka | Kafka | Alpha | Kafka + JSON |
ftp, sftp | FTP/SFTP | Alpha | Parquet, CSV |
glue | AWS Glue (opens in a new tab) | Alpha | Iceberg, Parquet, CSV |
http, https | HTTP(s) | Alpha | Parquet, CSV, JSON |
imap | IMAP | Alpha | IMAP Emails |
localpod | Local dataset replication (opens in a new tab) | Alpha | |
mongodb | MongoDB | Alpha | |
sharepoint | Microsoft SharePoint | Alpha | Unstructured UTF-8 documents |
elasticsearch | ElasticSearch | Roadmap | |
Supported Data Accelerators
Supported Model Providers
| Name | Description | Status | ML Format(s) | LLM Format(s) |
|---|
openai | OpenAI (or compatible) LLM endpoint | Release Candidate | - | OpenAI-compatible HTTP endpoint |
file | Local filesystem | Release Candidate | ONNX | GGUF, GGML, SafeTensor |
huggingface | Models hosted on HuggingFace | Release Candidate | ONNX | GGUF, GGML, SafeTensor |
spice.ai | Models hosted on the Spice.ai Cloud Platform | | ONNX | OpenAI-compatible HTTP endpoint |
azure | Azure OpenAI | | - | OpenAI-compatible HTTP endpoint |
bedrock | Amazon Bedrock (Nova models) | Alpha | - | OpenAI-compatible HTTP endpoint |
anthropic | Models hosted on Anthropic | Alpha | - | OpenAI-compatible HTTP endpoint |
xai | Models hosted on xAI | Alpha | - | OpenAI-compatible HTTP endpoint |
Supported Embeddings Providers
| Name | Description | Status | ML Format(s) | LLM Format(s)* |
|---|
openai | OpenAI (or compatible) LLM endpoint | Release Candidate | - | OpenAI-compatible HTTP endpoint |
file | Local filesystem | Release Candidate | ONNX | GGUF, GGML, SafeTensor |
huggingface | Models hosted on HuggingFace | Release Candidate | ONNX | GGUF, GGML, SafeTensor |
model2vec | Static embeddings (500x faster) | Release Candidate | Model2Vec | - |
azure | Azure OpenAI | Alpha | - | OpenAI-compatible HTTP endpoint |
bedrock | AWS Bedrock (e.g., Titan, Cohere) | Alpha | - | OpenAI-compatible HTTP endpoint |
Supported Vector Stores
| Name | Description | Status |
|---|
s3_vectors | Amazon S3 Vectors for petabyte-scale vector storage and querying | Alpha |
pgvector | PostgreSQL with pgvector extension | Alpha |
duckdb_vector | DuckDB with vector extension for efficient vector storage and search | Alpha |
sqlite_vec | SQLite with sqlite-vec extension for lightweight vector operations | Alpha |
Supported Catalogs
Catalog Connectors connect to external catalog providers and make their tables available for federated SQL query in Spice. Configuring accelerations for tables in external catalogs is not supported. The schema hierarchy of the external catalog is preserved in Spice.
| Name | Description | Status | Protocol/Format |
|---|
spice.ai | Spice.ai Cloud Platform | Stable | Arrow Flight |
unity_catalog | Unity Catalog | Stable | Delta Lake |
databricks | Databricks | Beta | Spark Connect, S3/Delta Lake |
iceberg | Apache Iceberg | Beta | Parquet |
glue | AWS Glue | Alpha | CSV, Parquet, Iceberg |
⚡️ Quickstart (Local Machine)
https://github.com/spiceai/spiceai/assets/88671039/85cf9a69-46e7-412e-8b68-22617dcbd4e0 (opens in a new tab)
Installation
Install the Spice CLI:
On macOS, Linux, and WSL:
curl https://install.spiceai.org | /bin/bash
Or using brew:
brew install spiceai/spiceai/spice
On Windows using PowerShell:
iex ((New-Object System.Net.WebClient).DownloadString("https://install.spiceai.org/Install.ps1"))
Usage
Step 1. Initialize a new Spice app with the spice init command:
spice init spice_qs
A spicepod.yaml file is created in the spice_qs directory. Change to that directory:
cd spice_qs
Step 2. Start the Spice runtime:
spice run
Example output will be shown as follows:
2025/01/20 11:26:10 INFO Spice.ai runtime starting...
2025-01-20T19:26:10.679068Z INFO runtime::init::dataset: No datasets were configured. If this is unexpected, check the Spicepod configuration.
2025-01-20T19:26:10.679716Z INFO runtime::flight: Spice Runtime Flight listening on 127.0.0.1:50051
2025-01-20T19:26:10.679786Z INFO runtime::metrics_server: Spice Runtime Metrics listening on 127.0.0.1:9090
2025-01-20T19:26:10.680140Z INFO runtime::http: Spice Runtime HTTP listening on 127.0.0.1:8090
2025-01-20T19:26:10.879126Z INFO runtime::init::results_cache: Initialized sql results cache; max size: 128.00 MiB, item ttl: 1s
The runtime is now started and ready for queries.
Step 3. In a new terminal window, add the spiceai/quickstart Spicepod. A Spicepod is a package of configuration defining datasets and ML models.
spice add spiceai/quickstart
The spicepod.yaml file will be updated with the spiceai/quickstart dependency.
version: v1
kind: Spicepod
name: spice_qs
dependencies:
- spiceai/quickstart
The spiceai/quickstart Spicepod will add a taxi_trips data table to the runtime which is now available to query by SQL.
2025-01-20T19:26:30.011633Z INFO runtime::init::dataset: Dataset taxi_trips registered (s3://spiceai-demo-datasets/taxi_trips/2024/), acceleration (arrow), results cache enabled.
2025-01-20T19:26:30.013002Z INFO runtime::accelerated_table::refresh_task: Loading data for dataset taxi_trips
2025-01-20T19:26:40.312839Z INFO runtime::accelerated_table::refresh_task: Loaded 2,964,624 rows (399.41 MiB) for dataset taxi_trips in 10s 299ms
Step 4. Start the Spice SQL REPL:
spice sql
The SQL REPL inferface will be shown:
Welcome to the Spice.ai SQL REPL! Type 'help' for help.
show tables; -- list available tables
sql>
Enter show tables; to display the available tables for query:
sql> show tables;
+---------------+--------------+---------------+------------+
| table_catalog | table_schema | table_name | table_type |
+---------------+--------------+---------------+------------+
| spice | public | taxi_trips | BASE TABLE |
| spice | runtime | query_history | BASE TABLE |
| spice | runtime | metrics | BASE TABLE |
+---------------+--------------+---------------+------------+
Time: 0.022671708 seconds. 3 rows.
Enter a query to display the longest taxi trips:
SELECT trip_distance, total_amount FROM taxi_trips ORDER BY trip_distance DESC LIMIT 10;
Output:
+---------------+--------------+
| trip_distance | total_amount |
+---------------+--------------+
| 312722.3 | 22.15 |
| 97793.92 | 36.31 |
| 82015.45 | 21.56 |
| 72975.97 | 20.04 |
| 71752.26 | 49.57 |
| 59282.45 | 33.52 |
| 59076.43 | 23.17 |
| 58298.51 | 18.63 |
| 51619.36 | 24.2 |
| 44018.64 | 52.43 |
+---------------+--------------+
Time: 0.045150667 seconds. 10 rows.
⚙️ Runtime Container Deployment
Using the Docker image (opens in a new tab) locally:
docker pull spiceai/spiceai
In a Dockerfile:
from spiceai/spiceai:latest
Using Helm:
helm repo add spiceai https://helm.spiceai.org
helm install spiceai spiceai/spiceai
🏎️ Next Steps
Explore the Spice.ai Cookbook
The Spice.ai Cookbook is a collection of recipes and examples for using Spice. Find it at https://github.com/spiceai/cookbook (opens in a new tab).
Access ready-to-use Spicepods and datasets hosted on the Spice.ai Cloud Platform using the Spice runtime. A list of public Spicepods is available on Spicerack: https://spicerack.org/ (opens in a new tab).
To use public datasets, create a free account on Spice.ai:
-
Visit spice.ai (opens in a new tab) and click Try for Free.
-
After creating an account, create an app to generate an API key.
Once set up, you can access ready-to-use Spicepods including datasets. For this demonstration, use the taxi_trips dataset from the Spice.ai Quickstart (opens in a new tab).
Step 1. Initialize a new project.
# Initialize a new Spice app
spice init spice_app
# Change to app directory
cd spice_app
Step 2. Log in and authenticate from the command line using the spice login command. A pop up browser window will prompt you to authenticate:
spice login
Step 3. Start the runtime:
# Start the runtime
spice run
Step 4. Configure the dataset:
In a new terminal window, configure a new dataset using the spice dataset configure command:
spice dataset configure
Enter a dataset name that will be used to reference the dataset in queries. This name does not need to match the name in the dataset source.
dataset name: (spice_app) taxi_trips
Enter the description of the dataset:
description: Taxi trips dataset
Enter the location of the dataset:
from: spice.ai/spiceai/quickstart/datasets/taxi_trips
Select y when prompted whether to accelerate the data:
Locally accelerate (y/n)? y
You should see the following output from your runtime terminal:
2024-12-16T05:12:45.803694Z INFO runtime::init::dataset: Dataset taxi_trips registered (spice.ai/spiceai/quickstart/datasets/taxi_trips), acceleration (arrow, 10s refresh), results cache enabled.
2024-12-16T05:12:45.805494Z INFO runtime::accelerated_table::refresh_task: Loading data for dataset taxi_trips
2024-12-16T05:13:24.218345Z INFO runtime::accelerated_table::refresh_task: Loaded 2,964,624 rows (8.41 GiB) for dataset taxi_trips in 38s 412ms.
Step 5. In a new terminal window, use the Spice SQL REPL to query the dataset
spice sql
SELECT tpep_pickup_datetime, passenger_count, trip_distance from taxi_trips LIMIT 10;
The output displays the results of the query along with the query execution time:
+----------------------+-----------------+---------------+
| tpep_pickup_datetime | passenger_count | trip_distance |
+----------------------+-----------------+---------------+
| 2024-01-11T12:55:12 | 1 | 0.0 |
| 2024-01-11T12:55:12 | 1 | 0.0 |
| 2024-01-11T12:04:56 | 1 | 0.63 |
| 2024-01-11T12:18:31 | 1 | 1.38 |
| 2024-01-11T12:39:26 | 1 | 1.01 |
| 2024-01-11T12:18:58 | 1 | 5.13 |
| 2024-01-11T12:43:13 | 1 | 2.9 |
| 2024-01-11T12:05:41 | 1 | 1.36 |
| 2024-01-11T12:20:41 | 1 | 1.11 |
| 2024-01-11T12:37:25 | 1 | 2.04 |
+----------------------+-----------------+---------------+
Time: 0.00538925 seconds. 10 rows.
You can experiment with the time it takes to generate queries when using non-accelerated datasets. You can change the acceleration setting from true to false in the datasets.yaml file.
📄 Documentation
Comprehensive documentation is available at spiceai.org/docs (opens in a new tab).
Over 45 quickstarts and samples available in the Spice Cookbook (opens in a new tab).
🔌 Extensibility
Spice.ai is designed to be extensible with extension points documented at EXTENSIBILITY.md (opens in a new tab). Build custom Data Connectors (opens in a new tab), Data Accelerators (opens in a new tab), Catalog Connectors (opens in a new tab), Secret Stores (opens in a new tab), Models (opens in a new tab), or Embeddings (opens in a new tab).
🔨 Upcoming Features
🚀 See the Roadmap (opens in a new tab) for upcoming features.
🤝 Connect with us
We greatly appreciate and value your support! You can help Spice in a number of ways:
⭐️ star this repo! Thank you for your support! 🙏
