RAG Snap

A CLI-based RAG implementation to locally manage knowledge bases and chat with them.

Quick start

Prerequisites

Before starting, the RAG snap depends on the OpenSearch snap, and optionally, one of the Inference snaps:

Install and setup the OpenSearch snap.

During the creation of the certificates, ensure that the ingest and ml roles are set in the node.

sudo snap run opensearch.setup                  \
    --node-name vdb0                            \
    --node-roles cluster_manager,data,ingest,ml \
    --tls-priv-key-root-pass root1234           \
    --tls-priv-key-admin-pass admin1234         \
    --tls-priv-key-node-pass node1234           \
    --tls-init-setup yes

Increase the JVM heap size to fit the sentence-transformer and cross-encoder models (at least 6 GB is recommended; adjust to your machine's available RAM):

echo '-Xms6g' | sudo tee /var/snap/opensearch/current/etc/opensearch/jvm.options.d/heap.options
echo '-Xmx6g' | sudo tee -a /var/snap/opensearch/current/etc/opensearch/jvm.options.d/heap.options

sudo snap restart opensearch

Validate your OpenSearch snap node roles:

curl -k -u admin:admin https://localhost:9200/_cat/nodes?v

(Recommended) Use AWS Bedrock as the Inference Server

Warning: When using a third-party inference API, your prompts and retrieved context are sent to an external service. Do not ingest or ask about confidential information in that configuration.

(Alternative) Install a Inference snap of your selection.

Ensure you are using the right engine available in your machine for better performance:

sudo <inference-snap-name> show-engine

Test your Inference snap installation:

curl http://localhost:8324/v1/chat/completions \
  -H 'Content-Type: application/json'          \
  -d '{
    "messages": [
      {"role": "system", "content": "You are a helpful assistant."},
      {"role": "user", "content": "Hello!"}
    ]
  }'

Installation

sudo snap install --dangerous ./rag_*.snap

Package setup

The package comes with sensible defaults set by the install hook. Override them only if your services run on non-default hosts or ports:

If you are using Bedrock as the Inference Server

sudo rag set --package chat.http.host="bedrock-runtime.us-east-2.amazonaws.com"
sudo rag set --package chat.http.port="443"
sudo rag set --package chat.http.tls="true"
sudo rag set --package chat.http.path="openai/v1"
sudo rag set --package knowledge.http.host="127.0.0.1"
sudo rag set --package knowledge.http.port="9200"
sudo rag set --package knowledge.http.tls="true"
sudo rag set --package tika.http.path="tika"
sudo rag set --package tika.http.port="9998"
sudo rag set --package tika.http.host="127.0.0.1"

export CHAT_API_KEY="bedrock-api-key-****"

rag chat mistral.mistral-large-3-675b-instruct

If you are using an Inference Snap

sudo rag set --package chat.http.host="127.0.0.1"
sudo rag set --package chat.http.port="8324"
sudo rag set --package chat.http.path="v1"
sudo rag set --package knowledge.http.host="127.0.0.1"
sudo rag set --package knowledge.http.port="9200"
sudo rag set --package knowledge.http.tls="true"
sudo rag set --package tika.http.path="tika"
sudo rag set --package tika.http.port="9998"
sudo rag set --package tika.http.host="127.0.0.1"

The optional secrets like OPENSEARCH_USERNAME, OPENSEARCH_PASSWORD, and CHAT_API_KEY are provided via environment variables:

export OPENSEARCH_USERNAME="admin"
export OPENSEARCH_PASSWORD="admin"

The snap manages the tika-server service. To start it run:

sudo snap start rag.tika-server

The status of the snap can be checked with rag status.

Basic Usage

Initialize pipelines and models

rag knowledge init

It will print the model IDs so you can add them to the package config:

sudo rag set --package knowledge.model.embedding=<embedding-model-id>
sudo rag set --package knowledge.model.rerank=<rerank-model-id>

Manage your knowledge bases

knowledge init sets up the pipelines and index template, but does not create any index yet. Create your first knowledge base (the default name is used by chat when no other base is selected):

rag k create default

Additional knowledge bases can be created to separate content into distinct contexts that you can activate during a chat session.

Create an example knowledge base:

rag k create example

List the knowledge bases with rag k list.

Ingest files into the example and default bases:

rag k ingest example <source-id-file> --file <path-to-local-file>

rag k ingest default <source-id-url> --url <url-to-document>

List the added sources with rag k list -s.

Back up a knowledge base to a compressed archive:

rag k export example --compress
# → ./example-export.tar.gz

Restore it — on the same machine or another — without re-embedding:

rag k import --input ./example-export.tar.gz
# restores under the original name stored in the archive

rag k import example-copy --input ./example-export.tar.gz
# restores under a different name

Chat with your knowledge bases

Start a new conversation:

rag chat

Activate the relevant knowledge bases for your conversation and ask questions:

Using inference server at http://127.0.0.1:8324/v1
Using the `default` knowledge base at https://127.0.0.1:9200
	> Use `/use-knowledge` to see other available knowledge bases

Type your prompt, then ENTER to submit. CTRL-C to quit
export CHAT_API_KEY="bedrock-api-key-****"

rag chat mistral.mistral-large-3-675b-instruct
» /use-knowledge 
┃ Select active knowledge bases
┃   • default (27 docs, 671.1kb)
┃ > ✓ example (102 docs, 1.2mb)
x toggle • ↑ up • ↓ down • / filter • enter submit • ctrl+a select all

» This a relevant question that can be answered from the example knowledge base ... ?

For a more detail usage, please see the usage docs.