Skip to content

Commit 1618b2f

Browse files
krishna-ybkrishna-yb
committed
feat: Add YugabyteDB embedding store examples
- Add four comprehensive examples demonstrating YugabyteDB integration: * Basic embedding store with similarity search * Metadata filtering with JSONB storage * PostgreSQL JDBC driver usage (recommended) * YugabyteDB Smart Driver for distributed deployments - Include helper scripts for easy execution: * run-example.sh: Run individual examples * run-all-examples.sh: Run all examples sequentially - Add comprehensive README with usage instructions - Use Testcontainers for isolated testing environment - Implement proper resource cleanup to avoid thread warnings - Configure containers matching langchain4j-community test standards
1 parent 0df7c86 commit 1618b2f

9 files changed

+701
-0
lines changed

‎pom.xml‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -53,6 +53,7 @@
5353
<module>helidon-examples</module>
5454
<module>payara-micro-example</module>
5555
<module>watsonx-ai-examples</module>
56+
<module>yugabytedb-example</module>
5657
</modules>
5758

5859
</project>

‎yugabytedb-example/README.md‎

Lines changed: 105 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,105 @@
1+
# YugabyteDB Embedding Store Examples
2+
3+
This directory contains examples demonstrating how to use YugabyteDB as an embedding store with LangChain4j.
4+
5+
## Prerequisites
6+
7+
- Java 17+
8+
- Docker (for Testcontainers)
9+
- Maven 3.8+
10+
11+
## Examples
12+
13+
### 1. Basic Example (`YugabyteDBEmbeddingStoreExample`)
14+
Demonstrates basic embedding storage and similarity search operations.
15+
16+
```bash
17+
./run-example.sh YugabyteDBEmbeddingStoreExample
18+
```
19+
20+
### 2. Metadata Example (`YugabyteDBEmbeddingStoreWithMetadataExample`)
21+
Shows how to store and filter embeddings using JSONB metadata.
22+
23+
```bash
24+
./run-example.sh YugabyteDBEmbeddingStoreWithMetadataExample
25+
```
26+
27+
### 3. PostgreSQL Driver Example (`YugabyteDBWithPostgreSQLDriverExample`)
28+
Demonstrates using the standard PostgreSQL JDBC driver (recommended).
29+
30+
```bash
31+
./run-example.sh YugabyteDBWithPostgreSQLDriverExample
32+
```
33+
34+
### 4. Smart Driver Example (`YugabyteDBWithSmartDriverExample`)
35+
Shows how to use the YugabyteDB Smart Driver for distributed deployments.
36+
37+
```bash
38+
./run-example.sh YugabyteDBWithSmartDriverExample
39+
```
40+
41+
## Running Examples
42+
43+
### Quick Start
44+
45+
```bash
46+
# Run the basic example
47+
./run-example.sh
48+
49+
# Run a specific example
50+
./run-example.sh YugabyteDBEmbeddingStoreWithMetadataExample
51+
```
52+
53+
### Alternative: Maven
54+
55+
You can also run examples directly with Maven:
56+
57+
```bash
58+
../mvnw compile exec:java -Dexec.mainClass="YugabyteDBEmbeddingStoreExample"
59+
```
60+
61+
**Note**: The `run-example.sh` script is recommended as it provides cleaner output and avoids Testcontainers classloader issues.
62+
63+
## What Each Example Demonstrates
64+
65+
| Example | Features |
66+
|---------|----------|
67+
| Basic | Vector storage, similarity search, basic operations |
68+
| Metadata | JSONB metadata storage, metadata filtering |
69+
| PostgreSQL Driver | Standard PostgreSQL JDBC driver usage |
70+
| Smart Driver | YugabyteDB-specific driver, topology awareness |
71+
72+
## Container Configuration
73+
74+
All examples use Testcontainers with:
75+
- **Image**: `yugabytedb/yugabyte:2025.1.0.1-b3`
76+
- **Ports**: 5433 (YSQL), 7000, 9000, 15433, 9042
77+
- **Wait Strategy**: Wait for port 5433 with 5-minute timeout
78+
- **Command**: `bin/yugabyted start --background=false`
79+
80+
## Troubleshooting
81+
82+
### Docker Not Running
83+
```
84+
ERROR: Could not find a valid Docker environment
85+
```
86+
**Solution**: Start Docker daemon before running examples.
87+
88+
### Port Already in Use
89+
Testcontainers automatically maps to available ports. Check with:
90+
```bash
91+
docker ps
92+
```
93+
94+
### Build Errors
95+
Clean and rebuild:
96+
```bash
97+
../mvnw clean compile
98+
```
99+
100+
## Learn More
101+
102+
- [YugabyteDB Documentation](https://docs.yugabyte.com/)
103+
- [LangChain4j Documentation](https://docs.langchain4j.dev/)
104+
- [YugabyteDB Embedding Store Integration](https://github.com/langchain4j/langchain4j-community)
105+

‎yugabytedb-example/pom.xml‎

Lines changed: 46 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,46 @@
1+
<?xml version="1.0" encoding="UTF-8"?>
2+
<project xmlns="http://maven.apache.org/POM/4.0.0"
3+
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
4+
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
5+
<modelVersion>4.0.0</modelVersion>
6+
7+
<groupId>dev.langchain4j</groupId>
8+
<artifactId>yugabytedb-example</artifactId>
9+
<version>1.8.0-beta15-SNAPSHOT</version>
10+
11+
<properties>
12+
<maven.compiler.source>17</maven.compiler.source>
13+
<maven.compiler.target>17</maven.compiler.target>
14+
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
15+
</properties>
16+
17+
<dependencies>
18+
19+
<dependency>
20+
<groupId>dev.langchain4j</groupId>
21+
<artifactId>langchain4j-community-yugabytedb</artifactId>
22+
<version>1.8.0-beta15-SNAPSHOT</version>
23+
</dependency>
24+
25+
<dependency>
26+
<groupId>org.testcontainers</groupId>
27+
<artifactId>testcontainers</artifactId>
28+
<version>1.19.7</version>
29+
</dependency>
30+
31+
<dependency>
32+
<groupId>dev.langchain4j</groupId>
33+
<artifactId>langchain4j-embeddings-all-minilm-l6-v2</artifactId>
34+
<version>1.7.1-beta14</version>
35+
</dependency>
36+
37+
<dependency>
38+
<groupId>org.slf4j</groupId>
39+
<artifactId>slf4j-simple</artifactId>
40+
<version>2.0.12</version>
41+
</dependency>
42+
43+
</dependencies>
44+
45+
</project>
46+

‎yugabytedb-example/run-all-examples.sh‎

Lines changed: 33 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,33 @@
1+
#!/bin/bash
2+
3+
# Run all YugabyteDB examples
4+
5+
set -e
6+
7+
EXAMPLES=(
8+
"YugabyteDBEmbeddingStoreExample"
9+
"YugabyteDBEmbeddingStoreWithMetadataExample"
10+
"YugabyteDBWithPostgreSQLDriverExample"
11+
"YugabyteDBWithSmartDriverExample"
12+
)
13+
14+
echo "🚀 Building project once..."
15+
../mvnw clean compile -q
16+
echo ""
17+
18+
for example in "${EXAMPLES[@]}"; do
19+
echo "========================================"
20+
echo "Running: $example"
21+
echo "========================================"
22+
23+
../mvnw exec:exec -Dexec.executable="java" \
24+
-Dexec.args="-cp %classpath $example" \
25+
-q 2>&1 | tail -10
26+
27+
echo ""
28+
echo "----------------------------------------"
29+
echo ""
30+
sleep 2
31+
done
32+
33+
echo "✅ All examples completed!"

‎yugabytedb-example/run-example.sh‎

Lines changed: 23 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,23 @@
1+
#!/bin/bash
2+
3+
# Run YugabyteDB examples as standalone applications
4+
# This avoids exec:java classloader issues with Testcontainers
5+
6+
set -e
7+
8+
EXAMPLE_CLASS=${1:-YugabyteDBEmbeddingStoreExample}
9+
10+
echo "🚀 Building project..."
11+
../mvnw clean compile -q
12+
13+
echo "📦 Running example: $EXAMPLE_CLASS"
14+
echo ""
15+
16+
# Run with proper classpath
17+
../mvnw exec:exec -Dexec.executable="java" \
18+
-Dexec.args="-cp %classpath $EXAMPLE_CLASS" \
19+
-q
20+
21+
echo ""
22+
echo "✅ Example execution completed!"
23+

‎yugabytedb-example/src/main/java/YugabyteDBEmbeddingStoreExample.java‎

Lines changed: 104 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,104 @@
1+
import dev.langchain4j.community.store.embedding.yugabytedb.YugabyteDBEmbeddingStore;
2+
import dev.langchain4j.community.store.embedding.yugabytedb.YugabyteDBEngine;
3+
import dev.langchain4j.data.embedding.Embedding;
4+
import dev.langchain4j.data.segment.TextSegment;
5+
import dev.langchain4j.model.embedding.EmbeddingModel;
6+
import dev.langchain4j.model.embedding.onnx.allminilml6v2.AllMiniLmL6V2EmbeddingModel;
7+
import dev.langchain4j.store.embedding.EmbeddingMatch;
8+
import dev.langchain4j.store.embedding.EmbeddingSearchRequest;
9+
import dev.langchain4j.store.embedding.EmbeddingStore;
10+
import org.testcontainers.containers.GenericContainer;
11+
import org.testcontainers.containers.wait.strategy.Wait;
12+
import org.testcontainers.utility.DockerImageName;
13+
14+
import java.time.Duration;
15+
import java.util.List;
16+
17+
public class YugabyteDBEmbeddingStoreExample {
18+
19+
public static void main(String[] args) {
20+
GenericContainer<?> yugabyteContainer = null;
21+
YugabyteDBEngine engine = null;
22+
23+
try {
24+
DockerImageName dockerImageName = DockerImageName.parse("yugabytedb/yugabyte:2025.1.0.1-b3");
25+
yugabyteContainer = new GenericContainer<>(dockerImageName)
26+
.withExposedPorts(5433, 7000, 9000, 15433, 9042)
27+
.withCommand("bin/yugabyted", "start", "--background=false")
28+
.waitingFor(Wait.forListeningPorts(5433).withStartupTimeout(Duration.ofMinutes(5)));
29+
30+
yugabyteContainer.start();
31+
32+
EmbeddingModel embeddingModel = new AllMiniLmL6V2EmbeddingModel();
33+
34+
// Create YugabyteDB engine with PostgreSQL driver
35+
engine = YugabyteDBEngine.builder()
36+
.host(yugabyteContainer.getHost())
37+
.port(yugabyteContainer.getMappedPort(5433))
38+
.database("yugabyte")
39+
.username("yugabyte")
40+
.password("yugabyte")
41+
.usePostgreSQLDriver(true) // Use PostgreSQL JDBC driver
42+
.build();
43+
44+
EmbeddingStore<TextSegment> embeddingStore = YugabyteDBEmbeddingStore.builder()
45+
.engine(engine)
46+
.tableName("test_embeddings")
47+
.dimension(embeddingModel.dimension())
48+
.createTableIfNotExists(true)
49+
.build();
50+
51+
TextSegment segment1 = TextSegment.from("I like football.");
52+
Embedding embedding1 = embeddingModel.embed(segment1).content();
53+
embeddingStore.add(embedding1, segment1);
54+
55+
TextSegment segment2 = TextSegment.from("The weather is good today.");
56+
Embedding embedding2 = embeddingModel.embed(segment2).content();
57+
embeddingStore.add(embedding2, segment2);
58+
59+
Embedding queryEmbedding = embeddingModel.embed("What is your favourite sport?").content();
60+
61+
EmbeddingSearchRequest embeddingSearchRequest = EmbeddingSearchRequest.builder()
62+
.queryEmbedding(queryEmbedding)
63+
.maxResults(1)
64+
.build();
65+
66+
List<EmbeddingMatch<TextSegment>> relevant = embeddingStore.search(embeddingSearchRequest).matches();
67+
68+
EmbeddingMatch<TextSegment> embeddingMatch = relevant.get(0);
69+
70+
System.out.println(embeddingMatch.score()); // ~0.8144
71+
System.out.println(embeddingMatch.embedded().text()); // I like football.
72+
73+
System.out.println("\n✅ Example completed successfully!");
74+
75+
} catch (Exception e) {
76+
System.err.println("❌ Error running example: " + e.getMessage());
77+
e.printStackTrace();
78+
} finally {
79+
// Give Testcontainers time to cleanup gracefully
80+
try {
81+
Thread.sleep(2000);
82+
} catch (InterruptedException ignored) {
83+
}
84+
85+
// Cleanup resources
86+
System.out.println("🧹 Cleaning up resources...");
87+
if (engine != null) {
88+
try {
89+
engine.close();
90+
} catch (Exception e) {
91+
System.err.println("Error closing engine: " + e.getMessage());
92+
}
93+
}
94+
if (yugabyteContainer != null) {
95+
try {
96+
yugabyteContainer.stop();
97+
} catch (Exception e) {
98+
System.err.println("Error stopping container: " + e.getMessage());
99+
}
100+
}
101+
}
102+
}
103+
}
104+

0 commit comments

Comments
 (0)