You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: judges/multimodal-evaluation.mdx
+30-6Lines changed: 30 additions & 6 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -14,7 +14,7 @@ LLM judges can evaluate spans that contain images alongside text. This is useful
14
14
15
15
The LLM sees both the span's text data (input/output) and any attached images, giving it full context for evaluation.
16
16
17
-
Images can be provided as **base64-encoded strings** or as **presigned S3 URLs**. In both cases, ZeroEval copies the image into its own storage during ingestion, so any presigned URL only needs to remain valid long enough for the ingest request to complete.
17
+
Images can be provided as **base64-encoded strings**, **presigned S3 URLs**, or **CDN image URLs**. In all cases, ZeroEval copies the image into its own storage during ingestion, so the source URL only needs to remain valid long enough for the ingest request to complete.
18
18
19
19
## Attaching images to spans
20
20
@@ -78,12 +78,14 @@ with ze.span(name="button_hover_test") as span:
78
78
)
79
79
```
80
80
81
-
### Option 2: Presigned S3 URLs
81
+
### Option 2: Image URLs (S3 presigned or CDN)
82
82
83
-
If your images are already stored in S3, you can pass a presigned HTTPS URL instead of base64 data. ZeroEval will download the image, validate it, and copy it into its own storage.
83
+
If your images are already hosted externally, you can pass an HTTPS URL instead of base64 data. ZeroEval will download the image, validate it, and copy it into its own storage.
84
84
85
85
Attach URLs via `attributes.attachments` using a `url` key instead of `base64`:
86
86
87
+
**Presigned S3 URL**
88
+
87
89
```python
88
90
import boto3
89
91
import zeroeval as ze
@@ -110,8 +112,30 @@ with ze.span(name="chart_generation") as span:
The presigned URL only needs to stay valid long enough for ZeroEval to download the image during ingestion (typically a few seconds). After that, ZeroEval serves the image from its own storage.
138
+
The URL only needs to stay valid long enough for ZeroEval to download the image during ingestion (typically a few seconds). After that, ZeroEval serves the image from its own storage. CDN URLs must be from a trusted domain configured in the backend.
115
139
</Note>
116
140
117
141
### Option 3: Structured output_data
@@ -155,7 +179,7 @@ with ze.span(
155
179
]
156
180
```
157
181
158
-
You can also use presigned S3 URLs in `output_data` by replacing the `base64` key with `url`:
182
+
You can also use URLs (presigned S3 or CDN) in `output_data` by replacing the `base64` key with `url`:
159
183
160
184
```python
161
185
span.output_data = [
@@ -251,7 +275,7 @@ Then configure your judge to only evaluate spans matching that tag. This prevent
251
275
- WebP
252
276
- GIF
253
277
254
-
Images can be provided as base64-encoded strings or presigned S3 URLs (`https://*.amazonaws.com`). In both cases, images are validated by magic bytes during ingestion. The maximum size is 10MB per image, with up to 5 images per span.
278
+
Images can be provided as base64-encoded strings, presigned S3 URLs, or CDN image URLs from trusted domains. In all cases, images are validated by magic bytes during ingestion. The maximum size is 10MB per image, with up to 5 images per span.
Copy file name to clipboardExpand all lines: tracing/sdks/python/reference.mdx
+27-3Lines changed: 27 additions & 3 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -575,16 +575,21 @@ with ze.span(name="chart_generation") as span:
575
575
)
576
576
```
577
577
578
-
##### Attaching images via presigned S3 URL
578
+
##### Attaching images via URL (S3 presigned or CDN)
579
579
580
-
If your images are already stored in S3, you can pass a presigned HTTPSURL instead of base64 data. ZeroEval will download, validate, and copy the image into its own storage during ingestion.
580
+
If your images are already hosted externally, you can pass an HTTPSURL instead of base64 data. ZeroEval will download, validate, and copy the image into its own storage during ingestion.
Attach URLs directly via `attributes.attachments` using the `url` key:
583
587
584
588
```python
585
589
import boto3
586
590
import zeroeval as ze
587
591
592
+
# Option A: Presigned S3 URL
588
593
s3 = boto3.client("s3")
589
594
presigned_url = s3.generate_presigned_url(
590
595
"get_object",
@@ -607,7 +612,26 @@ with ze.span(name="chart_generation") as span:
607
612
)
608
613
```
609
614
610
-
The presigned URL only needs to remain valid long enough for ZeroEval to download the image during ingestion (typically a few seconds). Supported hosts: `*.amazonaws.com`with valid presigned authentication parameters.
Images attached to spans can be evaluated by LLM judges configured for multimodal evaluation. See the [Multimodal Evaluation](/judges/multimodal-evaluation) guide for setup instructions.
0 commit comments