-
Notifications
You must be signed in to change notification settings - Fork 4.1k
Early Access Preview Docs for API Filtering #1731
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
Show all changes
46 commits
Select commit
Hold shift + click to select a range
0638aba
First pass Early Access doc
e9f7191
Update api/early-access_API-Filtering.md
iambmelt ab0643d
Update api/early-access_API-Filtering.md
iambmelt cdd5b83
Update api/early-access_API-Filtering.md
iambmelt b0d65e6
Update api/early-access_API-Filtering.md
iambmelt 683ef3c
Update api/early-access_API-Filtering.md
iambmelt ff3b077
Update api/early-access_API-Filtering.md
iambmelt 980f669
Update api/early-access_API-Filtering.md
iambmelt fb09930
Update api/early-access_API-Filtering.md
iambmelt e9b669f
Update api/early-access_API-Filtering.md
iambmelt d302e1b
Update api/early-access_API-Filtering.md
iambmelt c220aee
Update api/early-access_API-Filtering.md
iambmelt 4c28a95
Update api/early-access_API-Filtering.md
iambmelt 38a91a2
Update api/early-access_API-Filtering.md
iambmelt 0b44b5a
Update api/early-access_API-Filtering.md
iambmelt 521bc68
Merge branch 'main' into iambmelt/ea-api-filter-docs
iambmelt eeded62
Update api/early-access_API-Filtering.md
iambmelt 7763784
Update api/early-access_API-Filtering.md
iambmelt efffffa
Update api/early-access_API-Filtering.md
iambmelt c3211c8
Update api/early-access_API-Filtering.md
iambmelt a481088
Update api/early-access_API-Filtering.md
iambmelt e1c88df
Update api/early-access_API-Filtering.md
iambmelt 38cb23b
Update api/early-access_API-Filtering.md
iambmelt 3efb287
Update api/early-access_API-Filtering.md
iambmelt 53c24f7
Update api/early-access_API-Filtering.md
iambmelt 635b733
Update api/early-access_API-Filtering.md
iambmelt 8719040
Update api/early-access_API-Filtering.md
iambmelt 5b5cab5
Update api/early-access_API-Filtering.md
iambmelt 1affd10
Update api/early-access_API-Filtering.md
iambmelt 5f605ec
Update api/early-access_API-Filtering.md
iambmelt 17db8d0
Newline formatting
ed36226
Add date comparison example
17af48b
Update api/early-access_API-Filtering.md
iambmelt 5ef7219
Update api/early-access_API-Filtering.md
iambmelt 4cb4faa
Update api/early-access_API-Filtering.md
iambmelt 37445af
Update api/early-access_API-Filtering.md
iambmelt c675f8f
Update api/early-access_API-Filtering.md
iambmelt 89c4114
fmt
iambmelt 190c556
Clean up linebreak tags
fca3678
fmt
iambmelt 5de302a
Rebrand -> "API Filtering"
7f7cdb9
Merge branch 'iambmelt/ea-api-filter-docs' of https://github.com/ngro…
8ac7beb
Remove mentions of 'initial release', as folks may overindex on this
f7b1a7a
Conciseness, phrasing
4a39368
clarity
81dec5c
Updating file name and navigation
S3Prototype File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,190 @@ | ||
| --- | ||
| title: API Filtering (Preview) | ||
| sidebarTitle: API Filtering | ||
| description: Learn how to use API Filtering to make operational tooling faster and more precise. | ||
| tag: Preview | ||
| --- | ||
|
|
||
| <Info> | ||
| This feature is under active development. Behavior, supported fields, and limits may change before General Availability (GA). This guide is provided as forward-looking context for evaluation and feedback. | ||
| </Info> | ||
|
|
||
| When using ngrok's API, you can add the `filter` query parameter to `GET` requests to return only those results which match a provided criteria. This makes automated management of resources easier while eliminating the need to download large collections and filter client-side. | ||
|
|
||
| To use API Filtering, you pass a subset of CEL expressions to the `filter` query parameter, as demonstrated in the following example. | ||
|
|
||
| This example request fetches a list of all your Cloud and Agent endpoints. | ||
|
|
||
| ```http | ||
| GET /endpoints?filter='obj.type == "cloud" || obj.type == "agent"' | ||
| ``` | ||
|
|
||
| ## Request shape | ||
|
|
||
| ```http | ||
| GET /{resource}?filter='{CEL_EXPRESSION}' | ||
| ``` | ||
|
|
||
| ## Supported CEL (subset) | ||
|
|
||
| These core operators and helpers are supported: | ||
|
|
||
| - Logical operators: `!`, `&&`, `||` | ||
| - Comparative operators: `<`, `<=`, `==`, `!=`, `>=`, `>` | ||
| - Parentheses for grouping | ||
| - List membership using the `in` keyword | ||
| - String substring checks: `startsWith()`, `contains()`, `endsWith()` | ||
| - Length / emptiness checks: `size()`, `== ""`, `== null` | ||
| - Date parsing: `timestamp(RFC-3339)` | ||
|
|
||
| ### Instance Inspection (vs. List Comprehension) | ||
|
|
||
| Expressions are evaluated against a single **resource instance** exposed as `obj`. Compare fields **on the instance** rather than attempting list-wise checks on fields. | ||
|
|
||
| ✅ Valid | ||
|
|
||
| ```http | ||
| GET /endpoints?filter='obj.type == "cloud" || obj.type == "agent"' | ||
| GET /endpoints?filter='obj.type in ["agent", "cloud"]' | ||
| ``` | ||
|
|
||
| ❌ Not valid | ||
|
|
||
| ```http | ||
| GET /endpoints?filter='["agent","cloud"] in obj.types' | ||
| ``` | ||
|
|
||
| ## Dates and time helpers | ||
|
|
||
| - **Treat timestamps as numerics** by using `<`, `<=`, `==`, `>=`, `>` directly on `timestamp()` fields, e.g.: | ||
| ```http | ||
| GET /vaults?filter=’obj.created_at < timestamp(“2025-10-31T09:23:45-07:00”)’ | ||
| ``` | ||
| - **Convenience helpers:** `now()` and `daysAgo(n)` support concise relative filters, e.g.: | ||
| ```http | ||
| GET /endpoints?filter='obj.created_at >= daysAgo(7)' | ||
| ``` | ||
|
|
||
iambmelt marked this conversation as resolved.
Show resolved
Hide resolved
|
||
| ## Query restrictions and limitations | ||
|
|
||
| ### Unsupported CEL features | ||
|
|
||
| To keep filter evaluation small and predictable, the following CEL features are not supported. | ||
|
|
||
iambmelt marked this conversation as resolved.
Show resolved
Hide resolved
|
||
| - **No index access** (e.g., `a[0]`) | ||
| - **No arithmetic** (e.g., `a + b`) | ||
| - **No ternary** (e.g., `cond ? x : y`) | ||
| - **No type checks** (e.g., `type(a) == string`) | ||
| - **No regexes** | ||
| - **No fuzzy matching** | ||
|
|
||
| These exclusions intentionally keep evaluation small and predictable. | ||
|
|
||
| ### High-entropy fields and substring checks | ||
|
|
||
| **High entropy** fields are fields with values that are effectively random, usually because they're generated. The `id` field on a response object, such as `obj.id`, is a common example. | ||
|
|
||
| [Substring functions](https://github.com/google/cel-spec/blob/master/doc/langdef.md#string-functions), such as `startsWith()`, `contains()`, `endsWith()`, are **disallowed** on high entropy fields. Check for equality on these fields instead. For example, `obj.id == "ep_123"`. | ||
|
|
||
| ### Query complexity (budgeting/limits) | ||
|
|
||
| Very large expressions can stress the query engine. The service may enforce **limits on the number of conditions per query** or similar throttles in the future. | ||
iambmelt marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
|
||
| ## Filterable resources and fields | ||
|
|
||
| The initial release prioritizes the resource types and fields below. CEL filtering is not supported on deprecated endpoints. Field coverage is evolving and may change before GA. | ||
|
|
||
| | Resource Type | Filterable Fields | | ||
| | ------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | ||
| | **Endpoints** | - `id`<br />- `created_at`<br />- `description`<br />- `metadata`<br />- `principal.id`<br />- `type`<br />- `binding`<br />- `url`<br />- `pooling_enabled`<br />- `scheme`<br />- `region` | | ||
| | **Reserved Addresses** | - `id`<br />- `created_at`<br />- `description`<br />- `metadata`<br />- `addr`<br />- `region` | | ||
| | **Reserved Domains** | - `id`<br />- `created_at`<br />- `description`<br />- `metadata`<br />- `domain`<br />- `region`<br />- `cname_target`<br />- `certificate.id`<br />- `acme_challenge_cname_target` | | ||
| | **TLS Certificates** | - `id`<br />- `created_at`<br />- `description`<br />- `metadata`<br />- `subject_common_name`<br />- `not_after`<br />- `not_before`<br />- `serial_number` | | ||
| | **Certificate Authorities** | - `id`<br />- `created_at`<br />- `description`<br />- `metadata`<br />- `subject_common_name`<br />- `not_before`<br />- `not_after` | | ||
| | **IP Policies** | - `id`<br />- `created_at`<br />- `description`<br />- `metadata` | | ||
| | **IP Policy Rules** | - `id`<br />- `created_at`<br />- `description`<br />- `metadata`<br />- `ip_policy`<br />- `cidr`<br />- `action` | | ||
| | **Agent Ingress** | - `id`<br />- `created_at`<br />- `description`<br />- `metadata`<br />- `domain` | | ||
| | **Tunnel Sessions** | - `id`<br />- `metadata`<br />- `agent_version`<br />- `ip`<br />- `os`<br />- `region`<br />- `started_at`<br />- `credential` | | ||
| | **Event Destinations** | - `id`<br />- `created_at`<br />- `description`<br />- `metadata` | | ||
| | **Event Subscriptions** | - `id`<br />- `created_at`<br />- `description`<br />- `metadata` | | ||
| | **IP Restrictions** | - `id`<br />- `created_at`<br />- `description`<br />- `metadata` | | ||
| | **API Keys** | - `id`<br />- `created_at`<br />- `description`<br />- `metadata`<br />- `owner_id` | | ||
| | **SSH Credentials** | - `id`<br />- `created_at`<br />- `description`<br />- `metadata`<br />- `owner_id`<br />- `acl` | | ||
| | **Credentials** | - `id`<br />- `created_at`<br />- `description`<br />- `metadata`<br />- `owner_id`<br />- `acl` | | ||
| | **Service Users** | - `id`<br />- `created_at` | | ||
| | **SSH Certificate Authorities** | - `id`<br />- `created_at`<br />- `description`<br />- `metadata` | | ||
| | **Vaults** | - `id`<br />- `created_at`<br />- `description`<br />- `metadata`<br />- `name` | | ||
| | **Secrets** | - `id`<br />- `created_at`<br />- `description`<br />- `metadata`<br />- `name` | | ||
|
|
||
| ## Usage examples | ||
|
|
||
| ### **Filter endpoints by type and creation time** | ||
|
|
||
| ```http | ||
| GET /endpoints?filter='obj.type == "cloud" && obj.created_at < timestamp("2025-10-31T09:23:45-07:00")' | ||
| # or using helpers | ||
| GET /endpoints?filter='obj.type == "cloud" && obj.created_at >= daysAgo(6)' | ||
| ``` | ||
|
|
||
iambmelt marked this conversation as resolved.
Show resolved
Hide resolved
|
||
| Reference: | ||
|
|
||
| - [`LIST /endpoints`](/api-reference/endpoints/list) | ||
|
|
||
| ### **Reserved domains by prefix** | ||
|
|
||
| ```http | ||
| GET /reserved_domains?filter='obj.domain.startsWith("myapi.ngrok")' | ||
| ``` | ||
|
|
||
iambmelt marked this conversation as resolved.
Show resolved
Hide resolved
|
||
| Reference: | ||
|
|
||
| - [`LIST /reserved_domains`](/api-reference/reserveddomains/list) | ||
|
|
||
| ### **IP policy rules by CIDR and action** | ||
|
|
||
| ```http | ||
| GET /ip_policy_rules?filter='obj.cidr.contains("1.1.0.0/16") && obj.action == "deny"' | ||
| ``` | ||
|
|
||
iambmelt marked this conversation as resolved.
Show resolved
Hide resolved
|
||
| Reference: | ||
|
|
||
| - [`LIST /ip_policy_rules`](/api-reference/ippolicyrules/list) | ||
|
|
||
| ### **Credentials by owner with optional empty ACL** | ||
|
|
||
| ```http | ||
| GET /credentials?filter='obj.owner_id == "usr_2tEpN0yrxDI4j8jVnhVRoTNN2Tx" && (obj.acl == null || obj.acl == "")' | ||
| ``` | ||
|
|
||
iambmelt marked this conversation as resolved.
Show resolved
Hide resolved
|
||
| Reference: | ||
|
|
||
| - [`LIST /credentials`](/api-reference/credentials/list) | ||
|
|
||
| ### **Complex nesting** | ||
|
|
||
| ```http | ||
| GET /agent_ingresses?filter='obj.domain in ["foo.com","bar.com","baz.com"] || (obj.created_at < timestamp("2025-05-10Z") && obj.description.contains("cowbell"))' | ||
| ``` | ||
|
|
||
iambmelt marked this conversation as resolved.
Show resolved
Hide resolved
|
||
| Reference: | ||
|
|
||
| - [`LIST /agent_ingresses`](/api-reference/agentingresses/list) | ||
|
|
||
| ## Error handling | ||
|
|
||
| Invalid filters return **HTTP 400** with a structured error body (`category`, `status_code`, `message`, `details`). Example: | ||
|
|
||
| ```http | ||
| HTTP/1.1 400 Bad Request | ||
| Content-Type: application/json; charset=utf-8 | ||
| Cache-Control: no-store | ||
|
|
||
| { | ||
| "error_code": "invalid_cel_expression", | ||
| "status_code": 400, | ||
| "msg": "Invalid CEL query: unsupported field: endpoint.idk (must be endpoint.url, endpoint.id, endpoint.type, or endpoint.bindings).", | ||
| "details": { | ||
| "operation_id": "op_k23j45n134jkasdfk34jkjnlkjuhasdf" | ||
| } | ||
| } | ||
| ``` | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.