Skip to content

Commit 2635e48

Browse files
Copilotbaywettimotheeguerin
authored
Add OpenAPI 3.2 stream emission support with itemSchema (#8888)
## Implementation Plan for Stream Support in OpenAPI 3.2 This PR implements support for streaming responses (including Server-Sent Events) in OpenAPI 3.2 emission and import using the `itemSchema` field. ### Checklist - [x] 1. Add `itemSchema` support to OpenAPI 3.2 MediaType definition in types.ts - [x] 2. Create an SSE module (sse-module.ts) following the xml-module.ts pattern - [x] 3. Integrate SSE module into the OpenAPI emitter for emission support - [x] 4. Add package dependencies for @typespec/streams, @typespec/events, @typespec/sse - [x] 5. Use getStreamMetadata from @typespec/http/experimental to detect streams - [x] 6. Remove debug console.log statements - [x] 7. Tests passing for SSE emission - [x] 8. Handle @events decorator to emit proper SSE event structures - [x] 9. Handle @ContentType decorator on union variants - [x] 10. Handle @terminalevent decorator (preserve with @extension) - [x] 11. Run formatter and linter - [x] 12. Add changelog entry with chronus - [x] 13. Generalize stream support for all content types ### Changes Made **Type Definitions:** - Added `itemSchema` property to `OpenAPIMediaType3_2` for streaming event schema definitions **SSE Module (`sse-module.ts`):** - Created stream detection and emission logic following `xml-module.ts` pattern - Implements stream detection via `@typespec/streams.getStreamOf` - Generates `itemSchema` with `oneOf` variants for `@events` union members - Works with any stream content type (SSE, JSON Lines, etc.) **OpenAPI Emitter Integration:** - Integrated SSE module as optional dependency alongside XML/JSON Schema - Modified `getBodyContentEntry` to detect all stream responses via `getStreamMetadata` - Routes stream responses through emission path for OpenAPI 3.2 with `itemSchema` - Generalized to support any `HttpStream<ContentType, Type>`, not just SSE streams **Testing:** - All stream tests passing - Simplified test structure by removing namespaces and extra decorators - Used ApiTester base for common libraries with stream-specific imports on top **Diagnostics:** - Added "streams-not-supported" diagnostic for OpenAPI 3.0/3.1 - Warning message indicates streams with itemSchema require OpenAPI 3.2.0 ### Stream Support This implementation supports any stream type in OpenAPI 3.2.0: - Server-Sent Events (SSE) with `text/event-stream` - JSON Lines with `application/jsonl` - Any custom `HttpStream<ContentType, Type>` For OpenAPI 3.0/3.1, streams are emitted without `itemSchema` and a warning is generated. - Fixes #8887 <!-- START COPILOT CODING AGENT SUFFIX --> <details> <summary>Original prompt</summary> ---- *This section details on the original issue you should resolve* <issue_title>[Feat] OpenAPI - add support for SSE import and emission</issue_title> <issue_description>### Clear and concise description of the problem Now that we have support for OpenAPI 3.2.0 (#8828 ) both in emissions and in importing descriptions to TypeSpec, it'd be nice to have support for importing SSE (server-sent events). @timotheeguerin and I spent the better part of an hour combing through multiple specifications to understand what needs to happen here, so strap in, this is going to be a long issue. ## Relevant resources - #154 as the initial specification for sse in typespec - https://github.com/OAI/OpenAPI-Specification/discussions/5096 ongoing discussion about terminal events - https://github.com/Azure/azure-sdk-for-python/blob/894d166350f0ccbe2fae467e4093ed0c3b428213/sdk/ai/azure-ai-agents/azure/ai/agents/operations/_patch.py#L564 manual implementation of SSE for Azure AI Foundry: Agents threads run API - https://github.com/Azure/azure-sdk-for-python/blob/main/sdk/ai/azure-ai-agents/tests/assets/send_email_stream_response.txt an example of the payloads sent by this API - [WHATWG specification for SSE](https://html.spec.whatwg.org/multipage/server-sent-events.html#server-sent-events) - [OAI 3.2.0 example on how to describe SSE](https://spec.openapis.org/oas/v3.2.0.html#server-sent-event-streams) - [OpenAI terminal event description](https://platform.openai.com/docs/api-reference/runs/createRun#runs_createrun-stream) - [OpenAI streaming responses documentation](https://platform.openai.com/docs/guides/streaming-responses?api-mode=responses) - [Blog post about using SSE APIs with OpenAI](https://medium.com/better-programming/openai-sse-sever-side-events-streaming-api-733b8ec32897) - [TypeSpec very sparse documentation for SSE](https://typespec.io/docs/libraries/sse/reference/) ## Import ### Scenario 1 : no terminal event Given the following OpenAPI Response object. ```yaml content: description: A request body to add a stream of typed data. required: true content: text/event-stream: itemSchema: type: object properties: event: type: string data: type: string required: [event] # Define event types and specific schemas for the corresponding data oneOf: - properties: event: const: userconnect data: contentMediaType: application/json contentSchema: type: object required: [username] properties: username: type: string - properties: event: const: usermessage data: contentMediaType: application/json contentSchema: type: object required: [text] properties: text: type: string ``` I'd expect aresulting TypeSpec description looking like this. ```tsp import "@typespec/streams"; import "@typespec/sse"; import "@typespec/events"; using SSE; model UserConnect { username: string; } model UserMessage { text: string; } @TypeSpec.Events.events union ChannelEvents { userconnect: UserConnect, usermessage: UserMessage, } op subscribeToChannel(): SSEStream<ChannelEvents>; ``` Note that the terminal event is NOT present, this is ok because it's an invention of some APIs, and is not part of the WHATWG spec. It should remain optional. Here a couple of things are worth noting: - imports for typespec streams/sse/events are added - a using for SSE is added - a `@TypeSpec.Events.events` decorator is added to the union type - the return type of the operation is now `SSEStream<ChannelEvents>` (instead of ChannelEvents) - The union type discriminator values are obtained by conventions based on the event properties in the schema (name is a convention) - the union type member types are defined by convention by the schema of the data property, and the fact the content media type is application/json ### Scenario 2: with terminal events Given the following OpenAPI Response object. ```yaml content: description: A request body to add a stream of typed data. required: true content: text/event-stream: itemSchema: type: object properties: event: type: string data: type: string required: [event] # Define event types and specific schemas for the corresponding data oneOf: - properties: data: contentMediaType: text/plain const: "[done]" "x-ms-sse-terminal-event": true - properties: event: const: userconnect data: contentMediaType: app... </details> - Fixes #8887 <!-- START COPILOT CODING AGENT TIPS --> --- 💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more [Copilot coding agent tips](https://gh.io/copilot-coding-agent-tips) in the docs. --------- Signed-off-by: Vincent Biret <vibiret@microsoft.com> Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: baywet <7905502+baywet@users.noreply.github.com> Co-authored-by: Vincent Biret <vibiret@microsoft.com> Co-authored-by: Timothee Guerin <tiguerin@microsoft.com>
1 parent 0473ebe commit 2635e48

File tree

20 files changed

+1109
-30
lines changed

20 files changed

+1109
-30
lines changed
Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,7 @@
1+
---
2+
changeKind: feature
3+
packages:
4+
- "@typespec/openapi3"
5+
---
6+
7+
adds support for emission and import of SSE for OpenAPI 3.2

cspell.yaml

Lines changed: 4 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -6,13 +6,13 @@ dictionaries:
66
- typescript
77
words:
88
- Adoptium
9-
- Arize
10-
- arizeaiobservabilityeval
119
- agentic
1210
- aiohttp
1311
- alzimmer
1412
- amqp
1513
- AQID
14+
- Arize
15+
- arizeaiobservabilityeval
1616
- arraya
1717
- astimezone
1818
- astro
@@ -230,8 +230,8 @@ words:
230230
- rtype
231231
- rushx
232232
- safeint
233-
- sdkcore
234233
- SCME
234+
- sdkcore
235235
- segmentof
236236
- serde
237237
- setuppy
@@ -293,6 +293,7 @@ words:
293293
- wday
294294
- weidxu
295295
- westus
296+
- WHATWG
296297
- WINDOWSARMVMIMAGE
297298
- WINDOWSVMIMAGE
298299
- xiangyan

packages/openapi3/package.json

Lines changed: 18 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -41,7 +41,8 @@
4141
},
4242
"scripts": {
4343
"clean": "rimraf ./dist ./temp",
44-
"build": "pnpm gen-version && pnpm gen-extern-signature && tsc -p . && pnpm lint-typespec-library",
44+
"build": "pnpm gen-version && pnpm gen-extern-signature && pnpm quickbuild && pnpm lint-typespec-library",
45+
"quickbuild": "tsc -p .",
4546
"watch": "tsc -p . --watch",
4647
"gen-extern-signature": "tspd --enable-experimental gen-extern-signature .",
4748
"lint-typespec-library": "tsp compile . --warn-as-error --import @typespec/library-linter --no-emit",
@@ -73,7 +74,10 @@
7374
"@typespec/http": "workspace:^",
7475
"@typespec/json-schema": "workspace:^",
7576
"@typespec/openapi": "workspace:^",
76-
"@typespec/versioning": "workspace:^"
77+
"@typespec/versioning": "workspace:^",
78+
"@typespec/streams": "workspace:^",
79+
"@typespec/events": "workspace:^",
80+
"@typespec/sse": "workspace:^"
7781
},
7882
"peerDependenciesMeta": {
7983
"@typespec/json-schema": {
@@ -84,6 +88,15 @@
8488
},
8589
"@typespec/versioning": {
8690
"optional": true
91+
},
92+
"@typespec/streams": {
93+
"optional": true
94+
},
95+
"@typespec/events": {
96+
"optional": true
97+
},
98+
"@typespec/sse": {
99+
"optional": true
87100
}
88101
},
89102
"devDependencies": {
@@ -98,6 +111,9 @@
98111
"@typespec/tspd": "workspace:^",
99112
"@typespec/versioning": "workspace:^",
100113
"@typespec/xml": "workspace:^",
114+
"@typespec/streams": "workspace:^",
115+
"@typespec/events": "workspace:^",
116+
"@typespec/sse": "workspace:^",
101117
"@vitest/coverage-v8": "^4.0.4",
102118
"@vitest/ui": "^4.0.4",
103119
"c8": "^10.1.3",

packages/openapi3/src/cli/actions/convert/generators/generate-main.ts

Lines changed: 10 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -7,13 +7,21 @@ import { generateOperation } from "./generate-operation.js";
77
import { generateServiceInformation } from "./generate-service-info.js";
88

99
export function generateMain(program: TypeSpecProgram, context: Context): string {
10+
const sseImports = context.isSSEUsed()
11+
? ` import "@typespec/streams";
12+
import "@typespec/sse";
13+
import "@typespec/events";`
14+
: "";
15+
16+
const sseUsings = context.isSSEUsed() ? "\n using SSE;" : "";
17+
1018
return `
1119
import "@typespec/http";
1220
import "@typespec/openapi";
13-
import "@typespec/openapi3";
21+
import "@typespec/openapi3";${sseImports}
1422
1523
using Http;
16-
using OpenAPI;
24+
using OpenAPI;${sseUsings}
1725
1826
${generateServiceInformation(program.serviceInfo, program.servers, program.tags, context.rootNamespace)}
1927

packages/openapi3/src/cli/actions/convert/generators/generate-model.ts

Lines changed: 163 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -20,6 +20,8 @@ import {
2020
SchemaToExpressionGenerator,
2121
} from "./generate-types.js";
2222

23+
const SSE_TERMINAL_EVENT_EXTENSION = "x-ms-sse-terminal-event";
24+
2325
export function generateDataType(type: TypeSpecDataTypes, context: Context): string {
2426
switch (type.kind) {
2527
case "alias":
@@ -83,6 +85,145 @@ function generateScalar(scalar: TypeSpecScalar, context: Context): string {
8385
return definitions.join("\n");
8486
}
8587

88+
function generateSSEEventVariants(
89+
members: Refable<SupportedOpenAPISchema>[],
90+
union: TypeSpecUnion,
91+
context: Context,
92+
getVariantName: (member: Refable<SupportedOpenAPISchema>) => string,
93+
): string[] {
94+
// Generate SSE event variants: eventName: DataType,
95+
// Sort so terminal events come last
96+
const sortedMembers = [...members].sort((a, b) => {
97+
const aSchema = "$ref" in a ? context.getSchemaByRef(a.$ref) : a;
98+
const bSchema = "$ref" in b ? context.getSchemaByRef(b.$ref) : b;
99+
100+
const aIsTerminal = !!aSchema?.[SSE_TERMINAL_EVENT_EXTENSION];
101+
const bIsTerminal = !!bSchema?.[SSE_TERMINAL_EVENT_EXTENSION];
102+
103+
if (aIsTerminal && !bIsTerminal) return 1; // a comes after b
104+
if (!aIsTerminal && bIsTerminal) return -1; // a comes before b
105+
return 0; // maintain original order for same type
106+
});
107+
108+
return sortedMembers.map((member) => {
109+
try {
110+
const memberSchema = "$ref" in member ? context.getSchemaByRef(member.$ref) : member;
111+
if (!memberSchema || typeof memberSchema !== "object" || !memberSchema.properties) {
112+
// Fallback to regular generation if we can't parse the event structure
113+
return (
114+
getVariantName(member) + context.generateTypeFromRefableSchema(member, union.scope) + ","
115+
);
116+
}
117+
118+
// Use any to access properties since the types are complex
119+
const props = memberSchema.properties;
120+
121+
// Extract event name from event.const
122+
let eventName: string | undefined;
123+
if (props.event) {
124+
const eventProp = props.event;
125+
if ("const" in eventProp && eventProp.const && typeof eventProp.const === "string") {
126+
eventName = eventProp.const;
127+
} else if (
128+
"enum" in eventProp &&
129+
eventProp.enum?.[0] &&
130+
typeof eventProp.enum[0] === "string"
131+
) {
132+
eventName = eventProp.enum[0];
133+
}
134+
}
135+
136+
// Check for terminal events or special cases
137+
if (!eventName) {
138+
// Check if this is a terminal event (no event name, just data)
139+
if ("const" in props.data && props.data.const) {
140+
const terminalValue = props.data.const;
141+
const isTerminal = memberSchema[SSE_TERMINAL_EVENT_EXTENSION];
142+
const contentType = "contentMediaType" in props.data && props.data.contentMediaType;
143+
144+
let decorators = "";
145+
if (contentType) {
146+
decorators += `\n @TypeSpec.Events.contentType("${contentType}")`;
147+
}
148+
if (isTerminal) {
149+
decorators += `\n @TypeSpec.SSE.terminalEvent`;
150+
decorators += `\n @extension("${SSE_TERMINAL_EVENT_EXTENSION}", true)`;
151+
}
152+
153+
return `${decorators}\n "${terminalValue}",`;
154+
}
155+
// Fallback to regular generation
156+
return (
157+
getVariantName(member) + context.generateTypeFromRefableSchema(member, union.scope) + ","
158+
);
159+
}
160+
161+
// Extract data type and content type from data.contentSchema
162+
let dataType = "unknown";
163+
let contentType: string | undefined;
164+
165+
if (props.data) {
166+
const dataProp = props.data;
167+
168+
// Get content type if specified
169+
if ("contentMediaType" in dataProp && dataProp.contentMediaType) {
170+
contentType = dataProp.contentMediaType;
171+
}
172+
173+
// Check for contentSchema (OpenAPI extension)
174+
if ("contentSchema" in dataProp && dataProp.contentSchema) {
175+
const contentSchema = dataProp.contentSchema;
176+
// Special handling for byte data which should map to Base64/bytes
177+
if (
178+
contentSchema &&
179+
typeof contentSchema === "object" &&
180+
"type" in contentSchema &&
181+
contentSchema.type === "object" &&
182+
contentSchema.properties?.data
183+
) {
184+
const dataProperty = contentSchema.properties.data;
185+
if (
186+
"type" in dataProperty &&
187+
dataProperty.type === "string" &&
188+
"format" in dataProperty &&
189+
dataProperty.format === "byte"
190+
) {
191+
dataType = "Base64";
192+
} else {
193+
dataType = context.generateTypeFromRefableSchema(dataProp.contentSchema, union.scope);
194+
}
195+
} else {
196+
dataType = context.generateTypeFromRefableSchema(dataProp.contentSchema, union.scope);
197+
}
198+
} else if ("type" in dataProp && dataProp.type && typeof dataProp.type === "string") {
199+
// Simple type like string
200+
dataType = dataProp.type;
201+
}
202+
}
203+
204+
// Build decorators for this event variant
205+
let decorators = "";
206+
if (contentType && contentType !== "application/json") {
207+
decorators += `\n @TypeSpec.Events.contentType("${contentType}")`;
208+
}
209+
210+
// Check if this is a terminal event
211+
const isTerminal = memberSchema[SSE_TERMINAL_EVENT_EXTENSION];
212+
if (isTerminal) {
213+
decorators += `\n @TypeSpec.SSE.terminalEvent`;
214+
decorators += `\n @extension("${SSE_TERMINAL_EVENT_EXTENSION}", true)`;
215+
}
216+
217+
return `${decorators}\n ${eventName}: ${dataType},`;
218+
} catch (error) {
219+
// If any error occurs, fall back to regular generation
220+
return (
221+
getVariantName(member) + context.generateTypeFromRefableSchema(member, union.scope) + ","
222+
);
223+
}
224+
});
225+
}
226+
86227
function generateUnion(union: TypeSpecUnion, context: Context): string {
87228
const definitions: string[] = [];
88229

@@ -103,24 +244,38 @@ function generateUnion(union: TypeSpecUnion, context: Context): string {
103244

104245
const memberSchema = "$ref" in member ? context.getSchemaByRef(member.$ref)! : member;
105246

247+
const propertySchema =
248+
memberSchema.properties && memberSchema.properties[union.schema.discriminator.propertyName];
106249
const value =
107250
(union.schema.discriminator?.mapping && "$ref" in member
108251
? Object.entries(union.schema.discriminator.mapping).find((x) => x[1] === member.$ref)?.[0]
109252
: undefined) ??
110-
(memberSchema.properties?.[union.schema.discriminator.propertyName] as any)?.enum?.[0];
253+
(propertySchema && "enum" in propertySchema && propertySchema.enum?.[0]);
111254
// checking whether the value is using an invalid character as an identifier
112-
const valueIdentifier = value ? printIdentifier(value, "disallow-reserved") : "";
255+
const valueIdentifier = value ? printIdentifier(`${value}`, "disallow-reserved") : "";
113256
return value ? `${value === valueIdentifier ? value : valueIdentifier}: ` : "";
114257
};
115258
if (schema.enum) {
116259
definitions.push(...schema.enum.map((e) => `${JSON.stringify(e)},`));
117260
} else if (schema.oneOf) {
118-
definitions.push(
119-
...schema.oneOf.map(
120-
(member) =>
121-
getVariantName(member) + context.generateTypeFromRefableSchema(member, union.scope) + ",",
122-
),
261+
// Check if this is an SSE event union
262+
const isSSEEventUnion = union.decorators.some(
263+
(d) => d.name === "TypeSpec.Events.events" || d.name === "events",
123264
);
265+
266+
if (isSSEEventUnion) {
267+
definitions.push(...generateSSEEventVariants(schema.oneOf, union, context, getVariantName));
268+
} else {
269+
// Regular union generation
270+
definitions.push(
271+
...schema.oneOf.map(
272+
(member) =>
273+
getVariantName(member) +
274+
context.generateTypeFromRefableSchema(member, union.scope) +
275+
",",
276+
),
277+
);
278+
}
124279
} else if (schema.anyOf) {
125280
definitions.push(
126281
...schema.anyOf.map(
@@ -140,7 +295,7 @@ function generateUnion(union: TypeSpecUnion, context: Context): string {
140295
definitions.push("null,");
141296
} else {
142297
// Create a schema with a single type to reuse existing logic
143-
const singleTypeSchema = { ...schema, type: t as any, nullable: undefined };
298+
const singleTypeSchema = { ...schema, type: t, nullable: undefined };
144299
const type = context.generateTypeFromRefableSchema(singleTypeSchema, union.scope);
145300
definitions.push(`${type},`);
146301
}

packages/openapi3/src/cli/actions/convert/generators/generate-response-expressions.ts

Lines changed: 23 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -220,6 +220,29 @@ function generateResponseExpressions({
220220
}
221221

222222
return contents.map(([mediaType, content]) => {
223+
// Special handling for Server-Sent Events
224+
if (mediaType === "text/event-stream") {
225+
context.markSSEUsage();
226+
227+
// Check for itemSchema (OpenAPI 3.2 extension)
228+
if ("itemSchema" in content && content.itemSchema) {
229+
const itemSchema = content.itemSchema;
230+
if (itemSchema && typeof itemSchema === "object" && "$ref" in itemSchema) {
231+
const eventUnionType = context.generateTypeFromRefableSchema(itemSchema, operationScope);
232+
return `SSEStream<${eventUnionType}>`;
233+
}
234+
} else if (content.schema && "$ref" in content.schema) {
235+
// Fallback: use schema directly if no itemSchema
236+
const eventUnionType = context.generateTypeFromRefableSchema(
237+
content.schema,
238+
operationScope,
239+
);
240+
return `SSEStream<${eventUnionType}>`;
241+
}
242+
243+
// If no proper schema reference, fall through to regular handling
244+
}
245+
223246
// Attempt to emit just the Body or an intersection of Body & MappedResponse
224247
// if there aren't any custom headers.
225248
const bodySchema = content.schema;

packages/openapi3/src/cli/actions/convert/transforms/transform-component-schemas.ts

Lines changed: 11 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -141,10 +141,20 @@ export function transformComponentSchemas(context: Context, models: TypeSpecData
141141
// Extract description and decorators from meaningful union members
142142
const unionMetadata = extractUnionMetadata(schema);
143143

144+
let decorators = [...getDecoratorsForSchema(schema), ...unionMetadata.decorators];
145+
146+
// Check if this is an SSE event schema - if so, replace @oneOf with @events
147+
const schemaRef = `#/components/schemas/${name}`;
148+
if (context.isSSEEventSchema(schemaRef)) {
149+
// Remove @oneOf decorator if present and add @events
150+
decorators = decorators.filter((d) => d.name !== "oneOf");
151+
decorators.push({ name: "events", args: [] });
152+
}
153+
144154
const union: TypeSpecUnion = {
145155
kind: "union",
146156
...getScopeAndName(name),
147-
decorators: [...getDecoratorsForSchema(schema), ...unionMetadata.decorators],
157+
decorators,
148158
doc: schema.description ?? unionMetadata.description,
149159
schema,
150160
};

0 commit comments

Comments
 (0)