Skip to content

add swift connector example #438

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
fivetran-surabhisingh wants to merge 1 commit into
base: main
Choose a base branch
from
Open

add swift connector example #438

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
38 changes: 38 additions & 0 deletions connectors/swift/connector.py
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
"""
Swift connector for Fivetran Connector SDK.
Fetches payment data via SWIFT API (mock endpoint for base setup).
"""
Comment on lines +1 to +4
Copy link

Copilot AI Oct 31, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Missing required structure: The connector is missing the mandatory main block for debugging. Add the following at the end: if __name__ == '__main__': block with connector.debug(configuration=configuration) as shown in the template.

Copilot generated this review using guidance from repository custom instructions.

import requests
Copy link

Copilot AI Oct 31, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Missing import comment. Add a comment explaining the purpose of this import, e.g., # For making HTTP API requests (provided by SDK runtime).

Suggested change
import requests
import requests # For making HTTP API requests (provided by SDK runtime)

Copilot uses AI. Check for mistakes.
from fivetran_connector_sdk import connector, config, state, records, log, schema
Copy link

Copilot AI Oct 31, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

BLOCKER: Incorrect SDK imports. The Fivetran Connector SDK uses a different import pattern. Use these exact imports instead: from fivetran_connector_sdk import Connector, from fivetran_connector_sdk import Logging as log, and from fivetran_connector_sdk import Operations as op. The decorator-based pattern with @connector, config, state, records, and schema imports does not exist in the SDK.

Copilot generated this review using guidance from repository custom instructions.
Copy link

Copilot AI Oct 31, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Import of 'log' is not used.

Suggested change
from fivetran_connector_sdk import connector, config, state, records, log, schema
from fivetran_connector_sdk import connector, config, state, records, schema

Copilot uses AI. Check for mistakes.

CONFIG = config.Config(
base_url=config.StringField(description="SWIFT API base URL"),
api_key=config.SecretField(description="SWIFT API key")
)
Comment on lines +9 to +12
Copy link

Copilot AI Oct 31, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

BLOCKER: This configuration pattern is not supported by the Fivetran Connector SDK. Configuration should be defined in a configuration.json file which the SDK automatically validates. Remove this CONFIG object and create a configuration.json file instead with the required fields.

Copilot generated this review using guidance from repository custom instructions.

SCHEMA = schema.Schema(
name="swift_transactions",
columns={
"transaction_id": schema.StringColumn(),
"amount": schema.StringColumn(),
"currency": schema.StringColumn(),
"timestamp": schema.StringColumn(),
}
)
Comment on lines +14 to +22
Copy link

Copilot AI Oct 31, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

BLOCKER: This schema definition pattern is not supported. The SDK requires a schema() function that returns a list of table dictionaries. Replace this with a proper schema(configuration: dict) function following the template format.

Copilot generated this review using guidance from repository custom instructions.

@connector(
name="SwiftConnector",
version="0.1.0",
config=CONFIG,
schema=SCHEMA,
)
def run_connector(ctx: state.Context):
Comment on lines +24 to +30
Copy link

Copilot AI Oct 31, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

BLOCKER: The decorator-based @connector pattern is not part of the Fivetran Connector SDK. The SDK requires an update(configuration: dict, state: dict) function and a Connector object initialized as connector = Connector(update=update, schema=schema). Remove the decorator and implement the required functions.

Copilot generated this review using guidance from repository custom instructions.
headers = {"Authorization": f"Bearer {ctx.config.api_key}"}
response = requests.get(f"{ctx.config.base_url}/transactions", headers=headers)
response.raise_for_status()
Comment on lines +32 to +33
Copy link

Copilot AI Oct 31, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

BLOCKER: No retry logic implemented for the API request. Network calls must include retry logic with exponential backoff to handle transient failures (timeouts, connection errors, 5xx responses). Implement a retry mechanism with a maximum of 3-5 attempts.

Copilot generated this review using guidance from repository custom instructions.
Comment on lines +32 to +33
Copy link

Copilot AI Oct 31, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Missing specific exception handling. Generic exceptions from raise_for_status() should be caught and handled specifically (e.g., distinguish between 4xx client errors that shouldn't be retried vs 5xx server errors that should). Implement proper error handling with specific exception types.

Copilot generated this review using guidance from repository custom instructions.

for tx in response.json().get("data", []):
Comment on lines +32 to +35
Copy link

Copilot AI Oct 31, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

BLOCKER: Potential memory issue - the code loads all transactions into memory at once with response.json().get('data', []). For large datasets, this can cause memory overflow. Implement pagination or streaming to process data in chunks.

Copilot generated this review using guidance from repository custom instructions.
records.write("swift_transactions", tx)
Copy link

Copilot AI Oct 31, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

BLOCKER: Incorrect operation method. The SDK uses op.upsert(table=table_name, data=record) not records.write(). Additionally, this operation requires a comment explaining the upsert operation as per SDK requirements.

Copilot generated this review using guidance from repository custom instructions.

return ctx.update_state({"last_sync": "now"})
Copy link

Copilot AI Oct 31, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

BLOCKER: Incorrect state management pattern. The SDK uses op.checkpoint(state) to save state, and the update function should not return state. Replace with proper checkpointing using op.checkpoint() and include the required checkpoint comment.

Copilot generated this review using guidance from repository custom instructions.
Copy link

Copilot AI Oct 31, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Using string literal 'now' as a timestamp value is incorrect. Use a proper ISO 8601 timestamp format like datetime.datetime.now().isoformat() or a Unix timestamp.

Copilot uses AI. Check for mistakes.
Loading