Skip to content

Conversation

@goodwillpunning
Copy link
Contributor

Changes

What does this PR do?

This PR adds a new profiler for mssql.

Relevant implementation details

This profiler closely follows the implementation of the Azure Synapse profiler. However, this implementation provides a last_execution_time param for all mssql server queries, allowing for a future scheduler component to hook in an repeat queries.

Caveats/things to watch out for when reviewing:

This profiler will not support on-prem mssql servers.

Linked issues

N/A

Functionality

  • added relevant user documentation
  • added new CLI command
  • modified existing command: databricks labs lakebridge ...
  • added new profiler component
  • ... +add your own

Tests

  • manually tested
  • added unit tests
  • added integration tests

@goodwillpunning goodwillpunning requested a review from a team as a code owner November 17, 2025 16:20
@goodwillpunning goodwillpunning added the feat/profiler Issues related to profilers label Nov 17, 2025
@codecov
Copy link

codecov bot commented Nov 17, 2025

Codecov Report

❌ Patch coverage is 0% with 171 lines in your changes missing coverage. Please review.
✅ Project coverage is 13.47%. Comparing base (65ce810) to head (42d0028).
⚠️ Report is 2 commits behind head on main.

Files with missing lines Patch % Lines
...bridge/resources/assessments/mssql/info_extract.py 0.00% 71 Missing ⚠️
...ge/resources/assessments/mssql/activity_extract.py 0.00% 46 Missing ⚠️
...idge/resources/assessments/mssql/common/queries.py 0.00% 44 Missing ⚠️
...ge/resources/assessments/mssql/common/connector.py 0.00% 5 Missing ⚠️
...ge/resources/assessments/mssql/common/functions.py 0.00% 4 Missing ⚠️
...tabricks/labs/lakebridge/assessments/_constants.py 0.00% 1 Missing ⚠️
Additional details and impacted files
@@             Coverage Diff             @@
##             main    #2151       +/-   ##
===========================================
- Coverage   65.23%   13.47%   -51.77%     
===========================================
  Files         100      105        +5     
  Lines        8504     8673      +169     
  Branches      875      875               
===========================================
- Hits         5548     1169     -4379     
- Misses       2769     7445     +4676     
+ Partials      187       59      -128     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@github-actions
Copy link

github-actions bot commented Nov 17, 2025

✅ 51/51 passed, 11 flaky, 3m35s total

Flaky tests:

  • 🤪 test_validate_successful_schema_check (158ms)
  • 🤪 test_validate_mixed_checks (172ms)
  • 🤪 test_validate_invalid_schema_check (1ms)
  • 🤪 test_validate_invalid_schema_path (1ms)
  • 🤪 test_validate_non_empty_tables (7ms)
  • 🤪 test_transpiles_informatica_to_sparksql_non_interactive[False] (21.68s)
  • 🤪 test_transpiles_informatica_to_sparksql (22.807s)
  • 🤪 test_transpile_teradata_sql (24.177s)
  • 🤪 test_transpiles_informatica_to_sparksql_non_interactive[True] (4.256s)
  • 🤪 test_transpile_teradata_sql_non_interactive[True] (6.066s)
  • 🤪 test_transpile_teradata_sql_non_interactive[False] (7.674s)

Running from acceptance #3061

from databricks.labs.lakebridge.connections.database_manager import DatabaseManager


def get_sqlserver_reader(
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Database manager should already have the MSSQLConnector we should be using that.

steps:
- name: activity_extract
type: python
extract_source: src/databricks/labs/lakebridge/resources/assessments/mssql/activity_extract.py
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can't we just run sql queries directly? do we need python wrapper for SQL Server profilers?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

feat/profiler Issues related to profilers

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants