Refactor Customers Model: Convert CTEs into Intermediate Models

[even-wei]

Overview:
Refactored the customers model by turning two CTEs into intermediate models, enhancing readability and maintainability.

Changes:

• Created two new intermediate models from existing CTEs.
• Updated customers model to integrate these models.
• Conducted code cleanup for efficiency.

Rationale:
Improves clarity and allows easier future modifications.

Testing:
Ensured accuracy and consistency in results post-refactor.

Review Request:
Feedback on the implementation and integration of the new models is welcomed.

[recce-cloud]

Summary

PR #2 refactors the customers model by extracting two CTEs into intermediate models (int_customer_orders and int_customer_payments), improving code maintainability without altering data. Row count comparison shows the customers table remains stable at 1,856 records, and all 4 configured validation checks passed with 100% data accuracy. Two downstream models (customer_order_pattern, customer_segments) are flagged for verification due to dependency changes, though no breaking changes were detected.

---

Key Changes

• Lineage refactoring: lineage_diff reveals 2 new intermediate models added to the dependency graph
• Code simplification: customers model reduced from 69 lines to a cleaner implementation (52-line net reduction)
• Row count stability: row_count_diff confirms customers maintained 1,856 records (0% change), orders stable at 280,844 records
• Schema integrity: schema_diff detected 0 column changes - all 8 columns remain structurally identical
• Data consistency: profile_diff on customers shows all column metrics identical (min/max/avg/distinct values unchanged across 8 columns)

---

Impact Analysis

graph LR
    stg_orders["stg_orders<br/>(view)"]:::unchanged
    stg_customers["stg_customers<br/>(view)"]:::unchanged
    stg_payments["stg_payments<br/>(view)"]:::unchanged
    customers["customers<br/>(table)"]:::modified
    customer_order_pattern["customer_order_pattern<br/>(table)"]:::impacted
    customer_segments["customer_segments<br/>(table)"]:::impacted
    int_customer_payments["int_customer_payments<br/>(table)"]:::added
    int_customer_orders["int_customer_orders<br/>(table)"]:::added

    stg_customers --> customers
    int_customer_orders --> customers
    int_customer_payments --> customers
    customers --> customer_order_pattern
    customers --> customer_segments
    stg_orders --> int_customer_payments
    stg_payments --> int_customer_payments
    stg_orders --> int_customer_orders

    classDef added fill:#d4edda,stroke:#28a745,color:#000000
    classDef removed fill:#f8d7da,stroke:#dc3545,color:#000000
    classDef modified fill:#fff3cd,stroke:#ffc107,color:#000000
    classDef impacted fill:#ffffff,stroke:#ffc107,color:#000000
    classDef unchanged fill:#ffffff,stroke:#d3d3d3,color:#999999

Loading

• ✅ New models successfully created: int_customer_orders and int_customer_payments both populated with 1,856 records (matching customers table cardinality)
• ⚠️ 2 downstream models impacted: customer_order_pattern and customer_segments depend on modified customers model - require validation that calculations remain correct
• ✅ Upstream dependencies stable: stg_orders, stg_customers, and stg_payments unchanged, supporting both new intermediates and modified parent
• 📝 Refactoring isolated to customers layer: No impact on production fact tables (orders stable at 280,844 records)

---

☑️ Checklist

All checks configured in the PR validation suite have been executed:

Name	Run Status	Impact Analysis
Query Diff of Customers Avg Lifetime Value	✅ PASS	Weekly average customer lifetime values match perfectly across 98 weeks - no query logic degradation
Row Count of Customers, Orders and Modified Table Models	✅ PASS	customers: 1,856 → 1,856 (0% change); orders: 280,844 → 280,844 (stable); new models: 1,856 records each
Model Schema of Customers, Orders and Modified Nodes	✅ PASS	Zero schema changes - all 8 columns in customers retain original data types and structure
Value Diff of Customers	✅ PASS	100% row match (1,856/1,856) on customer IDs and customer lifetime values - perfect data consistency

---

🔍 Suggested Actions

• Verify downstream calculations: Run query_diff on customer_order_pattern and customer_segments to confirm aggregations correctly reference the refactored customers model structure
• Validate intermediate model logic: Execute profile_diff on int_customer_orders and int_customer_payments to ensure payment and order distributions match expectations from source data
• Monitor compilation warnings: Address the dbt macro rendering warning for new intermediate models despite successful table creation - verify model YAML definitions are complete
• Spot-check downstream impacts: Compare a sample of 10-20 rows from customer_order_pattern before/after to manually confirm aggregation accuracy post-refactor
• Review CTE extraction rationale: Confirm that extracting CTEs into int_customer_orders and int_customer_payments aligns with project modeling standards and naming conventions
  Please use the link below to launch your Recce Cloud session.

Launch Recce Cloud Session

---

Was this summary helpful? 👍 👎