(Demo) Fix: customer lifetime value calculation in customers

[even-wei]

👋 This is a demo pull request of dbt model changes, powered by Recce to provide proof of correctness during development and change review.
This is still early and we'd love your feedback!

Description & motivation

The customer lifetime value (CLV) in customers model incorrectly included orders that were not yet completed, leading to potentially inaccurate business insights.
I modified the CLV calculation within customers to consider only "completed" orders.

To-do before merge

Notify Stakeholders

• Ask Sarah to review the customer segment impacts on marketing (see impact section below)

Lineage:

Lineage DAG Diff

Modified customers model to adjust customer_lifetime_value calculation.

 

---

Validation of models:

As expected, the customer lifetime value has changed. The average value of CLV has been reduced and this affects 98% of customers.

Query Diff: customers

The customer_lifetime_value has reduced for certain customers. This is expected given the change to the CLV calculation.

 

---

Profile Diff: customers

Min/max remains unchanged, average value has dropped from 2758.6 to 1871.76 as expected.

 

---

Value Diff: customers

Only 1.19% match on customer_lifetime_value.

 

---

Magic metric of customers

Our magic metric, average lifetime value, goes down as well.

SQL

SELECT
    DATE_TRUNC('week', first_order) AS first_order_week,
    AVG(customer_lifetime_value) AS avg_lifetime_value
FROM
    {{ ref("customers") }}
WHERE first_order is not NULL    
GROUP BY
    first_order_week
ORDER BY
    first_order_week;

Impact considerations:

The adjustment to CLV values has resulted in 26.45% change of customer ‘value segment’. This impact is also expected.

Value Diff: customer_segments

customer_lifetime_value as expected matches parent table at 1.19%.
value_segment is changed by 26.45%.

 

---

Query Diff: customer_segments

value_segment has changed for some of those customers with a decreased CLV. This is expected and I'll notify the business team.

---

Top-k diff: value_segment

We've seen a 17% drop in high-value customers. We need to discuss with stakeholders whether an adjustment to the threshold is necessary.

There is no change in customer_order_pattern.

Value diff: customer_order_pattern

There is no change.

Changes to existing models:

N/A

Checklist:

This checklist is mostly useful as a reminder of small things that can easily be
forgotten – it is meant as a helpful tool rather than hoops to jump through.
Put an x in all the items that apply, make notes next to any that haven't been
addressed, and remove any items that are not relevant to this PR.

• My pull request represents one logical piece of work.
• My commits are related to the pull request and look clean.
• My SQL follows the style guide.
• I have materialized my models appropriately.
• I have added appropriate tests and documentation to any new models.
• I have updated the README file.

[clkao]

Should we also check customer_order_pattern?

[even-wei]

There is no change in customer_order_pattern.
I will add it and update the checklist.

[even-wei]

Since the sources have been updated in the production environment, I've rebased the PR, rerun the checks, and updated the body. Most of the checks yield the same results, but I noticed that 99% of the value segments are classified as 'high value' (refer to the Top-k diff above). I plan to review the threshold next week.

[recce-cloud]

Summary

PR #1 modifies the customers model to fix the customer lifetime value (CLV) calculation to include only completed orders. The change is isolated to a single file (models/customers.sql) with minimal syntax changes, but results in a 32.1% reduction in average customer lifetime values across the dataset. Lineage analysis reveals 2 downstream models (customer_order_pattern and customer_segments) will be impacted by the recalculated values.

---

Key Changes

• Model Modified: customers table

  • File: models/customers.sql (1 addition, 0 deletions)
  • Change: Fixed CLV calculation to exclude non-completed orders
  • Row count impact: Stable at 1,856 records (no additions/deletions)

• Customer Lifetime Value Recalculation:

  • Profile diff shows average CLV reduced from 2,758.60 → 1,871.77 (-32.1%)
  • Max CLV dropped from 10,092 → 6,852 (-32.1%)
  • Median CLV decreased from 2,126.50 → 1,451.00 (-31.8%)
  • Value diff indicates 98.81% mismatch (1,834 out of 1,856 records changed)
  • 5 new NULL values introduced in CLV column (0.27% of dataset)

---

Impact Analysis

Lineage Analysis reveals the customers model is the change epicenter with 2 downstream dependencies:

graph LR
    stg_customers["stg_customers<br/>(view)"]:::unchanged
    stg_orders["stg_orders<br/>(view)"]:::unchanged
    stg_payments["stg_payments<br/>(view)"]:::unchanged
    customers["customers<br/>(table)"]:::modified
    customer_order_pattern["customer_order_pattern<br/>(table)"]:::impacted
    customer_segments["customer_segments<br/>(table)"]:::impacted

    stg_customers --> customers
    stg_orders --> customers
    stg_payments --> customers
    customers --> customer_order_pattern
    customers --> customer_segments

    classDef added fill:#d4edda,stroke:#28a745,color:#000000
    classDef removed fill:#f8d7da,stroke:#dc3545,color:#000000
    classDef modified fill:#fff3cd,stroke:#ffc107,color:#000000
    classDef impacted fill:#ffffff,stroke:#ffc107,color:#000000
    classDef unchanged fill:#ffffff,stroke:#d3d3d3,color:#999999

Loading

• 🟨 Modified Model: customers - core change point for CLV logic
• 🟨 Impacted Downstream: customer_order_pattern and customer_segments inherit the recalculated CLV values
• ✅ Upstream Dependencies Unchanged: stg_customers, stg_orders, stg_payments remain stable

---

☑️ Checklist

Name	Run Status	Impact Analysis
Row Count Diff	✅ PASSED	✅ Row counts stable at 1,856 (customers) and 280,844 (orders) - no data loss detected
Schema Diff	✅ PASSED	✅ All 7 columns in customers unchanged - no new/removed/modified columns
Query Diff - Customers AVG Lifetime Value	⚠️ FAILED	⚠️ 30-33% reduction across all weekly cohorts - expected due to filtering completed orders only
Value Diff - Customer Lifetime Value	🔴 CRITICAL	🔴 98.81% mismatch rate (1,834 of 1,856 records differ) - fundamental recalculation confirmed; 5 new NULL values introduced

---

🔍 Suggested Actions

• Validate CLV calculation logic: Review the SQL modification in models/customers.sql to confirm the "completed orders only" filter is correctly implemented and matches business requirements.
• Investigate NULL values in customer_lifetime_value: Run query_diff on the 5 customers with new NULL CLV values to understand why the calculation failed for these specific records.
• Verify downstream impacts: Confirm that customer_order_pattern and customer_segments produce expected results with the ~32% reduction in CLV values, particularly for business metrics and reports that depend on these models.
• Review data quality in source: Run profile_diff on stg_orders to validate that the "completed" order status filtering logic correctly reflects the order data quality and completeness.
• Test affected reports: Check dependent dashboards and reports using customer_order_pattern and customer_segments to ensure they display correct insights with the recalculated lifetime values.
  Please use the link below to launch your Recce Cloud session.

Launch Recce Cloud Session

---

Was this summary helpful? 👍 👎