Change the amount type from DOUBLE to DECIMAL

[even-wei]

Overview:
This PR changes the amount type from DOUBLE to DECIMAL(10,2). Because for currency column, it make more sense to use the DECIMAL type instead of DOUBLE type.

Changes:

• CAST the type to DECIMAL

Rationale:
To better understand and mitigate any potential distortions caused by data rounding.

UPDATED (2025/03/07):
Please see the comment below

[popcornylu]

We found that the round effect may not be used in the jaffle shop analysis. We use to the type change instead.

Here is the old one

Overview:
This PR introduces a new Rounding Effect Analysis feature, aimed at analyzing and reporting the impacts of rounding in our data processing.

Changes:

• Implemented a new analysis module for rounding effects.
• Integrated this module with existing data processing pipelines.
• Added relevant documentation and examples for usage.

Rationale:
To better understand and mitigate any potential distortions caused by data rounding.

[recce-cloud]

Summary

PR #3 changes the amount type from DOUBLE precision number(38,6) to DECIMAL precision number(10,2) in the stg_payments staging model to better handle currency values and mitigate rounding distortions. Lineage analysis shows this impacts 5 downstream models with 280,844 order records affected, yet all row counts remain stable. All validation checks passed with zero data integrity issues detected, confirming the change is safe to merge.

---

Key Changes

• Modified Model: stg_payments (1 line changed in models/staging/stg_payments.sql)
• Type Conversion: AMOUNT column precision narrowed from number(38,6) to number(10,2) for better currency representation
• Cascading Effect: row_count_diff confirms 280,844 order records affected across impacted models with zero row count changes (orders: 280,844 → 280,844)
• Data Preservation: profile_diff shows AMOUNT values (range 1.00–30.00) fit safely within new precision constraints; all statistics unchanged

---

Impact Analysis

graph LR
    raw_payments["raw_payments<br/>(source)"]:::unchanged
    stg_payments["stg_payments<br/>(view)"]:::modified
    customers["customers<br/>(table)"]:::impacted
    customer_order_pattern["customer_order_pattern<br/>(table)"]:::impacted
    customer_segments["customer_segments<br/>(table)"]:::impacted
    orders["orders<br/>(table)"]:::impacted
    orders_daily_summary["orders_daily_summary<br/>(incremental)"]:::impacted

    raw_payments --> stg_payments
    stg_payments --> customers
    stg_payments --> orders
    customers --> customer_order_pattern
    customers --> customer_segments
    orders --> orders_daily_summary

    classDef added fill:#d4edda,stroke:#28a745,color:#000000
    classDef removed fill:#f8d7da,stroke:#dc3545,color:#000000
    classDef modified fill:#fff3cd,stroke:#ffc107,color:#000000
    classDef impacted fill:#ffffff,stroke:#ffc107,color:#000000
    classDef unchanged fill:#ffffff,stroke:#d3d3d3,color:#999999

Loading

• 📝 5 downstream models impacted: customers, orders, customer_order_pattern, customer_segments, orders_daily_summary
• ✅ Row counts stable: row_count_diff shows customers (1,856 → 1,856) and orders (280,844 → 280,844) with zero changes across all cascading models
• 📝 Precision narrowing applied consistently: schema_diff detected AMOUNT column type changes from number(38,6) to number(10,2) in stg_payments; cascades to orders payment columns with matching precision adjustments
• ✅ Data distribution preserved: profile_diff confirms AMOUNT statistics unchanged (min: 1.00, max: 30.00, avg: 15.50); all 30 distinct payment values remain valid
• ✅ Referential integrity maintained: No NULL value changes; 100% data matches across customers.CUSTOMER_LIFETIME_VALUE between base and current environments

---

☑️ Checklist

Name	Run Status	Impact Analysis
Schema Consistency	✅ OK	No unintended columns added/removed; precision changes are intentional
Row Count Stability	✅ OK	All impacted models maintain row counts; incremental orders_daily_summary behaves as expected
Query Result Matching	✅ OK	Customer lifetime value aggregations identical (104 rows, 100% match)
Value Distribution	✅ OK	All 1,856 customer lifetime values match 100%; payment amounts fit within number(10,2) range

---

🔍 Suggested Actions

• Verify orders aggregate calculations: Run query_diff on sum(AMOUNT) and avg(AMOUNT) by order_date to confirm no rounding side-effects in reporting queries after decimal precision change
• Test downstream dashboards: Confirm that any BI tools consuming customers and orders models display monetary values correctly with 2-decimal precision instead of 6
• Document precision rationale: Add dbt model metadata to stg_payments explaining the number(10,2) choice for USD/currency columns and why 6-decimal precision was excessive
• Review financial reconciliation: Cross-check total payment amounts in orders_daily_summary against source raw_payments to ensure rounding doesn't impact month-end reconciliation
• Monitor incremental rebuild: Validate that orders_daily_summary (incremental materialization) correctly applies the new precision when future daily inserts occur
  Please use the link below to launch your Recce Cloud session.

Launch Recce Cloud Session

---

Was this summary helpful? 👍 👎