Skip to content

FlatConnect boxes table not populating any data #1584

@KELSEYDOWLING7

Description

@KELSEYDOWLING7

(Issue created from Teams discussion)

Issue
FlatConnect.boxes is currently empty. The last known date that we have data was 2/28/26

Investigations thus far
The recent prod push on February 25th included changes to remove a "box data converter"

The schema changed in late February: https://console.cloud.google.com/bigquery?ws=!1m7!1m6!12m5!1m3!1snih-nci-dceg-connect-prod-6d04!2sus-central1!3sc72f615c-5a5b-4e98-85a9-ac0e9492ef1c!2e1

Additional information on the potential root cause:
On February 27, prod Connect data contained bags with populated d_234868461 arrays (tube data). On February 28, a full historical query of prod data shows bags_with_tubes = 0. Since the scheduled transformation performs UNNEST(tubesInBag), and UNNEST of empty arrays returns zero rows, the scheduled job correctly produced an empty result set and replaced FlatConnect.boxes with zero rows.
The root cause is the absence of tube data in prod as of February 28

Requested Outcome
The goal would be to move Warren's current flattening code out of BQ and have Eddie add this to his current Airflow flattening process as we have for all other FlatConnect tables

Metadata

Metadata

Labels

No labels
No labels

Type

No type

Projects

Status

Backlog

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions