Skip to content

LDLite fails to transform columns that contain some uuids and some non-uuids #54

@brycekbargar

Description

@brycekbargar

Unfortunately we have another problem since LDLite 3.0. The data from the /users endpoint is downloaded but the transformed tables are not generated.
The error message is (values deleted by me):

running JSON transform: error executing SQL: INSERT INTO "folio_users"."users__t__personal__addresses"(__id,"id","type","active","barcode","username","created_date","patron_group","updated_date","enrollment_date","external_system_id","personal__email","personal__phone","personal__last_name","personal__first_name","personal__middle_name","personal__mobile_phone","personal__preferred_first_name","personal__preferred_contact_type_id","personal__addresses__o","personal__addresses__city","personal__addresses__region","personal__addresses__country_id","personal__addresses__postal_code","personal__addresses__address_line1","personal__addresses__address_line2","personal__addresses__address_type_id","personal__addresses__primary_address")VALUES(...)
Table users__t is also not generated. I suspect the problem lies in the column external_system_id which is automatically generated with datatype uuid since LDLite 3.0. The field externalSystemId contains mostly uuids (30.000 user records with uuids) but we have also 1.800 user records with other types of text ids. If I edit the externalSystemId of the problematic record in FOLIO to a random uuid, the next record with a text id in that field causes the error. If I add a query that filters the records to those with only externalSystemIds containing text ids, everything works fine (and the datatype of column external_system_id is text)

Table users__t is also not generated. I suspect the problem lies in the column external_system_id which is automatically generated with datatype uuid since LDLite 3.0. The field externalSystemId contains mostly uuids (30.000 user records with uuids) but we have also 1.800 user records with other types of text ids. If I edit the externalSystemId of the problematic record in FOLIO to a random uuid, the next record with a text id in that field causes the error. If I add a query that filters the records to those with only externalSystemIds containing text ids, everything works fine (and the datatype of column external_system_id is text)

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions