Skip to content

Add selectExpression for transforming primary keys. #197

@IgorOleksyuk

Description

@IgorOleksyuk

Add a support for data transformation during migration using various hash functions. This feature is useful for:

  • Data anonymization: Hash sensitive data like emails, phone numbers, or personal identifiers
  • Data obfuscation: Transform data while maintaining referential integrity
  • Compliance requirements: Meet data privacy regulations by hashing PII data

We need supported Hash Functions:

  • MurmurHash3: Fast, non-cryptographic hash function (returns integer)
  • MD5: 128-bit cryptographic hash (returns hex string)
  • SHA-1: 160-bit cryptographic hash (returns hex string)
  • SHA-256: 256-bit cryptographic hash (returns hex string)
  • XXHASH64: Fast, 64-bit non-cryptographic hash (returns long integer)

Transform Expression Parameters

  • columnName: The source column to transform to hash value
  • rule: Hash function to apply (murmurhash3, md5, sha-1, sha-256, xxhash64)
  • alias: Optional new column name (if not provided, replaces original column)
  • keepSource: Boolean to keep original column when using alias (default: false)

--json-mapping '{ "keyspaces": { "transformation": { "enabled": true, "transformExpressions": [ { "columnName": "email", "rule": "md5", "alias": "email_hash", "keepSource": false }, { "columnName": "phone", "rule": "sha-256", "alias": "phone_hash", "keepSource": true }, { "columnName": "user_id", "rule": "murmurhash3" } ] } } }'

Metadata

Metadata

Assignees

Labels

enhancementNew feature or request

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions