generated from amazon-archives/__template_Apache-2.0
-
Notifications
You must be signed in to change notification settings - Fork 15
Labels
enhancementNew feature or requestNew feature or request
Description
Add a support for data transformation during migration using various hash functions. This feature is useful for:
- Data anonymization: Hash sensitive data like emails, phone numbers, or personal identifiers
- Data obfuscation: Transform data while maintaining referential integrity
- Compliance requirements: Meet data privacy regulations by hashing PII data
We need supported Hash Functions:
- MurmurHash3: Fast, non-cryptographic hash function (returns integer)
- MD5: 128-bit cryptographic hash (returns hex string)
- SHA-1: 160-bit cryptographic hash (returns hex string)
- SHA-256: 256-bit cryptographic hash (returns hex string)
- XXHASH64: Fast, 64-bit non-cryptographic hash (returns long integer)
Transform Expression Parameters
columnName: The source column to transform to hash valuerule: Hash function to apply (murmurhash3, md5, sha-1, sha-256, xxhash64)alias: Optional new column name (if not provided, replaces original column)keepSource: Boolean to keep original column when using alias (default: false)
--json-mapping '{ "keyspaces": { "transformation": { "enabled": true, "transformExpressions": [ { "columnName": "email", "rule": "md5", "alias": "email_hash", "keepSource": false }, { "columnName": "phone", "rule": "sha-256", "alias": "phone_hash", "keepSource": true }, { "columnName": "user_id", "rule": "murmurhash3" } ] } } }'
Metadata
Metadata
Assignees
Labels
enhancementNew feature or requestNew feature or request