feat: support alterate partitioning #1176
Conversation
| )[0] | ||
|
|
||
|
|
||
| def normalize_partition(partition: str) -> tuple: |
There was a problem hiding this comment.
when fetching the partition ids from the system table, they come out as a string representation of a tuple OR just the raw string. It's unfortunate and I couldn't find a cleaner way to do this within clickhouse.
There was a problem hiding this comment.
if you do SELECT tuple(partition) does that force it to be consistent, or no?
There was a problem hiding this comment.
It doesn't:
wmbbe-67c :) select tuple(partition), toTypeName(partition), toTypeName(tuple(partition)) FROM system.parts where table = 'my_table';
SELECT
tuple(partition),
toTypeName(partition),
toTypeName(tuple(partition))
FROM system.parts
WHERE `table` = 'my_table'
Query id: eaea3ec0-b4ce-46f5-8411-f8e69133dfd3
┌─(partition)────┬─toTypeName(partition)─┬─toTypeName((partition))─┐
1. │ ('1') │ String │ Tuple(String) │
2. │ ('1') │ String │ Tuple(String) │
3. │ ('(\'90\',1)') │ String │ Tuple(String) │
4. │ ('(\'93\',3)') │ String │ Tuple(String) │
5. │ ('(\'91\',2)') │ String │ Tuple(String) │
└────────────────┴───────────────────────┴─────────────────────────┘
5 rows in set. Elapsed: 0.001 sec.
And it actually gets worse.
| f""" | ||
| INSERT INTO {table_name_builder.staging_dst_table(ClickHouseTable.ENTRIES)} | ||
| SELECT * | ||
| INSERT INTO {table_name_builder.staging_dst_table(ClickHouseTable.ENTRIES)} ({dst_list}) |
There was a problem hiding this comment.
INSERT INTO SELECT * fails with columns misalignment. We really want INSERT EXCEPT (...)... which isn't supported though SELECT EXCEPT is.
| ) | ||
| self.assertCountEqual(gt_stats, [(1, 0)]) | ||
|
|
||
| def test_repartitioned_entries_table(self): |
There was a problem hiding this comment.
the existing tests cover the existing partitioning structure.
| )[0] | ||
|
|
||
|
|
||
| def normalize_partition(partition: str) -> tuple: |
There was a problem hiding this comment.
if you do SELECT tuple(partition) does that force it to be consistent, or no?
…es into benb/support_alternate_partitionings
No description provided.