Skip to content

Commit d6cb49b

Browse files
roshit-omanakuttanroshit-omanakuttan
committed
Add diagrams, helpers, and sample data for ecom and feed
Introduces new Mermaid diagrams for e-commerce and feed access patterns and key design, as well as shared DynamoDB concepts. Adds Python and TypeScript helper utilities for key formatting. Provides comprehensive sample CSV data for e-commerce and feed domains, including users, orders, products, posts, comments, likes, and more. Updates access pattern documentation and query samples for improved clarity and parameterization. Minor corrections to access pattern consistency levels and sample seed data.
1 parent 99dadc5 commit d6cb49b

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

41 files changed

+581
-83
lines changed

‎docs/assets/diagrams/ecom/A1_access-pattern-matrix.mmd‎

Lines changed: 58 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,58 @@
1+
---
2+
config:
3+
layout: dagre
4+
---
5+
flowchart LR
6+
subgraph SVC["Consumers / Services"]
7+
U1["ECOM-001 UserSvc<br>Get User"]
8+
O2["ECOM-002 OrderSvc<br>List Orders by User"]
9+
O3["ECOM-003 OrderSvc<br>Get Order"]
10+
O4["ECOM-004 OrderSvc<br>List OrderLines"]
11+
B5["ECOM-005 Backoffice<br>Orders by Status/Month"]
12+
P6["ECOM-006 PaymentSvc<br>Get Payments (by Order)"]
13+
P7["ECOM-007 PaymentSvc<br>Find Pending"]
14+
S8["ECOM-008 ShipmentSvc<br>Get Shipments (by Order)"]
15+
I9["ECOM-009 InventorySvc<br>Reserve/Decrement"]
16+
OP10["ECOM-010 Ops<br>Orders Dashboard (status+carrier)"]
17+
AR11["ECOM-011 Archive<br>Export Old Orders"]
18+
end
19+
subgraph PATHS["Physical Access Paths"]
20+
B0["BASE TABLE<br>PK+SK"]
21+
G1["GSI1 OrdersByUser<br>PK: USER#&lt;id&gt;<br>SK: &lt;ts&gt;#ORDER#&lt;id&gt;"]
22+
G2["GSI2 OrdersByStatus<br>PK: STATUS#&lt;status&gt;#&lt;yyyy-mm&gt;<br>SK: &lt;ts&gt;#&lt;shard&gt;#ORDER#&lt;id&gt;"]
23+
GP["GSI_PAY Pending<br>PK: PAYSTATUS#&lt;status&gt;<br>SK: &lt;attempted_at&gt;#ORDER#&lt;id&gt;"]
24+
GS["GSI_SHIP Ops slice<br>PK: SHIP#&lt;status&gt;#&lt;carrier&gt;#&lt;yyyy-mm&gt;<br>SK: &lt;updated_at&gt;#ORDER#&lt;id&gt;"]
25+
OFF["Offline / Analytics<br>Export-to-S3 • Athena/Glue"]
26+
end
27+
U1 -- "GetItem<br>PK=USER#id, SK=META" --> B0
28+
O2 -- "Query<br>GSI1PK=USER#id<br>DESC" --> G1
29+
O3 -- "GetItem<br>PK=ORDER#id, SK=META" --> B0
30+
O4 -- "Query<br>PK=ORDER#id + begins_with(SK,'LINE#')<br>ASC" --> B0
31+
B5 -- "Query<br>GSI2PK=STATUS#S#YYYY-MM<br>DESC" --> G2
32+
P6 -- "Query<br>PK=ORDER#id + begins_with(SK,'PAYMENT#')<br>DESC" --> B0
33+
P7 -- Query<br>PAYSTATUS#Pending<br>ASC --> GP
34+
S8 -- "Query<br>PK=ORDER#id + begins_with(SK,'SHIPMENT#')<br>DESC" --> B0
35+
I9 -- "UpdateItem + Condition<br>PK=PRODUCT#id, SK=META" --> B0
36+
OP10 -- Query (ops slice)<br>status+carrier+month --> GS
37+
AR11 -- Periodic export --> OFF
38+
U1:::base
39+
O2:::gsi
40+
O3:::base
41+
O4:::base
42+
B5:::gsi
43+
P6:::base
44+
P7:::gsi
45+
S8:::base
46+
I9:::wr
47+
OP10:::gsi
48+
AR11:::off
49+
B0:::base
50+
G1:::gsi
51+
G2:::gsi
52+
GP:::gsi
53+
GS:::gsi
54+
OFF:::off
55+
classDef base fill:#ecfdf5,stroke:#10b981,color:#064e3b
56+
classDef gsi fill:#fff7ed,stroke:#fb923c,color:#7c2d12
57+
classDef wr fill:#eff6ff,stroke:#60a5fa,color:#0c4a6e
58+
classDef off fill:#f8fafc,stroke:#94a3b8,color:#334155,stroke-dasharray:4 3

‎docs/assets/diagrams/ecom/A2_key-design-cheatsheet.mmd‎

Lines changed: 85 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,85 @@
1+
flowchart TB
2+
%% Styles
3+
classDef base fill:#ecfdf5,stroke:#10b981,rx:6,ry:6,color:#064e3b;
4+
classDef gsi fill:#fff7ed,stroke:#fb923c,rx:6,ry:6,color:#7c2d12;
5+
classDef item fill:#f8fafc,stroke:#94a3b8,rx:6,ry:6,color:#334155;
6+
classDef rule fill:#eff6ff,stroke:#60a5fa,rx:6,ry:6,color:#0c4a6e;
7+
classDef misc fill:#fefce8,stroke:#eab308,rx:6,ry:6,color:#713f12;
8+
9+
subgraph BASE["Base Table Keys"]
10+
BPK["PK formats:
11+
• USER#<user_id>
12+
• ORDER#<order_id>
13+
• PRODUCT#<product_id>"]:::base
14+
BSK["SK formats:
15+
• META
16+
• LINE#<0001..>
17+
• PAYMENT#<iso>
18+
• SHIPMENT#<iso>
19+
• INV#SNAP#<yyyy-mm-dd>"]:::base
20+
end
21+
22+
subgraph GSIs["Global Secondary Indexes"]
23+
G1["GSI1 OrdersByUser
24+
PK: USER#<user_id>
25+
SK: <ts>#ORDER#<order_id> (DESC)"]:::gsi
26+
G2["GSI2 OrdersByStatus
27+
PK: STATUS#<status>#<yyyy-mm>
28+
SK: <ts>#<shard>#ORDER#<order_id> (DESC)"]:::gsi
29+
G3["GSI_PAY PendingPayments (optional)
30+
PK: PAYSTATUS#<status>
31+
SK: <attempted_at>#ORDER#<order_id>"]:::gsi
32+
G4["GSI_SHIP OpsSlice (optional)
33+
PK: SHIP#<status>#<carrier>#<yyyy-mm>
34+
SK: <updated_at>#ORDER#<id>"]:::gsi
35+
end
36+
37+
subgraph ITEMS["Item Types & SK prefixes"]
38+
IU["User
39+
PK=USER#id, SK=META"]:::item
40+
IO["Order (meta)
41+
PK=ORDER#id, SK=META"]:::item
42+
IL["OrderLine
43+
PK=ORDER#id, SK=LINE#<no>"]:::item
44+
IP["Payment
45+
PK=ORDER#id, SK=PAYMENT#<iso>"]:::item
46+
IS["Shipment
47+
PK=ORDER#id, SK=SHIPMENT#<iso>"]:::item
48+
IV["Inventory Snapshot
49+
PK=PRODUCT#id, SK=INV#SNAP#<ymd>"]:::item
50+
end
51+
52+
subgraph RULES["Write Rules / Idempotency / Optimistic Concurrency"]
53+
R1["Create Order:
54+
PutItem IF attribute_not_exists(PK) AND attribute_not_exists(SK)"]:::rule
55+
R2["Add Line:
56+
PutItem IF attribute_not_exists(SK) (idempotent line_no)"]:::rule
57+
R3["Authorize Payment:
58+
PutItem IF attribute_not_exists(SK) (idempotent by timestamp/id)"]:::rule
59+
R4["Inventory Decrement:
60+
Update SET available = available - :q
61+
IF available >= :q"]:::rule
62+
R5["Shipment Update:
63+
Update with version/etag OR condition on current status"]:::rule
64+
end
65+
66+
subgraph MISC["TTL / Streams / Size / Projection"]
67+
T1["TTL (optional):
68+
• old snapshots
69+
• temp reservations (hold TTL)"]:::misc
70+
T2["Streams:
71+
• Project events → outbox/bus
72+
• Build GSI_PAY / metrics
73+
• Backfill sinks"]:::misc
74+
T3["Item size:
75+
• keep < 400KB
76+
• denormalize only hot attrs"]:::misc
77+
T4["Projection:
78+
• GSIs = keys + minimal INCLUDE"]:::misc
79+
end
80+
81+
%% Link the sections
82+
BPK --> BSK
83+
GSIs --> ITEMS
84+
ITEMS --> RULES
85+
RULES --> MISC

‎docs/assets/diagrams/feed/B1_access-pattern-matrix.mmd‎

Lines changed: 56 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,56 @@
1+
flowchart LR
2+
subgraph SVC["Consumers / Services"]
3+
U1["FEED-001 UserSvc<br>Get User"]
4+
P2["FEED-002 PostSvc<br>List Posts"]
5+
F3["FEED-003 FeedSvc<br>Get Home Feed"]
6+
P4["FEED-004 PostSvc<br>Get Post"]
7+
E5["FEED-005 EngageSvc<br>List Likes"]
8+
E6["FEED-006 EngageSvc<br>List Comments"]
9+
S7["FEED-007 SearchSvc<br>Posts by Tag"]
10+
G8["FEED-008 GraphSvc<br>Followers of"]
11+
N9["FEED-009 NotifySvc<br>Notifications (unread)"]
12+
O10["FEED-010 Ops<br>Top Authors (period)"]
13+
end
14+
subgraph PATHS["Physical Access Paths"]
15+
B0["BASE TABLE<br>PK+SK"]
16+
G1["GSI1 PostsByAuthor<br>PK: USER#&lt;author&gt;<br>SK: &lt;ts&gt;#POST#&lt;id&gt;"]
17+
G2["GSI2 PostsByTag<br>PK: TAG#&lt;tag&gt;<br>SK: &lt;ts&gt;#POST#&lt;id&gt;"]
18+
G3["GSI3 FeedByUser<br>PK: FEED#&lt;user&gt;<br>SK: &lt;ts&gt;#&lt;shard&gt;#POST#&lt;id&gt;"]
19+
GF["GSI_FOL Followers<br>PK: FOLLOWEE#&lt;user&gt;<br>SK: &lt;ts&gt;#FOLLOWER#&lt;id&gt;"]
20+
GN["GSI_NOTIF Unread<br>PK: USER#&lt;id&gt;#UNREAD<br>SK: &lt;ts&gt;#&lt;notif&gt;"]
21+
M1["Metrics / Leaderboard<br>(precomputed via Streams→Lambda)"]
22+
end
23+
U1 -- "GetItem<br>PK=USER#id, SK=META" --> B0
24+
P2 -- "Query<br>GSI1PK=USER#author<br>DESC" --> G1
25+
F3 -- "Query<br>GSI3PK=FEED#user<br>DESC" --> G3
26+
P4 -- "GetItem<br>PK=POST#id, SK=META" --> B0
27+
E5 -- "Query<br>PK=POST#id + begins_with(SK,'LIKE#')<br>DESC" --> B0
28+
E6 -- "Query<br>PK=POST#id + begins_with(SK,'COMMENT#')<br>ASC" --> B0
29+
S7 -- "Query<br>GSI2PK=TAG#tag<br>DESC" --> G2
30+
G8 -- "Query<br>PK=FOLLOWEE#user<br>DESC" --> GF
31+
N9 -- "Query<br>PK=USER#id#UNREAD<br>DESC" --> GN
32+
O10 -- Read<br>Top N by month --> M1
33+
34+
U1:::base
35+
P2:::gsi
36+
F3:::gsi
37+
P4:::base
38+
E5:::base
39+
E6:::base
40+
S7:::gsi
41+
G8:::gsi
42+
N9:::gsi
43+
O10:::off
44+
B0:::base
45+
G1:::gsi
46+
G2:::gsi
47+
G3:::gsi
48+
GF:::gsi
49+
GN:::gsi
50+
M1:::off
51+
classDef base fill:#ecfdf5,stroke:#10b981,color:#064e3b
52+
classDef gsi fill:#fff7ed,stroke:#fb923c,color:#7c2d12
53+
classDef wr fill:#eff6ff,stroke:#60a5fa,color:#0c4a6e
54+
classDef off fill:#f8fafc,stroke:#94a3b8,color:#334155,stroke-dasharray:4 3
55+
56+

‎docs/assets/diagrams/feed/B2_key-design-cheatsheet.mmd‎

Lines changed: 87 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,87 @@
1+
flowchart TB
2+
%% Styles
3+
classDef base fill:#ecfdf5,stroke:#10b981,rx:6,ry:6,color:#064e3b;
4+
classDef gsi fill:#fff7ed,stroke:#fb923c,rx:6,ry:6,color:#7c2d12;
5+
classDef item fill:#f8fafc,stroke:#94a3b8,rx:6,ry:6,color:#334155;
6+
classDef rule fill:#eff6ff,stroke:#60a5fa,rx:6,ry:6,color:#0c4a6e;
7+
classDef misc fill:#fefce8,stroke:#eab308,rx:6,ry:6,color:#713f12;
8+
9+
subgraph BASE["Base Table Keys"]
10+
BPK["PK formats:
11+
• USER#<user_id>
12+
• POST#<post_id>
13+
• FOLLOWEE#<user_id>"]:::base
14+
BSK["SK formats:
15+
• META
16+
• COMMENT#<comment_id>
17+
• LIKE#<user_id>
18+
• TAG#<tag>
19+
• FOLLOWER#<user_id>#<ts>"]:::base
20+
end
21+
22+
subgraph GSIs["Global Secondary Indexes"]
23+
G1["GSI1 PostsByAuthor
24+
PK: USER#<author_id>
25+
SK: <ts>#POST#<post_id> (DESC)"]:::gsi
26+
G2["GSI2 PostsByTag
27+
PK: TAG#<tag>
28+
SK: <ts>#POST#<post_id> (DESC)"]:::gsi
29+
G3["GSI3 FeedByUser
30+
PK: FEED#<user_id>
31+
SK: <ts>#<shard>#POST#<post_id> (DESC)"]:::gsi
32+
G4["GSI_FOL Followers
33+
PK: FOLLOWEE#<user_id>
34+
SK: <ts>#FOLLOWER#<follower_id>"]:::gsi
35+
G5["GSI_NOTIF Unread
36+
PK: USER#<user_id>#UNREAD
37+
SK: <ts>#<notif_id>"]:::gsi
38+
end
39+
40+
subgraph ITEMS["Item Types & SK prefixes"]
41+
UU["User
42+
PK=USER#id, SK=META"]:::item
43+
UP["Post (meta)
44+
PK=POST#id, SK=META"]:::item
45+
UC["Comment
46+
PK=POST#id, SK=COMMENT#<id>"]:::item
47+
UL["Like
48+
PK=POST#id, SK=LIKE#<user_id>"]:::item
49+
UT["Post Tag
50+
PK=POST#id, SK=TAG#<tag>"]:::item
51+
UF["Follow Edge
52+
PK=FOLLOWEE#user, SK=FOLLOWER#follower#<ts>"]:::item
53+
UN["Notification (Unread)
54+
PK=USER#id#UNREAD, SK=<ts>#<notif_id>"]:::item
55+
end
56+
57+
subgraph RULES["Write Rules / Idempotency / Ordering"]
58+
R1["Create Post:
59+
PutItem IF attribute_not_exists(PK) AND attribute_not_exists(SK)"]:::rule
60+
R2["Add Like:
61+
PutItem IF attribute_not_exists(SK) (idempotent)"]:::rule
62+
R3["Add Comment:
63+
PutItem IF attribute_not_exists(SK) (idempotent by comment_id)"]:::rule
64+
R4["Fanout → Feed:
65+
PutItem to GSI3 partition(s) with sharded SK"]:::rule
66+
R5["Follow:
67+
PutItem IF attribute_not_exists(SK)"]:::rule
68+
end
69+
70+
subgraph MISC["TTL / Streams / Sharding / Projection"]
71+
T1["TTL:
72+
• ephemeral notifications (read true → move or delete)
73+
• temporary fanout artifacts"]:::misc
74+
T2["Streams:
75+
• Build fanout / notifications
76+
• Metrics (top authors, tag velocity)"]:::misc
77+
T3["Sharding:
78+
• home feed SK: <ts>#<shard>#POST#<id>"]:::misc
79+
T4["Projection:
80+
• minimal INCLUDE on GSIs to keep RCU low"]:::misc
81+
end
82+
83+
%% Link the sections
84+
BPK --> BSK
85+
GSIs --> ITEMS
86+
ITEMS --> RULES
87+
RULES --> MISC

‎docs/assets/diagrams/shared/C1_partition-bookshelf.png‎

7.09 KB
Loading

‎docs/assets/diagrams/shared/C2_lsi-vs-gsi.mmd‎

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,21 @@
1+
flowchart LR
2+
classDef base fill:#ecfdf5,stroke:#10b981,rx:6,ry:6;
3+
classDef lsi fill:#f5f3ff,stroke:#8b5cf6,rx:6,ry:6;
4+
classDef gsi fill:#fff7ed,stroke:#fb923c,rx:6,ry:6;
5+
6+
subgraph Base["Base Table (PK, SK)"]
7+
BPK[PK]:::base
8+
BSK[SK]:::base
9+
end
10+
11+
subgraph LSI["Local Secondary Index (same PK, different SK)"]
12+
LPK[PK]:::lsi
13+
LSK[SK']:::lsi
14+
note1["Strongly consistent reads; 10GB per item-collection applies"]:::lsi
15+
end
16+
17+
subgraph GSI["Global Secondary Index (new PK, new SK)"]
18+
GPK[PK']:::gsi
19+
GSK[SK']:::gsi
20+
note2["Eventually consistent; capacity decoupled; projection controls cost"]:::gsi
21+
end

‎docs/assets/diagrams/shared/C3_conditional-write-truth-table.png‎

7.11 KB
Loading

‎docs/assets/diagrams/shared/C4_transactions-limits.mmd‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,7 @@
1+
flowchart TB
2+
classDef t fill:#eef2ff,stroke:#6366f1,rx:6,ry:6;
3+
T["TransactWriteItems limits"]:::t
4+
T --> A["Up to 25 items or 4 MB payload"]
5+
T --> B["All-or-nothing atomicity"]
6+
T --> C["Higher latency, higher cost"]
7+
T --> D["Use for tight atomic groups only"]

‎docs/assets/diagrams/shared/C5_error-taxonomy.png‎

6.97 KB
Loading

‎docs/assets/diagrams/shared/C6_migration-timeline.mmd‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,7 @@
1+
timeline
2+
title Schema Evolution (Blue/Green Read Path)
3+
2025-10-01 : Design new shape + feature flags
4+
2025-10-10 : Dual-write old+new
5+
2025-10-20 : Backfill historical data
6+
2025-10-25 : Flip reads to new path (canary -> 100%)
7+
2025-11-05 : Remove old writes (after bake time)

0 commit comments

Comments
 (0)