Suggested Network Architecture Improvements for OpenMANET

[yash7158]

Discussion: Network Provisioning and Conflict Handling

Hey guys, I was reading through the networking docs and came across some questions about how IP allocation works in OpenMANET. I haven't actually deployed it yet, so maybe I'm missing something or this is already handled somehow. I discussed my concerns and potential solutions with Claude AI to help organize my thoughts, and the detailed write-up below is a result of that conversation.

Would love to hear from folks who've actually run this in production whether these are real issues or if I'm overthinking things.

---

Context

I've been reading through the OpenMANET networking documentation and trying to understand the design choices around IP allocation. I haven't physically deployed OpenMANET yet, so I might be missing something obvious or there may already be solutions to what I'm about to describe. Would appreciate feedback from folks who've actually run this in the field.

Potential Issue: IP Assignment Conflicts?

From the docs, it looks like each mesh point randomly selects a static address like 10.41.113.1, and then runs DHCP for clients starting around that address. My concern is about potential conflicts:

Scenario 1: Static IP collision
If two nodes happen to randomly pick 10.41.113.1, they'd conflict. Statistically rare with a /16, but possible.

Scenario 2: DHCP range overlap
If Node A picks 10.41.113.1 and hands out leases starting at .2, and Node B randomly picks 10.41.113.8, now Node B's static IP falls inside Node A's DHCP pool.

Questions:

• Is there duplicate address detection happening that I missed in the docs?
• Does the random selection algorithm avoid these cases somehow?
• In practice, have people seen these conflicts or is the address space large enough that it's not a real problem?

Proposed Solution: API-Based Provisioning

If conflicts are indeed a concern worth addressing, here's an approach that might help. Again, this might already exist in some form or there might be good reasons why this wouldn't work:

Concept

Have the mesh gate act as a central coordinator for IP allocation:

1. Bootstrap Phase

   • Fresh node boots, gets temporary DHCP from gate (range: 10.41.x.2-254)
   • This is just for initial connectivity

2. Registration Phase

   • Node contacts gate API: "I'm MAC XX:XX:XX, need an IP assignment"
   • Gate maintains registry of MAC → IP mappings
   • Gate responds: "You're 10.41.3.1, serve clients .2-.254"

3. Configuration Phase

   • Node converts temp IP to assigned static
   • Configures local DHCP with assigned range
   • Restarts services, fully operational

Benefits

• Eliminates possibility of conflicts
• Central visibility (gate knows all nodes)
• Nodes can request different client capacities
• If node reboots, gets same IP back (based on MAC)

Allocation Strategy

Could be simple sequential:

First node:  10.41.2.1 → clients 10.41.2.2-254
Second node: 10.41.3.1 → clients 10.41.3.2-254
Third node:  10.41.4.1 → clients 10.41.4.2-254

Trade-off: Gate Dependency

However, this approach introduces a new dependency: nodes now need the gate to be online during initial provisioning. The current random assignment has an advantage here - nodes can join the mesh even when the gate is completely offline.

So if we go with API-based provisioning, we'd need a fallback mechanism to maintain that resilience.

Complementary Solution: Emergency Provisioning Mode

This would only be needed if the API-based approach is adopted. The idea is to preserve the current system's ability to add nodes without the gate, while still getting the benefits of centralized coordination when the gate is available.

Reserve Emergency Range

Set aside something like 10.41.250-254.* for emergency operation (5 subnets, max 5 emergency nodes).

How It Works

When a node can't reach the gate API:

1. Scan for other emergency nodes

   • Quick ping of 10.41.250.1, 251.1, 252.1, 253.1, 254.1
   • Only checking .1 addresses (where nodes would be)

2. Grab next available slot

   • If 250.1 exists but 251.1 doesn't → take 251.1
   • Self-configure, start serving clients immediately

3. Run emergency API

   • Each emergency node can help later arrivals
   • Acts as proxy: "Let me check if gate is back..."
   • New nodes always try gate first (through emergency node proxy)
   • If gate is still down, new node joins emergency pool

4. Automatic recovery

   • Emergency nodes periodically check for gate (every 30 sec)
   • When gate returns, request permanent assignment from gate API
   • Gate responds: "You're now 10.41.5.1 permanently"
   • Node transitions from emergency IP to permanent
   • Clients experience brief reconnect
   • Emergency slot freed up for future use

Why This Combination Works

• Normal operation: Gate coordinates everything, zero conflicts
• Gate offline: Nodes can still join via emergency mode
• Gate returns: Automatic recovery and transition to permanent IPs
• Best of both worlds: Centralized coordination when possible, decentralized fallback when needed

Questions for the Community

1. Are IP conflicts actually happening? Maybe the random assignment works fine in practice and this is overengineering?

2. Is there existing conflict detection? Something I'm missing from the docs that already handles this?

3. Worth the complexity? The API approach adds moving parts. Is the current "good enough is good enough" approach more appropriate for tactical scenarios?

4. Real-world deployments? For folks who've deployed 10+ nodes, what's your experience? Any conflicts? How'd you handle them?

5. Alternative approaches? Maybe there's a simpler way to prevent conflicts without the full API infrastructure?

I'm trying to figure out if these are edge cases worth addressing or if the current design philosophy (simple, stateless, accept occasional conflicts) is actually the right call for this use case.

Thanks for any insights!

[coreywagehoft]

Answers
IP Conflicts are not happening. We used to do "random" assignments, but now the nodes communicate on initial start up. Assuming your node is within range of others on your mesh, the new node will send out a request, and all other nodes reply with their current address and DHCP assignment range. The new node then will pick a static IP and DHCP range that does not overlap or conflict with it.

This is all done through our binary running on every node called openmanetd. You can find the source code here if interested: https://github.com/OpenMANET/openmanetd

This new system is still being developed and there are a few edge cases I will address later, but it does work.

[yash7158]

Got it, that makes sense.

I have another question about a potential edge case:

Let's say a new node joins the mesh and sends out its request for available addresses. But at that exact moment, one of the existing nodes is temporarily disconnected or unreachable (maybe it's rebooting, out of range temporarily, or having a radio issue).

So the new node collects responses from all the nodes it can hear, but it doesn't get a response from that disconnected node. Based on the incomplete information, the new node picks an IP address and DHCP range that it thinks is free.

Then later, the previously disconnected node comes back online - and now both nodes have the same or overlapping addresses.

What happens in that situation? Does openmanetd have conflict detection to catch this? Like:

Does the returning node detect the duplicate and reconfigure itself?
Does the new node realize there's a conflict and pick a different address?
Do they negotiate somehow to resolve it?
Or is this one of the edge cases you mentioned that still needs to be addressed?

[coreywagehoft]

This is one of the edge cases that needs to be addressed. I have some ideas how to handle this, but just need to get to it.

[yash7158]

Makes sense, thanks for confirming!

I'd be more than happy to help with the networking side if you need an extra hand working through those edge cases. Feel free to reach out if you want to discuss potential approaches or need someone to test scenarios.

Thanks for the work on this project!