Parser of Vertex IDs should be of type 64-bit unsigned integer

[vaastav]

The way the vertex IDs are currently being parsed, they would overflow if the vertex IDs are greater than 4.2 Billion.

[frankmcsherry]

You are totally correct that the repo does not handle vertex identifiers that don't fit in 32 bits. The main reason is that there is a significant cost imposed by making the identifiers 64 bits for folks who don't need that (and, for most of the graphs that the COST paper targets, that isn't needed).

I think that if you have large identifiers, the to_vertex program will panic rather than overflow. It does not appear to give you an especially helpful error message when it does so, but it shouldn't just silently overflow and produce incorrect results (like the stats program does).

It's reasonable to ask "should there be a 64 bit version of the COST code", and it's reasonable to want that, but at the moment it is more complicated than just switching some u32 types to u64. For example, the Hilbert layout stuff requires that two u32 values can be packed together into one value, and u64 works great for this; this would need to become something like u128, and the logic updated to reflect the greater number of bytes. The file formats would need to be enriched to explain the types they use (to avoid costing for the smaller identifiers), stuff like that.

Anyhow, I'm inclined to i. double check that the repo isn't producing incorrect results, and ii. probably introduce better error messages warning folks about what they should do if we see values that exceed u32 identifiers. But as the COST repo is mostly a tech demo rather than a "library spanning broad use cases" I'm not inclined to actually generalize it (myself, at least). I'm happy to leave the issue open as a warning to others (who might not realize that these restrictions exist).

If it is something you need (vs just something you've noticed) it could be a good exercise to work through the changes yourself in a fork, as they probably aren't deeply difficult.

[vaastav]

Thanks for the great responses!
I haven't tried running any of the binaries with > 4.2B vertices and I don't think I plan to anytime soon. The sole goal for me was to bring this to your attention. I completely agree with what you have said and I think you are correct in leaving it as is.
Thanks once again!