Add benchmark tests for machine creation latency

[jonessha]

What is your idea?

Add benchmark tests for machine creation latency

Would you be willing to make the change?

Maybe

Additional context

#104

[timbray]

Benchmarks.java already has a few benchmarks that pound away at addRule() - they measure various things, might be good enough to add latency measurement to an existing one.

[baldawar]

Thanks. Found this

event-ruler/src/test/software/amazon/event/ruler/Benchmarks.java

Line 394 in b223ae5

public void ruleMemoryBenchmark() throws Exception {

Its just missing couple of lines to track total-time or number of machines built per sec.

[timbray]

Those benchmarks are generally lousy for their declared purpose, back when I was doing that there was really no good way to track Java memory usage and relate it to code. Maybe the tooling is better now? But yeah, might give useful latency numbers.

[baldawar]

JMH is the standard solution though we need to break apart the benchmark into smaller files with tests. Last time I gave it a try, the numbers were had more than 5% of variance compared to our current tests. Not sure if that's just a quirk of benchmarks tests or because of the changes we've made. I don't think there's any big changes coming up soon, so I might give adding JMH again a try once I've wrapped up the first draft for epml-spec/draft-ietf-patterns-protocol#1