Retrieving the matched rule fields

[danrpts]

What is your idea?

Is there currently any way to retrieve the matched rule fields? For example, given the following event and rule, can event-ruler return me the list: ['detail-type', 'resources', 'detail.state']?

{
  "version": "0",
  "id": "ddddd4-aaaa-7777-4444-345dd43cc333",
  "detail-type": "EC2 Instance State-change Notification",
  "source": "aws.ec2",
  "account": "012345679012",
  "time": "2017-10-02T16:24:49Z",
  "region": "us-east-1",
  "resources": [
    "arn:aws:ec2:us-east-1:123456789012:instance/i-000000aaaaaa00000"
  ],
  "detail": {
    "c-count": 5,
    "d-count": 3,
    "x-limit": 301.8,
    "source-ip": "10.0.0.33",
    "instance-id": "i-000000aaaaaa00000",
    "state": "running"
  }
}

{
  "detail-type": [ "EC2 Instance State-change Notification" ],
  "resources": [ "arn:aws:ec2:us-east-1:123456789012:instance/i-000000aaaaaa00000" ],
  "detail": {
    "state": [ "initializing", "running" ]
  }
}

Would you be willing to make the change?

Yes

Additional context

Consider also how it'd work for an or group. Only one of the list sets would be returned

Add any other context (such as images, docs, posts) about the idea here.

[baldawar]

hey @danrpts this isn't possible today but I'm unclear why this is needed when you can fetch this from the rule itself. Could you share more details on why this will be helpful to add as a new functionality?

[danrpts]

hey @baldawar, the difference between using the rule itself and the matched attributes is more clear in an example using $or since the resulting lists would be dependent on what or group was hit (in retrospect I should have supplied this example):

{
    "source": [ "aws.cloudwatch" ],
    "$or": [
        { "metricType": [ "MetricType" ] } , 
        { "namespace": [ "AWS/EC2", "AWS/ES" ] }
    ]
}

for the following event and above rule, the resulting list would be ['source', 'metricType']

{
  "source": "aws.cloudwatch",
  "metricType": "MetricType",
   // ... rest of event
}

for this other event and above rule, the resulting list would be ['source', 'namespace']

{
  "source": "aws.cloudwatch",
  "namespace": "AWS/EC2"
   // ... rest of event
}

aside: my use case is to generate an idempotency token for the event based on the matched attributes, so I was looking for a way to extract the matched keys only

[baldawar]

Oh I see. That makes sense though I'm a big unclear on why you'd want to combine multiple rules into one when Ruler supports a ton of rules at the same time. You could have this and it'll work better (and be less complicated to make sense of)

{
  "source": [ "aws.cloudwatch" ],
  "metricType": [ "MetricType" ],
}

and

{
  "source": [ "aws.cloudwatch" ],
  "namespace": [ "AWS/EC2" ]
}

I've mostly seen $or as helpful way to check for different combinations of the same set of fields. Nothing wrong with the rule you shared above (I fixed a typo in it); but its a use-case I don't see often enough to make changes.

I do see a path to help gain more visibility into which all paths were compared, and where ruler found a mismatch but it'll come with memory & perf impact; so I'm hesitant on turning on outside of a new debugging-specific API.

Let me know if you'd like to add this in. I'll be happy to dig into the code more and come back with the effort involved. Otherwise its probably best to close this until someone can volunteer to work on the this.

[danrpts]

Yeah I agree that would be a less complicated way to define a rule and would allow us to extract the key set cleanly off the definition. However, our rules are defined by our customers and they can do so with $or group config like I shared above (whether the fields are related or not).

I also agree this may not be a great addition until more users are requesting it. It would be great if you could provide some context, my team could put up a PR if called for.

If not added to the library, I was even thinking there may be a helper we could build for ourselves to run against a one-off machine with only the matching rule(s) added. Appreciate your help!

[baldawar]

I've backed up a very rough but functional solution in 6544dbc but I do not expect to push into mainline. As I was working on this today, I realized a more usable solution requires logging / metricing what ruler is doing under the cover (flattening events, processing a specific field, matching a specific part) as it helps with debugging as user and also when developing new features (or just learning how ruler works).

I'm hesitant on adding a new logger framework as dependency as it can conflict with what consumers use, so we might have to go towards the route of implementing a placeholder interface that consumers can implement based on their needs.