aws.hostedRecords collection creates too many records

[coryb]

This is related to issue #7.

There seems to be a problem with hostedRecords creating to many document revisions. I have found thousands of new documents with identical content as the previous revision (_diff also show no output). My hunch is that multiple records are colliding, perhaps multiple CNAME records acting as a RR dns entry? So at a minimum I think we cannot use the "name" attribute as the id, but looking at the resource I dont see any attribute that could be used as the resource id. We probably need to keep the recordSets together as a single resource.

-Cory

[ralph-tice]

Records are colliding on Type for sure (A records vs NS records vs SOA etc). I think we need to create a compound key on name+type, I'll try to take another look at this soon.

[coryb]

Yeah, we also have several cases of multiple records with the same name and same type (ie CNAME that are in round-robin dns) via route53. Not sure the best way to solve this. We could make up arbitrary ids, but that seems silly. Perhaps it makes sense to do what you originally had and just merge the data into the hostedZone record and just have large zone documents. That is not ideal, but fits logically with the data that exists. Alternately we can have the records in hostedRecords be compound records, where it would show all the records of any type in the set with the same name. like [{ "type":"A",...}, {"type": "NS", ....}, {"type":"CNAME",...}, {"type":"CNAME",...}]

[ralph-tice]

I revisited this issue after looking at the RDS changes and I'm leaning towards going back towards having large zone documents. Any objection?

[evannuil]

Sounds great. I have 400+ hosted zones and after two weeks running the aws.hostedRecords table is already more than 1 GB.

[coryb]

Yeah, sounds good to me too. Either merge them as you suggest or create a new collection for just the recordSets, in which case the primary key is the zone id and then there is just a large array of records in the document.
-Cory