Improve retry logic for Bulk Requests #1905
Description
When trying to push Elasticsearch to extremes, some requests may not succeed due to e.g circuit breakers triggering 429s amongst other things.
To ensure that we ingest all documents that we planned to, we need to look at improving the robustness of the BulkIndex action.
Currently rally relies on the elasitcsearch client for retries - as such most runners don't retry - only certain runners that are not normally really part of the benchmark, but rather just setting up configuration / waiting for cluster to be healthy etc.
One approach is #1868 this is aimed at document level 429s - where a single request may return as 200, but certain documents maybe failed to ingest due to CBE or similar.
Where we have an entire request that fails, we could look at wrapping Bulk in Retry - https://github.com/elastic/rally/blob/master/esrally/driver/runner.py#L2953