neigh_table_tuner: tune all thresholds and limit growth

[Hannibal404]

increase all thresholds simultaneously instead of only gc_thresh3. Also set a limit to how large the thresholds can grow

[alan-maguire]

So by moving gc_thresh1 we are saying we want to increase the threshold before we run garbage collection. I would question if that's the right answer. Let's say our neighbor table is filling up; while we have granted more space with gc_thresh3 to prevent drops, do we want to wait longer until we start doing garbage collection? In a sense I would say it might make sense to move gc_thresh2 down towards gc_thresh1 since that will mean we do more aggressive GC in cases where the table fills up.

Now the case of hard maximums is tricky. I don't know that we can really pick a number here, but I would say something like this might make sense:

• while we approach full neigh table, increase gc thresh 3 if and only if gc thresh 2> gc thresh1; also reduce gc thresh 2 towards gc thresh 1 since that means "do more aggressive gc"
• if we approach full neigh table and gc_thresh2 == gc_thresh1, we are in a situation where aggressive reclaim isn't helping. It may be a buggy situation, so stop doing gc thresh 3 increases.

Does the above make sense? What do you think about submitting something like that approach (documenting the change in the neigh table tuner doc)?