Commit a990107
[WFRunner] Estimate resources dynamically
* introduce functionality to update resource estimates based on values
which are collected during monitoring
(the same that go into pipeline_metric_<pid>.log)
* disentangle resource management from WorkflowExecutor
* manage resources via ResourceManager object
* memory and cpu
* semaphores
* number of parallel tasks to be launched
* niceness
* main interfaces of ResourceManager
* ok_to_submit
* book
* unbook
* add_monitored_resources
* group corresponding tasks together to be able to update resource
estimates for upcoming tasks
* TaskResources as container of a task's resource estimate
holding
* resource estimates (initial and new estimates)
* references to Semaphore object (optional
* reference to list of related tasks (optional)
* make sure, final niceness assigned to a launched process is the same
that the TaskResources hold for that process
* enable functionality with --dynamic-resources
* add more comments to classes and functions
* update --cgroup option to require entire path
--> handle both cgroups v1 and cgroups v2
(cherry picked from commit ef0e84d)1 parent 547ffd0 commit a990107
1 file changed
+541
-231
lines changed
0 commit comments