add additional documentation for the with_overrides feature#1181
add additional documentation for the with_overrides feature#1181dado5688 wants to merge 8 commits intoflyteorg:masterfrom
Conversation
Signed-off-by: jill <gohd8731@gmail.com>
Signed-off-by: jill <gohd8731@gmail.com>
| requests=Resources(cpu="1", mem="200Mi"), | ||
| limits=Resources(cpu="2", mem="350Mi"), | ||
| ) | ||
| def run_tfjob() -> str: |
There was a problem hiding this comment.
This task shouldn't just return a string — it should actually showcase a TF operation.
There was a problem hiding this comment.
Could you please explain it? If I understand correctly, are you suggesting that I should providing a detailed description of the entire TensorFlow training code, as this might overshadow the importance of discussing the with_overrides method? or can I provide a link to this file that explains the TensorFlow processing, allowing us to focus on how to override the task_conf and how to run it? I want to emphasize that our primary focus here is on demonstrating the usage of overrides, or what my thought that is wrong?
There was a problem hiding this comment.
You're right. Could you then include this as part of a note but not as a code block? Not a code block because we aren't including a full-fledged code snippet.
| return run_tfjob().with_overrides(task_config=TfJob( | ||
| num_workers=num_workers, | ||
| num_ps_replicas=1, | ||
| num_chief_replicas=1)) |
There was a problem hiding this comment.
Please format the code using black and isort.
examples/productionizing/productionizing/customizing_resources.py
Outdated
Show resolved
Hide resolved
|
You shouldn't remove the existing example. Please update the doc by adding a new one to it. |
|
ok, I see! |
Signed-off-by: jill <gohd8731@gmail.com>
| limits=resources, | ||
| container_image=custom_image, | ||
| ) | ||
| def mnist_tensorflow_job(hyperparameters: Hyperparameters) -> training_outputs: |
There was a problem hiding this comment.
This should be a simpler task. Let's not make this complicated, and every task has to have a definition.
|
@dado5688, just added a comment. Is it possible for you to take a look at them and incorporate the changes? |
|
@samhita-alla sure! I still trying to write a simpler tf job for example. |
Signed-off-by: jill <gohd8731@gmail.com>
3 participants
TL;DR
Enable users to understand how
with_overridescan be used in dynamically update various task configurationsComplete description
Add a new example in
Using with_overridessectionTracking Issue
Fixes flyteorg/flyte#4067
Follow-up issue
NA