Rebuilding function docker image

[DrJohnDale]

Hi,

If I make a change to the function I have already dockerised with cloudknot and run the building again, it does not seem to change the erc stored docker image.

For example if my script is:

import cloudknot as ck

def my_cloudknot_function(my_intput):
    print("hello")
    .....

knot = ck.Knot(name='my_cloudknot',
               func=my_cloudknot_function,
               base_image="my_base_image",
               pars_policies=('AmazonS3FullAccess',)
               no_image_cache=True)

result_futures = knot.map(my_intputs_list)

I can run this and all goes as expected.

If I then change the print to say

    print("hello world")

and re-run the function in the docker image in ECR does not change, or at least the docker image which runs does not include the change

what I have to do is run

knot.clobber(clobber_pars=True, clobber_repo=False, clobber_image=True)

and then re-running the script updates the docker image and all runs as expected.

Is this expected behavior?
Is there a error in my script?

The clobbering and rebuilding takes time (I think this comes from the removal and building of the pars) If the function change was detected and the docker file updated it would reduce the development loop time.

[richford]

Hi @DrJohnDale, thanks for starting the discussion.

This is not exactly expected behavior, but we haven't gotten around to fixing it just yet. I believe it's related to #257. For now, there are two workarounds:

1. Give the Knot a different name (i.e. by incrementing some suffix index) every time you edit your underlying function.
2. Use the DockerImage object (as in this example). DockerImage does not suffer from the same flaws as Knot.

[richford]

Can you let me know if either of those work out for you?

[DrJohnDale]

Hi

Thanks for the help.

The DockerImage approach seems to work, but I get several warnings.
Script

import cloudknot as ck


def hello_name(name: str):
    print("hello welcome to cloudknot " + name)
    return 0


docker_im = ck.DockerImage(name='hello-name', func=hello_name, overwrite=True)

# Create a Knot
knot = ck.Knot(name='hello-name', func=hello_name, docker_image=docker_im, no_image_cache=True)

# Submit the jobs
result_futures = knot.map(["John"])

warnings

WARNING:cloudknot.dockerimage:Found docker-image hello-name in your config file but the input parameters have changed. The updated parameters are ['func']. Continuing with the new input parameters and disregarding any old, potentially conflicting ones.

WARNING:cloudknot.dockerimage:Warning, your Dockerfile will have a base image of python:3, which may default to Python 3.8. This may cause dependency conflicts. If this build fails, consider rerunning with the `base_image='python:3.7' parameter.

WARNING:cloudknot.cloudknot:You specified configuration arguments for a knot that already exists. Cloudknot has returned the pre-existing knot, ignoring all of your other input parameters, which may or may not be the same. You should proceed with caution and confirm that this knot's parameters are as expected. If you want to be extra-safe, choose a different name or clobber this pre-existing knot and instantiate a new one with your input arguments.

[richford]

Don't forget to do

docker_im.build()
repo = ck.aws.DockerRepo(name=ck.get_ecr_repo())
docker_im.push(repo=repo)

before passing docker_im as an input param to your Knot. Additionally, when you create your Knot, specifying both func and docker_image as input is redundant. In fact, this line should raise an error when you do this, but perhaps it's skipping it because you are recovering the pre-existing knot.

I would omit the func argument to Knot and specify only the docker_image.

As for the last Knot warning, whether this is okay depends a bit on the context. If the knot you are retrieving is the one from your initial post (i.e. without using DockerImage), then any changes you make to the function will not propagate through to the knot. However, if the pre-existing knot was created with a DockerImage input, then to update the function, you only need to build and push the DockerImage instance each time you change the function. That is, the knot can be left unchanged.

Once you build a knot using a DockerImage instance as input, I would recommend explicitly recovering the old knot by omitting all of the other input parameters except the name.

Hope that makes things clearer rather than muddier. Please let us know if you have any questions.

[arokem]

That last warning would worry me. I think that the current best practice is to give each new knot a different 'name' key-word argument, to make sure that you don't get over-ridden by previously-created knots.