Additional developer documentation?

[talister]

Hi folks, I'm most of the way through completing a feature branch to add ingestion of small bodies through the MPC Explorer API interface. I've a couple of questions on development workflow before generating the PR:

1. Is there any documentation on setting up the development environment for working on a fork feature request ? During the development process I needed to add some fixture data for tests and a new package but ran into issues. Normally what I would do is add the package (importlib_resources) to a requirements.txt and do a pip install -e . to make it editable. With the poetry (?) based tom_base this doesn't work; there isn't a requirements.txt file and running pip install -e . results in uninstalling the tomtoolkit-2.21.0 version and installing a 0.0.0 version, which then breaks the dependencies for things like tom_fink which need a tomtoolkit version >2.17.2 for example. Can some documentation be added for how to setup your development environment ?
2. As part of making unit tests for the new code, I needed to create a test data directory of static captured results from the web service to avoid hitting that service in the tests - is this the right thing to do ?
3. Related to 2. above, I made use of importlib_resources to allow the test to find the test datafile transparently but this requires adding a new dependency - what is the procedure for wanting or adding new dependencies ?
4. In the interim until the pull request is merged upstream and a new version of tom_base/tomtoolkit is released, how do I go about using the new code/service within my TOM (FOMO) for developing ?

[jchate6]

Hi @talister,
I don't think the answers to these questions are in the documentation at the moment, but I will attempt to answer them to my best ability here for now.

1. If you are working on a development branch of tomtoolkit (TOM_Base) that you want installed into your local TOM, you should be able to install all of the features that you want (tom_fink, etc) which will pull down a pypi version of tomtoolkit, and then, when those are installed, just use pip install -e path.to.your.dev.tomtoolkit and it will install the local version. This will give you an error about how the dependency resolver is unable to account for all packages, but this is fine for now, and the dependent packages should not be removed. You can confirm with a pip freeze.

2. That is certainly one way to run tests of code that hit public infrastructure. However, this creates a snapshot in time and doesn't capture failures when then the webservice changes its behavior. I'd recommend implementing canary tests that are run less frequently (very few weeks or so?) via github workflows that actually DO hit the web service to make sure everything is still behaving as expected on that end as well. The tomtoolkit has some examples of this, where we tag tests as 'canary' and they get run separately.

3. You can obviously test the new dependency locally by just doing a pip install. To capture it in tombase, you will want to do a poetry add in the development branch. This will update the poetry.lock and pyproject.toml files with the new updates. Committing these files with the other changes updates the project dependencies.

4. Until the new release is made, you can continue to include the local branch or even a github repo as a dependency for your project locally. However, If you are trying to test it while deployed, we will happily do an alpha release to pypi that you can pull down as normal.

Hope that helps!

[jchate6]

Add a version of this explanation to "Contributors" section of the docs.