Extending GaNDLF
Environment¶
Before starting to work on the code-level on GaNDLF, please follow the instructions to install GaNDLF from sources. Once that's done, please verify the installation using the following command:
# continue from previous shell
(venv_gandlf) $>
# you should be in the "GaNDLF" git repo
(venv_gandlf) $> python ./gandlf_verifyInstall
Submodule flowcharts¶
- The following flowcharts are intended to provide a high-level overview of the different submodules in GaNDLF.
- Navigate to the
README.md
file in each submodule folder for details.
Overall Architecture¶
- Command-line parsing: gandlf_run
- Parameters from training configuration get passed as a
dict
via the config manager - Training Manager:
- Handles k-fold training
- Main entry point from CLI
- Training Function:
- Performs actual training
- Inference Manager:
- Handles inference functionality
- Main entry point from CLI
- Inference Function:
- Performs actual inference
Dependency Management¶
To update/change/add a dependency in setup, please ensure at least the following conditions are met:
- The package is being actively maintained.
- The new dependency is being testing against the minimum python version supported by GaNDLF (see the
python_requires
variable in setup). - It does not clash with any existing dependencies.
Adding Models¶
- For details, please see README for
GANDLF.models
submodule. - Update Tests
Adding Augmentation Transformations¶
- Update or add dependency in setup, if appropriate.
- Add transformation to
global_augs_dict
, defined inGANDLF/data/augmentation/__init__.py
- Ensure probability is used as input; probability is not used for any preprocessing operations
- For details, please see README for
GANDLF.data.augmentation
submodule. - Update Tests
Adding Preprocessing functionality¶
- Update or add dependency in setup, if appropriate; see section on Dependency Management for details.
- All transforms should be defined by inheriting from
torchio.transforms.intensity_transform.IntensityTransform
. For example, please see the threshold/clip functionality in theGANDLF/data/preprocessing/threshold_and_clip.py
file. - Define each option in the configuration file under the correct key (again, see threshold/clip as examples)
- Add transformation to
global_preprocessing_dict
, defined inGANDLF/data/preprocessing/__init__.py
- For details, please see README for
GANDLF.data.preprocessing
submodule. - Update Tests
Adding Training Functionality¶
- Update Training Function
- Update Training Manager, if any training API has changed
- Update Tests
Adding Inference Functionality¶
- Update Inference Function
- Update Inference Manager, if any inference API has changed
- Update Tests
Update parameters¶
For any new feature, please ensure the corresponding option in the sample configuration is added, so that others can review/use/extend it as needed.
Update Tests¶
Once you have made changes to functionality, it is imperative that the unit tests be updated to cover the new code. Please see the full testing suite for details and examples.
Run Tests¶
Prerequisites¶
There are two types of tests: unit tests for GaNDLF code, which tests the functionality, and integration tests for deploying and running mlcubes. Some additional steps are required for running tests:
- Ensure that the install optional dependencies [ref] have been installed.
- Tests are using sample data, which gets downloaded and prepared automatically when you run unit tests. Prepared data is stored at
${GaNDLF_root_dir}/testing/data/
folder. However, you may want to download & explore data by yourself.
Unit tests¶
Once you have the virtual environment set up, tests can be run using the following command:
# continue from previous shell
(venv_gandlf) $> pytest --device cuda # can be cuda or cpu, defaults to cpu
Any failures will be reported in the file ${GaNDLF_HOME}/testing/failures.log
.
Integration tests¶
All integration tests are combined to one shell script:
Code coverage¶
The code coverage for the unit tests can be obtained by the following command: