-
Notifications
You must be signed in to change notification settings - Fork 11
Handle rasters without a defined NoData value in align_and_resize_raster_stack
#477
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -1102,6 +1102,44 @@ def align_and_resize_raster_stack( | |
| n_pixels * align_pixel_size[index] + | ||
| align_bounding_box[index]) | ||
|
|
||
| # Set a nodata value if the raster does not have one defined. This | ||
| # ensures that a raster that is smaller than the target extent will | ||
| # not get filled with 0s that could be confused with real data. | ||
| # We do this by creating a temporary VRT with an explicit nodata value | ||
| # for rasters that are missing one. | ||
| fixed_base_raster_path_list = list(base_raster_path_list) | ||
| temp_working_dir_nodata = None | ||
| for index, (raster_info, base_path) in enumerate( | ||
| zip(raster_info_list, base_raster_path_list)): | ||
| nodata_list = raster_info['nodata'] | ||
| if None in nodata_list: | ||
| if temp_working_dir_nodata is None: | ||
| temp_working_dir_nodata = tempfile.mkdtemp(dir=working_dir, | ||
| prefix='align-nodata-') | ||
| raster = gdal.OpenEx(base_path, gdal.OF_RASTER) | ||
| chosen_nodata = choose_nodata(raster_info['numpy_type']) | ||
|
|
||
| if set(nodata_list) != {None}: | ||
| LOGGER.warning( | ||
| "Input raster %s has some bands with a defined nodata " | ||
| "value and some bands without. Setting %s as the nodata " | ||
| "value for all bands in a temporary VRT before aligning " | ||
| "and resizing.", base_path, chosen_nodata) | ||
| else: | ||
| LOGGER.warning( | ||
| "Input raster %s has no defined nodata value. Setting " | ||
| "%s as nodata in a temporary VRT before aligning and " | ||
| "resizing.", base_path, chosen_nodata) | ||
|
|
||
| vrt_path = os.path.join(temp_working_dir_nodata, | ||
| f'input_with_nodata_{index}.vrt') | ||
| vrt = gdal.Translate(vrt_path, raster, format='VRT', | ||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I'm wondering if the creation of the VRT is necessary or if we could take advantage of I'm not sure it's a better approach than the VRT one other than it reduces a call to
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Related to my comment above, I've tried using the |
||
| noData=chosen_nodata) | ||
| vrt = None | ||
| raster = None | ||
|
|
||
| fixed_base_raster_path_list[index] = vrt_path | ||
|
|
||
| if mask_options: | ||
| # Create a warped VRT. | ||
| # This allows us to cheaply figure out the dimensions, projection, etc. | ||
|
|
@@ -1110,7 +1148,7 @@ def align_and_resize_raster_stack( | |
| temp_working_dir = tempfile.mkdtemp(dir=working_dir, prefix='mask-') | ||
| warped_vrt = os.path.join(temp_working_dir, 'warped.vrt') | ||
| warp_raster( | ||
| base_raster_path=base_raster_path_list[0], | ||
| base_raster_path=fixed_base_raster_path_list[0], | ||
| target_pixel_size=target_pixel_size, | ||
| target_raster_path=warped_vrt, | ||
| resample_method='near', | ||
|
|
@@ -1142,7 +1180,7 @@ def align_and_resize_raster_stack( | |
| else None)) | ||
|
|
||
| for index, (base_path, target_path, resample_method) in enumerate(zip( | ||
| base_raster_path_list, target_raster_path_list, | ||
| fixed_base_raster_path_list, target_raster_path_list, | ||
| resample_method_list)): | ||
| warp_raster( | ||
| base_path, target_pixel_size, target_path, resample_method, | ||
|
|
@@ -1160,6 +1198,8 @@ def align_and_resize_raster_stack( | |
|
|
||
| LOGGER.info("aligned all %d rasters.", n_rasters) | ||
|
|
||
| if temp_working_dir_nodata: | ||
| shutil.rmtree(temp_working_dir_nodata, ignore_errors=True) | ||
| if mask_options: | ||
| shutil.rmtree(temp_working_dir, ignore_errors=True) | ||
|
|
||
|
|
||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
From what I can tell, gdal does not create geotiffs that have different NoData values (even if you try to explicitly set different NoData values for different bands, it won't work), so I think this would be a very rare situation and therefore not super important or easy to handle gracefully (i.e., by setting NoData only for the bands that lack a defined NoData value, since gdal doesn't support this for tiffs). I also decided not to raise an error as a raster that has the pygeoprocessing-approved NoData value for its datatype (e.g.,
32767for anint16raster) would be fine.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think gdal handles different nodata values for different bands. See the "UNIFIED_SRC_NODATA" option from gdalwarpoptions.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Unfortunately that doesn't seem to work! Not sure why, because I can't seem to find documentation saying this doesn't work for geotiffs, but I do get this warning
Warning 1: multi_diffND.tif, band 2: Setting nodata to 255 on band 2, but band 1 has nodata at 9999. The TIFFTAG_GDAL_NODATA only support one value per dataset. This value of 255 will be used for all bands on re-opening