mnp.preparation package

Submodules

class mnp.preparation.aggregate_land_type_map.LandTypeMap(land_type_map_path: str, land_type_map_table_path: str, environmentals_folder: str)

Bases: object

Class for holding a land type map.

Parameters:

  • land_type_map_path (path to the geospatial tiff with the land types) –

  • land_type_map_table_path (path to *.csv file with legend of land type map) –

aggregate()

Aggregate 2.5m management map to 25m, as a dictionary of ManagementType objects that contain the areas of said type in each 25m cell.

More generic: aggregate with aggregation_factor 10.

Returns:

management_map_aggregated – Aggregated management map as a dictionary of ManagementTypes.

Return type:

dict

aggregate_multiprocess()

Use multiple process to aggregate lt map in parallel and speed up processing.

make_geospatial_profile(environmentals_folder: str)

Determine the geospatial profile of all subsequent geospatial output.

Return type:

json written to the cover

save(output_folder: str)

Save this land type map to output folder

Parameters:

output_folder –

save_geospatial_profile(output_folder)

Pickle geospatial profile.

Parameters:

output_folder (str) – Folder to dump pickle

mnp.preparation.aggregate_land_type_map.aggregate_land_type_map(folders: dict)

Task for aggregating land type map from 2.5 to 25 meter cell size

Parameters:

folders –

Returns:

land_type_map.land_types – dictionary of aggregated land types in sparse array format

Return type:

dict

mnp.preparation.aggregate_land_type_map.aggregate_row(row_array: ~numpy.array, row_offset: int, land_types: dict, unrecognized_land_type_ids: set, aggregation_factor: int, nodata_array: ~numpy.array) -> (dict[int, scipy.sparse._dok.dok_array], <class 'set'>)

Function that aggregates a row. Is shared between regular and multiprocess. Because of the quirks of multiprocessing casn only take constants from self, input variables that change have to be passed as arguments and output variables have to be returned.

Parameters:

  • row_array (np.array) – Array containing one aggregated row, which using default settings means 10 unaggregated rows, read from unaggregated raster

  • window (Window) – Rasterio read windows that has been used to read unaggregated rows

  • land_types (dict) – Dictionary of land type sparse arrays

  • unrecognized_land_type_ids (set) – Set containing ids for land type values that are in array but not in table.

Returns:

  • land_types (dict) – The updated land types dictionary

  • unrecognized_land_type_ids (set) – the updated unrecognized land type ids set

mnp.preparation.aggregate_land_type_map.read_aggregated_map(folders: dict) → dict[str, csr_array]

Read aggregated land types map

mnp.preparation.filter_incomplete_cells.filter_incomplete_cells(arrays: dict, spatial_data_coverage_path: str) → dict

Apply spatial data coverage mask, to filter out incomplete cells.

Parameters:

arrays (dict) – Dictionary of numpy arrays

mnp.preparation.input_verification_functions.csv_is_readable(file_path: str) → bool
mnp.preparation.input_verification_functions.directory_contains(directory_path: str, pattern: str, n) -> (<class 'bool'>, <class 'str'>)

Directory contains n or more files matching a glob patter :param directory_path: :type directory_path: target directory path :param patter: :type patter: string pattern to match for :param n: :type n: minimal nr of files

mnp.preparation.input_verification_functions.is_directory(directory_path: str) -> (<class 'bool'>, <class 'str'>)

Is something a directory? :param directory_path: :type directory_path: target directory path

mnp.preparation.input_verification_functions.is_geospatial_raster(file_path: str) -> (<class 'bool'>, <class 'str'>)

Is a file a gdal compatible geospatial raster? :param file_pathL target file path:

mnp.preparation.input_verification_functions.is_valid_csv(file_path: str, table_type: str) -> (<class 'bool'>, <class 'str'>)

Determine if a file is a valid CSV with contents and corret header :param file_path: :type file_path: path to file :param table_type: :type table_type: table type to deterime required headers

mnp.preparation.input_verification_functions.occurs_in_formula(name: str, species_traits: str)

Check if name for environmental variable occurs in at least one species environmental variable formula.

Parameters:

  • name (str) – name for this environmental variable

  • species_traits (str) – path to species traits csv file

Returns:

  • out (bool) – boolean indicating status

  • msg (str) – message specifying status

mnp.preparation.input_verification_functions.path_exists(path)
mnp.preparation.input_verification_functions.raster_has_resolution(file_path: str, target_resolution) -> (<class 'bool'>, <class 'str'>)

Check if a raster has target resolution :param file_path: :type file_path: target file path :param target_resolution: :type target_resolution: expected resolution of the raster

mnp.preparation.input_verification_functions.rasters_have_identical_bounds(rasters: list) -> (<class 'bool'>, <class 'str'>)

Verify that all rasters have identical bound :param rasters: :type rasters: list with one or more paths to raster files on disk

mnp.preparation.input_verification_functions.verify_local_distance(species_traits: str, target_resolution: int)

Verify that all species local distances are conmensurable with 25m raster resolution :param species_traits: :type species_traits: path species traits table :param target_resolution: :type target_resolution: raster resolution

mnp.preparation.input_verification_functions.verify_parameters(configuration: ~configparser.ConfigParser) -> (<class 'bool'>, <class 'str'>)

Verify parameters in user generated ini file comply to expectatations

Parameters:

configuration (output from read_ini) –

Return type:

boolean, msg

class mnp.preparation.io_pathways.ErrorMessage(message: str)

Bases: VerificationProcedure

class mnp.preparation.io_pathways.InputPathway

Bases: object

Class to examine the user provided input in the configuration file and to compile a list of required verification procedures. Which procedures are required, depends on the user input.

A specific combination of user input is called an Input Pathway. The currently supported pathways are: 1. land_types as a *.tif, 1 or more environmental_factors 2. land_types as a *.tif, no environmental_factors 3. directory with one or more pre-aggregated land_types, 1 or more environmental_factors 4. directory with one or more pre-aggregated land_types, no environmental_factors 5. No land type data, no environmental_factors, one or more HSI rasters in directory

identify(configuration: ConfigParser)

Examine the contents of the configuration file and add verification procedures as needed. Verification procedures are run on instantiation.

log_messages()
verify()

Update and check status of all input verification procedures. Output all messages to log. Every time an error is detected in the user input (for example, a file does not exist, or does not have the expected header) a prefect ERROR is raised. If one or more errors where detected, this function raises a RuntimeError to stop flow execution.

class mnp.preparation.io_pathways.OutputPathway(configuration: ConfigParser)

Bases: object

class mnp.preparation.io_pathways.VerificationProcedure

Bases: ABC

update_status()
class mnp.preparation.io_pathways.VerifyEnvironmentalTable(configuration: ConfigParser, environmental_section: str)

Bases: VerifyTable

Procedure to verify that user provided:

environmental_factor raster is a valid geospatial file environmental_factor response table is a valid CSV file

class mnp.preparation.io_pathways.VerifyHSIFromDisk(configuration: ConfigParser)

Bases: VerificationProcedure

Procedure to verify that user provided HSI directory exists and contains at least 1 Tif file. Tif file must be formatted with species code: S01234567.tif!

class mnp.preparation.io_pathways.VerifyHasEnvironmentalFactors(has_environmentals: bool)

Bases: VerificationProcedure

class mnp.preparation.io_pathways.VerifyIsGeospatialRasterAndResolution(raster_path: str, target_resolution: int)

Bases: VerificationProcedure

Procedure to check if file is a geospatial raster and if so check its resolution.

Parameters:

  • raster_path (str) – Path to raster file

  • target_resolution – Resolution that the raster should have

class mnp.preparation.io_pathways.VerifyLandTypeDirectory(configuration: ConfigParser)

Bases: VerificationProcedure

Procedure to verify that user provided directory for pre-aggregated land_types exists and contains at least 2 NPZ files and the geospatial profile.

class mnp.preparation.io_pathways.VerifyMetadataComment(configuration: ConfigParser)

Bases: VerificationProcedure

Procedure to verify that a Metadata comment has been provided in the ini

class mnp.preparation.io_pathways.VerifyParameters(configuration: ConfigParser)

Bases: VerificationProcedure

Procedure to verify that the user provided parameters are within permitted ranges.

class mnp.preparation.io_pathways.VerifyRasterAndGeospatialProfileBounds(geospatial_profile_path: str, raster_path: str)

Bases: VerificationProcedure

Procedure to verify that the bounds of the first environmental factor raster match with the bounds of the Geospatial Profile, in case of pre-aggregated NPZ files.

class mnp.preparation.io_pathways.VerifyRasterBounds(*args: str)

Bases: VerificationProcedure

Procedure to verify that the bounds of two rasters match first environmental raster and the land_type raster match. Only the first environmental raster is checked as the environmental rasters are compared to each other. So if one matches, they all match. :param args: Paths for rasters to compare. Can be a list of paths or can be separate paths as function arguments. :type args: strings

class mnp.preparation.io_pathways.VerifySpeciesSubselection(configuration: ConfigParser)

Bases: VerificationProcedure

Procedure to verify that at least one species subselection is provided by the user and is not empty.

class mnp.preparation.io_pathways.VerifySpeciesTraits(configuration: ConfigParser, target_resolution: int)

Bases: VerificationProcedure

Procedure to verify that species local distances are commensurable with raster resolution. Assumes raster resolution = 25m

class mnp.preparation.io_pathways.VerifyTable(table_path: str, table_type: str)

Bases: VerificationProcedure

mnp.preparation.io_pathways.verify_user_input(configuration: ConfigParser)
Parameters:

configuration –

mnp.preparation.read_rasters_from_cover.read_rasters_from_cover(folder: str) → dict[slice(<class 'str'>, <class 'numpy.ndarray'>, None)]

Read the rasters from the given directory in the cover.

Parameters:

folder_key (str) – Key for a cover directory to read all rasters from.

Returns:

rasters – Dictionary with raster name : numpy array. Name is taken from file name.

Return type:

dict[str:np.ndarray]

mnp.preparation.select_valid_species.get_valid_species(hsi_from_disk: bool, parameters) → set

Inspect all user provided identifiers in the species subselection list(s). Verify that: 1. user provided identifier is in fact a species, cq reference species_names.csv 2. species has traits 3. species is matched to at least one land type 4. species is matched to at least one land type that is employed in the land type raster 4. species has a valid trait group 5. species has a response to each provided abiotic

Parameters:

cover (dictionary with cover design, output of of create_directories) –

Returns:

  • msg (<description of user provided identifiers that fail to meet criteria 1-5 OR success message>,)

  • species_codes (<set of species codes for all species that meet criteria 1-5>,)

mnp.preparation.select_valid_species.get_valid_species_selections(valid_species: set, parameters) → dict

Parse the user provided species selections to a dictionary with the sluggified selection name as keys and a list of valid only species codes as value. Any duplicates in the provided species lists are removed.

Parameters:

  • cover (dictionary with cover design, output of of create_directories) –

  • valid_species (output from get_valid_species) –

Returns:

dictionary {<sluggified subselection name> – <species_code_02>, <species_code_n>]}

Return type:

[<species_code_01>,

mnp.preparation.select_valid_species.melt_boolean_dataframe(df: DataFrame) → str

Melts a pandas dataframe and retains rows where the value != True. Parses a message from the combination of the dataframe index, variable and value Parameters ———- df: Pandas dataframe with

string separated by newlines

Input pandas DataFrame:

is_species has_traits

generic_identifier Carex digitata True True Vingerzegge True False S02000950 True True foo False False Moeslook True True

Output string:

‘Vingerzegge failed test “has_traits”

foo failed test “is_species” foo failed test “has_traits”’

class species database carrying all known species in the MNP environment

class mnp.preparation.species_database.SpeciesDataBase(species_names: DataFrame)

Bases: object

Read and store the master species list, plus several helper dictionaries

any2any(identifier: str, of: str) → str

Generic mapping between species codes, local names, scientific names

Parameters:

  • identifier (species code, local name or scientific name or anything else) –

  • of (output format, chose from {code, local, scientific}) –

Return type:

species code, local name or scientific name

is_species(identifier: str) → bool

Determine if a identifier string is species code, local name or scientific name in the master species table :param identifier: :param Returns bool: :param ——-:

mnp.preparation.verify_spatial_data_coverage.save_spatial_mask(array: array, folders: dict)
mnp.preparation.verify_spatial_data_coverage.verify_spatial_data_coverage(land_types: dict, environmentals: dict, parameters: MNPParameters)

Verify spatial congruence of landtype and abiotic maps. Determine which 25m cells are valid for use. Only if the land type AND all participating abiotic rasters have valid data on 25m cell, is the cell approved for use

Parameters:

cover (valid MNP cover) –

Return type:

Geotiff in the cover showing identifying with complete spatial data coverage