RICorDE produces gridded water depth estimates from flood inundation data by incorporating a HAND sub-model and cost distancing algorithms to extrapolate edge values into the inundated region.
RICorDE is an open source software originally released under the Government of Canada’s Open Government License. RICorDE was co-developed by Bryant, McGrath and Boudreault. Full description of the methodology can be found here: Bryant, S., McGrath, H., and Boudreault, M.: Gridded flood depth estimates from satellite-derived inundations, Nat. Hazards Earth Syst. Sci., 22, 1437–1450, https://doi.org/10.5194/nhess-22-1437-2022, 2022.
To read more about the algorithm, and its performance and applications, see Bryant 2022.
To see what's new, check out the CHANGELOG.md
tldr: clone, create your own definitions.py file, run from pyqgis command line
RICorDE is a standalone application built on open source QGIS python bindings and WhiteboxTools. Before using, ensure the following are installed:
- QGIS 3.22.8
- WhiteboxTools Open Core v2.1.0. (this should install a whitebox_tools.exe file you'll reference in ./definitions.py)
RICorDE must be run within a working pyqgis environment. To test if your environment is working, try the following:
~python
>>>import qgis.core
>>>import processing
If any of these result in an error, your environment is not set up correctly.
The tutorials assume your pyqgis environment is activated by calling ./dev/activate_py.bat within windows cmd. This is not strictly necessary, but makes scripting model runs much easier. See ./CONTRIBUTING.md for some advice on setting up such a batch script and a virtual pyqgis environment.
- clone this repo
git clone https://github.com/NRCan/RICorDE.git - create a
./definitions.pyfile as shown below modified to your system - add the source folder to your PYTHONPATH (see
./dev/activate_py.bat)
Now you're ready to activate your python environment and use the Command Line Interface (see below)!
see ./tests/README.md for testing
example ./definitions.py:
import os
#location of whitebox executable
"""
Change this to match your whitebox exe path
"""
whitebox_exe_d = {'v2.1.0':r'C:\LS\06_SOFT\whitebox\v2.1.0\whitebox_tools.exe'}
#maximum processors to use
max_procs = 4
#location of source code
proj_dir = os.path.dirname(os.path.abspath(__file__))
#path to python logging config file
logcfg_file=os.path.join(proj_dir, 'logger.conf')
#root directory for building default directories in
root_dir=r'C:\LS\10_IO\ricorde'
RICorDE routines can be run in sequence using the command-line parsing in main.py or by calling the session methods in ricorde/scripts.py directly in custom python scripts.
An end-to-end run of the RICorDE algorithm is provided through the CLI by specifying a parameter.ini file:
~python main.py path/to/parameter.ini
Additional arguments can be passed to control RICorDE's file behaviour and some defaults. Information on these controls can be obtained through the help command:
~python main.py -h
usage: RICorDE [-h] [-exit_summary] [-compress {hiT,hi,med,none}]
[-root_dir ROOT_DIR] [-out_dir OUT_DIR] [-temp_dir TEMP_DIR]
[-tag TAG] [-write] [-prec PREC] [-overwrite]
[-relative]
param_fp
Compute a depths grid from flood inundation and DEM
positional arguments:
param_fp filepath to parameter .txt file (see documentation for
format)
optional arguments:
-h, --help show this help message and exit
-compress {hiT,hi,med,none}, -c {hiT,hi,med,none}
set the default raster compression level
-root_dir ROOT_DIR, -rd ROOT_DIR
Base directory of the project. Used for generating
default directories. Defaults to value in definitions
-tag TAG, -t TAG tag for the run
A RICorDE parameter file is a pythonic ini file where the input data and algorithm parameters are specified. The parameter file expects 20 sections, each of which corresponds to an intermediate or end result file, except for the [session] section where the primary inputs are specified. Here's the first 10 rows from RICorDE_params_default.ini
[session]
name=project1 #project name
aoi_fp = #optional area of interest polygon filepath
dem_fp= #dem rlay filepath
pwb_fp= #permanent water body filepath (raster or polygon)
inun_fp= #inundation filepath (raster or polygon)
crsid=EPSG:4326 #CRSID
[dem]
#resolution =10 #optional resolution for resampling the DEM
Characters following '#' are ignored.
To prepare a RICorDE run, first copy the provided RICorDE_params_default.ini into your working directory, then edit as needed, before saving and executing using main.py (see above).
Once your parameter.ini file is prepared and you've decided on your run arguments, prepare your python call as shown above. For example:
python main.py -rd path/to/my/work -t r0 path/to/parameter.ini
then execute this in your pyqgis environment. On Windows, this is typically accomplished via a batch script which performs the environment setup then makes the RICorDE call. An example of such a batch script is provided in the tutorials folder (be sure to edit this with your own paths). Once you're confident the run is configured correctly, python's '-O' flag can be passed to remove some checks.
For more flexibility, RICorDE methods can be called in custom python scripts by referencing the session methods in ricorde/scripts.py directly. The function run_from_params provides a nice example (this is the default behaviour of the CLI call) and calls the following hi-level function sequence:
run_dataPrep() #Clean and load inputs into memory.
run_HAND() #Build the HAND raster from the DEM using whiteboxtools
run_imax() #Perform the Phase 1: Inundation Correction
run_HANDgrid() #Perform PHASE2: Compute Rolling HAND grid
run_wslRoll() #Perform PHASE3: Rolling WSL grid
run_depths() #PHASE4: Resultant Depths computation
When developing your custom script, parameters from the parameter.ini file should be passed to the session as a dictionary in the bk_lib key word argument (these can be loaded from the load_params function if you'd like to still use the parameter.ini file).
A typical end-to-end RICorDE run generates ~20 intermediate result data files. This architecture is useful for debugging, resuming a partial run, or for quickly replacing the results of some internal sub-routine (e.g., supplying your own HAND layer rather than using the one computed by RICorDE/WBT). Filepaths to an alternate intermediate (pre-compiled) result data file can be passed to a 'Session' object (e.g., via 'run_from_params') with the 'compiled_fp_d' key. For example:
run_from_params(
param_fp='path_to_parameters',
#use this hand raster file instead
compiled_fp_d = {'hand_fp':'path_to_layer'},
)
This will tell RICorDE to use the raster specified rather than call the building function (build_hand in this case). Most the building and (pre-compiled) loading functions are specified here. Using this feature to supply an intermediate data file computed outside of RICorDE is not recommended as this may cause downstream issues in the computation pipeline.
A pre-configured run of the Fredericton 2018 flood is provided in the tutorials folder. See Bryant 2022 for data sources. Edit the batch file to reflect your own paths. A short video showing the execution of this tutorial can be found here.
For each run, RICorDE creates four log files [in a directory]:
- root.log: This is the debug log where all messages from all runs are stored [root_dir].
- rootwarn.log: Same as root.log, but only warnings and errors are stored [root_dir].
- Qproj.log: Same as root.log, but only QGIS feedback messages are stored [root_dir].
- tag_datetime.log: This log stores all messages for an individual run and is used to document the results [out_dir]
Good luck! and thank you for your interest in the project. We'd love to hear from you, so please post an issue with suggestions or questions.