Practical Use of Computationally Frugal Model Analysis Methods
Abstract
Three challenges compromise the utility of mathematical models of groundwater and other environmental systems: (1) a dizzying array of model analysis methods and metrics make it difficult to compare evaluations of model adequacy, sensitivity, and uncertainty; (2) the high computational demands of many popular model analysis methods (requiring 1000's, 10,000 s, or more model runs) make them difficult to apply to complex models; and (3) many models are plagued by unrealistic nonlinearities arising from the numerical model formulation and implementation. This study proposes a strategy to address these challenges through a careful combination of model analysis and implementation methods. In this strategy, computationally frugal model analysis methods (often requiring a few dozen parallelizable model runs) play a major role, and computationally demanding methods are used for problems where (relatively) inexpensive diagnostics suggest the frugal methods are unreliable. We also argue in favor of detecting and, where possible, eliminating unrealistic model nonlinearities—this increases the realism of the model itself and facilitates the application of frugal methods. Literature examples are used to demonstrate the use of frugal methods and associated diagnostics. We suggest that the strategy proposed in this paper would allow the environmental sciences community to achieve greater transparency and falsifiability of environmental models, and obtain greater scientific insight from ongoing and future modeling efforts.
Citing Literature
Number of times cited according to CrossRef: 29
- Chen Yu, Deyong Hu, Xin Duan, Yani Zhang, Manqing Liu, Shasha Wang, Rainfall-runoff simulation and flood dynamic monitoring based on CHIRPS and MODIS-ET, International Journal of Remote Sensing, 10.1080/01431161.2020.1714779, 41, 11, (4206-4225), (2020).
- Marina R.L. Mautner, Laura Foglia, Graciela S. Herrera, Rosa Galán, Jonathan D. Herman, Urban growth and groundwater sustainability: evaluating spatially distributed recharge alternatives in the Mexico City Metropolitan Area, Journal of Hydrology, 10.1016/j.jhydrol.2020.124909, (124909), (2020).
- Ahmed S Elshall, Aida D Arik, Aly I El-Kadi, Suzanne Pierce, Ming Ye, Kimberly M Burnett, Christopher A Wada, Leah L Bremer, Gregory Chun, Groundwater sustainability: a review of the interactions between science and policy, Environmental Research Letters, 10.1088/1748-9326/ab8e8c, 15, 9, (093004), (2020).
- Dmitri Kavetski, Parameter Estimation and Predictive Uncertainty Quantification in Hydrological Modelling, Handbook of Hydrometeorological Ensemble Forecasting, 10.1007/978-3-642-39925-1, (481-522), (2019).
- Thorsten Wagener, Francesca Pianosi, What has Global Sensitivity Analysis ever done for us? A systematic review to support scientific advancement and to inform policy-making in earth system modelling, Earth-Science Reviews, 10.1016/j.earscirev.2019.04.006, (2019).
- Joseph H.A. Guillaume, John D. Jakeman, Stefano Marsili-Libelli, Michael Asher, Philip Brunner, Barry Croke, Mary C. Hill, Anthony J. Jakeman, Karel J. Keesman, Saman Razavi, Johannes D. Stigter, Introductory overview of identifiability analysis: A guide to evaluating whether you have the right type of data for your modeling purpose, Environmental Modelling & Software, 10.1016/j.envsoft.2019.07.007, (2019).
- Yogesh Khare, Christopher J. Martinez, Rafael Muñoz-Carpena, Adelbert “Del” Bottcher, Andrew James, Effective Global Sensitivity Analysis for High-Dimensional Hydrologic and Water Quality Models, Journal of Hydrologic Engineering, 10.1061/(ASCE)HE.1943-5584.0001726, 24, 1, (04018057), (2019).
- Thomas R. B. Grandjean, Liuying Li, Maria Ximena Odio, Widanalage D. Widanage, undefined, 2019 IEEE Vehicle Power and Propulsion Conference (VPPC), 10.1109/VPPC46532.2019.8952455, (1-7), (2019).
- D. Tolley, L. Foglia, T. Harter, Sensitivity Analysis and Calibration of an Integrated Hydrologic Model in an Irrigated Agricultural Basin With a Groundwater‐Dependent Ecosystem, Water Resources Research, 10.1029/2018WR024209, 55, 9, (7876-7901), (2019).
- Juliane Mai, Bryan A. Tolson, Model Variable Augmentation (MVA) for Diagnostic Assessment of Sensitivity Analysis Results, Water Resources Research, 10.1029/2018WR023382, 55, 4, (2631-2651), (2019).
- Katherine R. Barnhart, Rachel C. Glade, Charles M. Shobe, Gregory E. Tucker, Terrainbento 1.0: a Python package for multi-model analysis in long-term drainage basin evolution, Geoscientific Model Development, 10.5194/gmd-12-1267-2019, 12, 4, (1267-1297), (2019).
- David P. Wright, Mark Thyer, Seth Westra, Benjamin Renard, David McInerney, A generalised approach for identifying influential data in hydrological modelling, Environmental Modelling & Software, 10.1016/j.envsoft.2018.03.004, (2018).
- Dmitri Kavetski, Parameter Estimation and Predictive Uncertainty Quantification in Hydrological Modelling, Handbook of Hydrometeorological Ensemble Forecasting, 10.1007/978-3-642-40457-3, (1-42), (2018).
- Dmitri Kavetski, Youwei Qin, George Kuczera, The Fast and the Robust: Trade‐Offs Between Optimization Robustness and Cost in the Calibration of Environmental Models, Water Resources Research, 10.1029/2017WR022051, 54, 11, (9432-9455), (2018).
- Youwei Qin, Dmitri Kavetski, George Kuczera, A Robust Gauss‐Newton Algorithm for the Optimization of Hydrological Models: From Standard Gauss‐Newton to Robust Gauss‐Newton, Water Resources Research, 10.1029/2017WR022488, 54, 11, (9655-9683), (2018).
- Youwei Qin, Dmitri Kavetski, George Kuczera, A Robust Gauss‐Newton Algorithm for the Optimization of Hydrological Models: Benchmarking Against Industry‐Standard Algorithms, Water Resources Research, 10.1029/2017WR022489, 54, 11, (9637-9654), (2018).
- Joshua W. Brownlow, Scott C. James, Joe C. Yelderman, Uncertainty analysis: influence of hydraulic fracturing on overlying aquifers in the presence of leaky abandoned wells, Environmental Earth Sciences, 10.1007/s12665-018-7586-0, 77, 13, (2018).
- Luk J.M. Peeters, Daniel E. Pagendam, Russell S. Crosbie, Praveen K. Rachakonda, Warrick R. Dawes, Lei Gao, Steve P. Marvanek, YongQiang Zhang, Tim R. McVicar, Determining the initial spatial extent of an environmental impact assessment with a probabilistic screening methodology, Environmental Modelling & Software, 10.1016/j.envsoft.2018.08.020, 109, (353-367), (2018).
- Yohann Cousquer, Alexandre Pryet, Olivier Atteia, Ty P.A. Ferré, Célestine Delbart, Rémi Valois, Alain Dupuy, Developing a particle tracking surrogate model to improve inversion of ground water – Surface water models, Journal of Hydrology, 10.1016/j.jhydrol.2018.01.043, 558, (356-365), (2018).
- Fabrizio Fenicia, Dmitri Kavetski, Peter Reichert, Carlo Albert, Signature‐Domain Calibration of Hydrological Models Using Approximate Bayesian Computation: Empirical Analysis of Fundamental Properties, Water Resources Research, 10.1002/2017WR021616, 54, 6, (3958-3987), (2018).
- M. Ye, M.C. Hill, Global Sensitivity Analysis for Uncertain Parameters, Models, and Scenarios, Sensitivity Analysis in Earth Observation Modelling, 10.1016/B978-0-12-803011-0.00010-0, (177-210), (2017).
- Youwei Qin, George Kuczera, Dmitri Kavetski, Comparison of Newton-type and SCE optimisation algorithms for the calibration of conceptual hydrological models, Australasian Journal of Water Resources, 10.1080/13241583.2017.1298180, 20, 2, (169-176), (2017).
- Eun-Mi Hong, Yakov A. Pachepsky, Gene Whelan, Thomas Nicholson, Simpler Models in Environmental Studies and Predictions, Critical Reviews in Environmental Science and Technology, 10.1080/10643389.2017.1393264, (0-0), (2017).
- Christa Kelleher, Brian McGlynn, Thorsten Wagener, Characterizing and reducing equifinality by constraining a distributed catchment model with regional signatures, local observations, and process understanding, Hydrology and Earth System Sciences, 10.5194/hess-21-3325-2017, 21, 7, (3325-3352), (2017).
- Martyn P. Clark, Marc F. P. Bierkens, Luis Samaniego, Ross A. Woods, Remko Uijlenhoet, Katrina E. Bennett, Valentijn R. N. Pauwels, Xitian Cai, Andrew W. Wood, Christa D. Peters-Lidard, The evolution of process-based hydrologic models: historical challenges and the collective quest for physical realism, Hydrology and Earth System Sciences, 10.5194/hess-21-3427-2017, 21, 7, (3427-3440), (2017).
- H. Delottier, A. Pryet, A. Dupuy, Why Should Practitioners be Concerned about Predictive Uncertainty of Groundwater Management Models?, Water Resources Management, 10.1007/s11269-016-1508-2, 31, 1, (61-73), (2016).
- Francesco La Vigna, Mary C. Hill, Rudy Rossetto, Roberto Mazza, Parameterization, sensitivity analysis, and inversion: an investigation using groundwater modeling of the surface-mined Tivoli-Guidonia basin (Metropolitan City of Rome, Italy)Paramétrage, analyse de sensibilité et inversion: une étude utilisant la modélisation des eaux souterraines du bassin de Tivoli-Guidonia (Métropole de Rome, Italie) avec une exploitation du sous-solParametrización, análisis de sensibilidad e inversión: una investigación utilizando modelos de agua subterránea de la cuenca con minería de superficie del Tivoli-Guidonia (Ciudad Metropolitana de Roma, Italia)参数化、灵敏度分析和反演:采用地下水模拟对(意大利罗马市)露天开采的Tivoli-Guidonia盆地进行调查Parametrização, análise de sensibilidade e inversão: uma investigação utilizando modelagem de águas subterrâneas da bacia superficialmente mineirada Tivoli-Guidonia (Cidade Metropolitana de Roma, Itália), Hydrogeology Journal, 10.1007/s10040-016-1393-z, 24, 6, (1423-1441), (2016).
- Jeremy T. White, Michael N. Fienen, John E. Doherty, A python framework for environmental model uncertainty analysis, Environmental Modelling & Software, 10.1016/j.envsoft.2016.08.017, 85, (217-228), (2016).
- Laura Cattaneo, Alessandro Comunian, Giovanna de Filippis, Mauro Giudici, Chiara Vassena, Modeling Groundwater Flow in Heterogeneous Porous Media with YAGMod, Computation, 10.3390/computation4010002, 4, 1, (2), (2015).




