Volume 54, Issue 2
Issue Paper/

Practical Use of Computationally Frugal Model Analysis Methods

by
Mary C. Hill

Corresponding Author

U.S. Geological Survey, Boulder, CO

Corresponding author: Department of Geology, University of Kansas, 1475 Jayhawk Blvd., Lawrence, KS 66049; 785‐864‐2728; E-mail address: mchill@ku.eduSearch for more papers by this author
Dmitri Kavetski

E-mail address: dmitri.kavetski@adelaide.edu.au

University of Adelaide, Adelaide, Australia

Search for more papers by this author
Martyn Clark

E-mail address: mclark@ucar.edu

National Center for Atmospheric Research, Boulder, CO

Search for more papers by this author
Ming Ye

E-mail address: mye@fsu.edu

Florida State University, Tallahassee, FL

Search for more papers by this author
Mazdak Arabi

E-mail address: marabi@engr.colostate.edu

Colorado State University, Fort Collins, CO

Search for more papers by this author
Dan Lu

E-mail address: lud1@ornl.gov

Oak Ridge National Laboratory, Oak Ridge, TN

Search for more papers by this author
Laura Foglia

E-mail address: foglia@geo.tu-darmstadt.de

University of Darmstadt, Darmstadt, Germany

Search for more papers by this author
Steffen Mehl

E-mail address: smehl@csuchico.edu

California State University, Chico, CA

Search for more papers by this author
First published: 21 March 2015
Citations: 29

Abstract

Three challenges compromise the utility of mathematical models of groundwater and other environmental systems: (1) a dizzying array of model analysis methods and metrics make it difficult to compare evaluations of model adequacy, sensitivity, and uncertainty; (2) the high computational demands of many popular model analysis methods (requiring 1000's, 10,000 s, or more model runs) make them difficult to apply to complex models; and (3) many models are plagued by unrealistic nonlinearities arising from the numerical model formulation and implementation. This study proposes a strategy to address these challenges through a careful combination of model analysis and implementation methods. In this strategy, computationally frugal model analysis methods (often requiring a few dozen parallelizable model runs) play a major role, and computationally demanding methods are used for problems where (relatively) inexpensive diagnostics suggest the frugal methods are unreliable. We also argue in favor of detecting and, where possible, eliminating unrealistic model nonlinearities—this increases the realism of the model itself and facilitates the application of frugal methods. Literature examples are used to demonstrate the use of frugal methods and associated diagnostics. We suggest that the strategy proposed in this paper would allow the environmental sciences community to achieve greater transparency and falsifiability of environmental models, and obtain greater scientific insight from ongoing and future modeling efforts.

Number of times cited according to CrossRef: 29

  • Rainfall-runoff simulation and flood dynamic monitoring based on CHIRPS and MODIS-ET, International Journal of Remote Sensing, 10.1080/01431161.2020.1714779, 41, 11, (4206-4225), (2020).
  • Urban growth and groundwater sustainability: evaluating spatially distributed recharge alternatives in the Mexico City Metropolitan Area, Journal of Hydrology, 10.1016/j.jhydrol.2020.124909, (124909), (2020).
  • Groundwater sustainability: a review of the interactions between science and policy, Environmental Research Letters, 10.1088/1748-9326/ab8e8c, 15, 9, (093004), (2020).
  • Parameter Estimation and Predictive Uncertainty Quantification in Hydrological Modelling, Handbook of Hydrometeorological Ensemble Forecasting, 10.1007/978-3-642-39925-1, (481-522), (2019).
  • What has Global Sensitivity Analysis ever done for us? A systematic review to support scientific advancement and to inform policy-making in earth system modelling, Earth-Science Reviews, 10.1016/j.earscirev.2019.04.006, (2019).
  • Introductory overview of identifiability analysis: A guide to evaluating whether you have the right type of data for your modeling purpose, Environmental Modelling & Software, 10.1016/j.envsoft.2019.07.007, (2019).
  • Effective Global Sensitivity Analysis for High-Dimensional Hydrologic and Water Quality Models, Journal of Hydrologic Engineering, 10.1061/(ASCE)HE.1943-5584.0001726, 24, 1, (04018057), (2019).
  • undefined, 2019 IEEE Vehicle Power and Propulsion Conference (VPPC), 10.1109/VPPC46532.2019.8952455, (1-7), (2019).
  • Sensitivity Analysis and Calibration of an Integrated Hydrologic Model in an Irrigated Agricultural Basin With a Groundwater‐Dependent Ecosystem, Water Resources Research, 10.1029/2018WR024209, 55, 9, (7876-7901), (2019).
  • Model Variable Augmentation (MVA) for Diagnostic Assessment of Sensitivity Analysis Results, Water Resources Research, 10.1029/2018WR023382, 55, 4, (2631-2651), (2019).
  • Terrainbento 1.0: a Python package for multi-model analysis in long-term drainage basin evolution, Geoscientific Model Development, 10.5194/gmd-12-1267-2019, 12, 4, (1267-1297), (2019).
  • A generalised approach for identifying influential data in hydrological modelling, Environmental Modelling & Software, 10.1016/j.envsoft.2018.03.004, (2018).
  • Parameter Estimation and Predictive Uncertainty Quantification in Hydrological Modelling, Handbook of Hydrometeorological Ensemble Forecasting, 10.1007/978-3-642-40457-3, (1-42), (2018).
  • The Fast and the Robust: Trade‐Offs Between Optimization Robustness and Cost in the Calibration of Environmental Models, Water Resources Research, 10.1029/2017WR022051, 54, 11, (9432-9455), (2018).
  • A Robust Gauss‐Newton Algorithm for the Optimization of Hydrological Models: From Standard Gauss‐Newton to Robust Gauss‐Newton, Water Resources Research, 10.1029/2017WR022488, 54, 11, (9655-9683), (2018).
  • A Robust Gauss‐Newton Algorithm for the Optimization of Hydrological Models: Benchmarking Against Industry‐Standard Algorithms, Water Resources Research, 10.1029/2017WR022489, 54, 11, (9637-9654), (2018).
  • Uncertainty analysis: influence of hydraulic fracturing on overlying aquifers in the presence of leaky abandoned wells, Environmental Earth Sciences, 10.1007/s12665-018-7586-0, 77, 13, (2018).
  • Determining the initial spatial extent of an environmental impact assessment with a probabilistic screening methodology, Environmental Modelling & Software, 10.1016/j.envsoft.2018.08.020, 109, (353-367), (2018).
  • Developing a particle tracking surrogate model to improve inversion of ground water – Surface water models, Journal of Hydrology, 10.1016/j.jhydrol.2018.01.043, 558, (356-365), (2018).
  • Signature‐Domain Calibration of Hydrological Models Using Approximate Bayesian Computation: Empirical Analysis of Fundamental Properties, Water Resources Research, 10.1002/2017WR021616, 54, 6, (3958-3987), (2018).
  • Global Sensitivity Analysis for Uncertain Parameters, Models, and Scenarios, Sensitivity Analysis in Earth Observation Modelling, 10.1016/B978-0-12-803011-0.00010-0, (177-210), (2017).
  • Comparison of Newton-type and SCE optimisation algorithms for the calibration of conceptual hydrological models, Australasian Journal of Water Resources, 10.1080/13241583.2017.1298180, 20, 2, (169-176), (2017).
  • Simpler Models in Environmental Studies and Predictions, Critical Reviews in Environmental Science and Technology, 10.1080/10643389.2017.1393264, (0-0), (2017).
  • Characterizing and reducing equifinality by constraining a distributed catchment model with regional signatures, local observations, and process understanding, Hydrology and Earth System Sciences, 10.5194/hess-21-3325-2017, 21, 7, (3325-3352), (2017).
  • The evolution of process-based hydrologic models: historical challenges and the collective quest for physical realism, Hydrology and Earth System Sciences, 10.5194/hess-21-3427-2017, 21, 7, (3427-3440), (2017).
  • Why Should Practitioners be Concerned about Predictive Uncertainty of Groundwater Management Models?, Water Resources Management, 10.1007/s11269-016-1508-2, 31, 1, (61-73), (2016).
  • Parameterization, sensitivity analysis, and inversion: an investigation using groundwater modeling of the surface-mined Tivoli-Guidonia basin (Metropolitan City of Rome, Italy)Paramétrage, analyse de sensibilité et inversion: une étude utilisant la modélisation des eaux souterraines du bassin de Tivoli-Guidonia (Métropole de Rome, Italie) avec une exploitation du sous-solParametrización, análisis de sensibilidad e inversión: una investigación utilizando modelos de agua subterránea de la cuenca con minería de superficie del Tivoli-Guidonia (Ciudad Metropolitana de Roma, Italia)参数化、灵敏度分析和反演:采用地下水模拟对(意大利罗马市)露天开采的Tivoli-Guidonia盆地进行调查Parametrização, análise de sensibilidade e inversão: uma investigação utilizando modelagem de águas subterrâneas da bacia superficialmente mineirada Tivoli-Guidonia (Cidade Metropolitana de Roma, Itália), Hydrogeology Journal, 10.1007/s10040-016-1393-z, 24, 6, (1423-1441), (2016).
  • A python framework for environmental model uncertainty analysis, Environmental Modelling & Software, 10.1016/j.envsoft.2016.08.017, 85, (217-228), (2016).
  • Modeling Groundwater Flow in Heterogeneous Porous Media with YAGMod, Computation, 10.3390/computation4010002, 4, 1, (2), (2015).