Uncertainty Analysis and Experimental Design

by Joep Vanlier and Natal van Riel

Systems Biology employs mathematical modelling of biological reaction networks by systems of nonlinear differential equations. Model parameters need to be estimated using experimental data. Given the complexity of the models in combination with the limited amount of quantitative data it is important to infer how well model parameters can be determined and how uncertainty in parameters propagates into model predictions. Model identifiability and uncertainty analysis are becoming increasingly important topics in Systems Biology.
An unique approach and associated numerical tools have been developed for uncertainty analysis for dynamic models described by differential equations. The strategy for Prediction Uncertainty Analysis (PUA) integrates Profile Likelihood analysis with Bayesian sampling. Different analyses are performed sequentially to detect and avoid problems associated with the individual techniques. The PUA approach enables computation of a Posterior Predictive Distribution (PPD), which subsequently can be used for different types of analyses and experimental design.
Our Bayesian approach to Experimental Design (BED) provides a method to select experiments that will reduce the uncertainty of specific predictions in an optimal manner. By applying importance sampling to the PPD, the efficacy of a new measurement is predicted. Because predictive distributions can be computed for a wide range of quantities, this approach is very flexible. The combinatorial effect of designing multiple experiments simultaneously can also be probed.
Optimal Experimental Design for Model Selection (OED_MS) can be used to determine which experiments should be done to effectively discriminate between competing hypotheses on how a biochemical pathway operates, as captured in different models. Differences in the hypotheses can relate to molecular mechanisms as well as the network topologies.

Downloads:

Software can be used under GNU General Public License. See http://www.gnu.org/copyleft/gpl for terms and conditions.


ODEMEX (CVode Wrapper)

A toolkit to speed-up the simulation of (nonlinear) ODE models in Matlab. See ODEMEX (CVode wrapper for Matlab) for details.


Prediction Uncertainty Analysis (PUA)

The Prediction Uncertainty Analysis package for Matlab includes:

When using this toolbox please refer to:
J. Vanlier, C.A. Tiemann, P.A.J. Hilbers and N.A.W. van Riel. 'An Integrated Strategy for Prediction Uncertainty Analysis' Bioinformatics, 2012; 28(8): 1130-5, link journal website

To reproduce the results from the paper the software is also available as win32 executables:

To run these, one first needs to install the MATLAB Runtime Environment (also provided in the zip file). Please note that a reboot is required after installation. The different steps from the paper can be reproduced by consecutively running the following files:

Please note that JAKMCMC_log takes a long time to complete (2.5 days on an AMD Athlon X64 X2 5600+). One can view what the chain has done so far by moving to the path and running mcmcAnalysis or predictionAnalysis; note however that due to the burn-in, there have to be at least 50 batches already available (these are generated in a subdirectory).


Bayesian Experimental Design (BED)

The Bayesian Experimental Design package for Matlab is composed of 2 parts. OED v3 includes:

OED_SMC contains our Sequential Monte Carlo (SMC) sampler.

In order to use the package, first install the PUA package.

When using this toolbox please refer to:
J. Vanlier, C.A. Tiemann, P.A.J. Hilbers and N.A.W. van Riel. 'A Bayesian Approach to Targeted Experiment Design' Bioinformatics. 2012; 28(8):1136-42, link journal website

With this code the results found in the paper can be reproduced.


Optimal Experimental Design for Model Selection (OED_MS)

The Optimal Experimental Design for Model Selection (OED_MS) package for Matlab includes:

In order to use the package, first install the PUA package.

Optimal Experimental Design for Model Selection: OED_MS (zip file)

When using this toolbox please refer to:
J. Vanlier, C.A. Tiemann, P.A.J. Hilbers and N.A.W. van Riel. 'Optimal experiment design for model selection in biochemical networks' BMC Syst. Biol. 2014; 8:20, link journal website