This thesis presents a framework for sequential Bayesian optimal experimental design to learn the behaviors of a black-box function. A Gaussian process regression model is used to model the black-box function. A Quantity of Interest (QoI) defines the goals of the experimental design, where both one-dimensional and multidimensional QoIs are considered. The use of multidimensional QoIs in experimental design is demonstrated in an Analytical example. Approximate dynamic programming is presented as one of the few numerical strategies to solve the sequential experimental design problem. The approximate dynamic programming method is demonstrated through a numerical example, where costs of transitioning between the experiments are incorporated into the design.