Approximate Dynamic Programming for Commodity and Energy Merchant Operations

We study the merchant operations of commodity and energy conversion assets. Examples of such assets include natural gas pipelines systems, commodity swing options, and power plants. Merchant operations involves managing these assets as real options on commodity and energy prices with the objective of maximizing the market value of these assets. The economic relevance of natural gas conversion assets has increased considerably since the occurrence of the oil and gas shale boom; for example, the Energy Information Agency expects natural gas to be the source of 30% of the world's electricity production by 2040 and the McKinsey Global Institute projects United States spending on energy infrastructure to be about 100 Billion dollars by 2020. Managing commodity and energy conversion assets can be formulated as intractable Markov decision problems (MDPs), especially when using high dimensional price models commonly employed in practice. We develop approximate dynamic programming (ADP) methods for computing near optimal policies and lower and upper bounds on the market value of these assets. We focus on overcoming issues with the standard math programming and financial engineering ADP methods, that is, approximate linear programing (ALP) and least squares Monte Carlo (LSM), respectively. In particular, we develop: (i) a novel ALP relaxation framework to improve the ALP approach and use it to derive two new classes of ALP relaxations; (ii) an LSM variant in the context of popular practice-based price models to alleviate the substantial computational overhead when estimating upper bounds on the market value using existing LSM variants; and (iii) a mixed integer programming based ADP method that is exact with respect to a policy performance measure, while methods in the literature are heuristic in nature. Computational experiments on realistic instances of natural gas storage and crude oil swing options show that both our ALP relaxations and LSM methods are efficient and deliver near optimal policies and tight lower and upper bounds. Our LSM variant is also between one and three orders of magnitude faster than existing LSM variants for estimating upper bounds. Our mixed integer programming ADP model is computationally expensive to solve but its exact nature motivates further research into its solution. We provide theoretical support for our methods: By deriving bounds on approximation error we establish the optimality of our best ALP relaxation class in limiting regimes of practical relevance and provide a theoretical perspective on the relative performance of our LSM variant and existing LSM variants. We also unify different ADP methods in the literature using our ALP relaxation framework, including the financial engineering based LSM method. In addition, we employ ADP to study the novel application of jointly managing storage and transport assets in a natural gas pipeline system; the literature studies these assets in isolation. We leverage our structural analysis of the optimal storage policy to extend an LSM variant for this problem. This extension computes near optimal policies and tight bounds on instances formulated in collaboration with a major natural gas trading company. We use our extension and these instances to answer questions relevant to merchants managing such assets. Overall, our findings highlight the role of math programming for developing ADP methods. Although we focus on managing commodity and energy conversion assets, the techniques in this thesis have potential broader relevance for solving MDPs in other application contexts, such as inventory control with demand forecast updating, multiple sourcing, and optimal medical treatment design.