Hostname: page-component-cd9895bd7-mkpzs Total loading time: 0 Render date: 2024-12-23T07:06:25.709Z Has data issue: false hasContentIssue false
  • English
  • Français

APPROXIMATING THE VALUE FUNCTION FOR OPTIMAL EXPERIMENTATION

Published online by Cambridge University Press:  14 November 2018

Hans M. Amman ,
David A. Kendrick  and
Marco P. Tucci
Hans M. Amman*
Affiliation:
University of Amsterdam
David A. Kendrick
Affiliation:
University of Texas
Marco P. Tucci
Affiliation:
University of Siena
*
Address correspondence to: Hans M. Amman, Faculty of Economics and Business, University of Amsterdam, Roetersstraat 11, 1018 WB Amsterdam, The Netherlands; e-mail: [email protected], [email protected]. Mobile: +31651532162.
Get access Rights & Permissions [Opens in a new window]

Abstract

In the economics literature, there are two dominant approaches for solving models with optimal experimentation (also called active learning). The first approach is based on the value function and the second on an approximation method. In principle the value function approach is the preferred method. However, it suffers from the curse of dimensionality and is only applicable to small problems with a limited number of policy variables. The approximation method allows for a computationally larger class of models, but may produce results that deviate from the optimal solution. Our simulations indicate that when the effects of learning are limited, the differences may be small. However, when there is sufficient scope for learning, the value function solution seems more aggressive in the use of the policy variable.

Keywords

Optimal ExperimentationValue FunctionApproximation MethodAdaptive ControlActive LearningTime-Varying ParametersNumerical Experiments
Type
Articles
Information
Macroeconomic Dynamics , Volume 24 , Issue 5 , July 2020 , pp. 1073 - 1086
Copyright
© Cambridge University Press 2018

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Footnotes

We would like to thank Volker Wieland for providing us with his software used in this paper. Furthermore, in writing this paper we have benefited greatly from the discussions we had with Tom Cosimano and Volker Wieland and the feedback from an anonymous referee.

References

Aghion, P., Bolton, P., Harris, C. and Jullien, B. (1991) Optimal learning by experimentation. Review of Economic Studies 58, 621–654.CrossRefGoogle Scholar
Amman, H. M. (1989) Nonlinear control simulation on a vector machine. Parallel Computing 10, 123–127.CrossRefGoogle Scholar
Amman, H. M. (1996) Numerical methods for linear-quadratic models. In: Amman, H. M., Kendrick, D. A., and Rust, J. (eds.), Handbook of Computational Economics. Handbook in Economics, Volume 13, pp. 579–618. Amsterdam, The Netherlands: North-Holland Publishers (Elsevier).Google Scholar
Amman, H. M. and Tucci, M. P. (2017) The Dual Approach in an Infinite Horizon Model. Quaderni del Dipartimento di Economia Politica 766, Università di Siena, Siena, Italy.Google Scholar
Amman, H. M. and Kendrick, D. A. (1990) A User’s Guide for DUAL: A Program for Quadratic-Linear Stochastic Control Problems. Technical Report T90-4, Center for Economic Research, University of Texas, Austin, Texas, USA.Google Scholar
Amman, H. M. and Kendrick, D. A. (1995) Nonconvexities in stochastic control models. International Economic Review 36, 455–475.CrossRefGoogle Scholar
Amman, H. M. and Kendrick, D. A. (1999) Linear-quadratic optimization for models with rational expectations. Macroeconomic Dynamics 3, 534–543.CrossRefGoogle Scholar
Amman, H. M. and Kendrick, D. A. (2003) Mitigation of the Lucas critique with stochastic control methods. Journal of Economic Dynamics and Control 27, 2035–2057.CrossRefGoogle Scholar
Amman, H. M. and Kendrick, D. A. (2008) Comparison of policy functions from the optimal learning and adaptive control framework. Discussion paper no. 08-19, Tjalling Koopman Institute, Utrecht School of Economics, Utrecht University, Utrecht, The Netherlands.Google Scholar
Bar-Shalom, Y. and Sivan, R. (1969) On the optimal control of discrete-time linear systems with random parameters. IEEE Transactions on Automatic Control 14, 3–8.CrossRefGoogle Scholar
Beck, G. and Wieland, V. (2002) Learning and control in a changing economic environment. Journal of Economic Dynamics and Control 26, 1359–1377.CrossRefGoogle Scholar
Bellman, R. E. (1957) Dynamic Programming. Princeton, NJ: Princeton University Press.Google ScholarPubMed
Bertsekas, D. P. (1976) Dynamic Programming and Stochastic Control. Mathematics in Science and Engineering, Volume 125. New York: Academic Press.Google Scholar
Bolton, P. and Harris, C. (1999) Strategic experimentation. Econometrica 67(2), 349–374.CrossRefGoogle Scholar
Buera, F. J., Monge-Naranjo, A. and Primiceri, G. E. (2011) Learning the wealth of nations. Econometrica 79(1), 1–45.Google Scholar
Coenen, G., Levin, A. and Wieland, V. (2005) Data uncertainty and the role of money as an information variable for monetary policy. European Economic Review 49, 975–1006.CrossRefGoogle Scholar
Cosimano, T. F. (2008) Optimal experimentation and the perturbation method in the neighborhood of the augmented linear regulator problem. Journal of Economics, Dynamics and Control 32, 1857–1894.CrossRefGoogle Scholar
Cosimano, T. F. and Gapen, M. T. (2005a) Program Notes for Optimal Experimentation and the Perturbation Method in the Neighborhood of the Augmented Linear Regulator Problem. Working paper, Department of Finance, University of Notre Dame, Notre Dame, Indiana, USA.Google Scholar
Cosimano, T. F. and Gapen, M. T. (2005b) Recursive Methods of Dynamic Linear Economics and Optimal Experimentation Using the Perturbation Method. Working paper, Department of Finance, University of Notre Dame, Notre Dame, Indiana, USA.Google Scholar
Easley, D. and Kiefer, N. M. (1988) Controlling a stochastic process with unknown parameters. Econometrica 56, 1045–1064.CrossRefGoogle Scholar
Hansen, L. P. and Sargent, T. J. (2007) Robustness. Princeton, NJ: Princeton University Press.Google Scholar
Judd, K. L. (1998) Numerical Methods in Economics. Cambridge, MA: MIT Press.Google Scholar
Kendrick, D. A. (1978) Non-convexities from probing an adaptive control problem. Economic Letters 1, 347–351.CrossRefGoogle Scholar
Kendrick, D. A. (1981) Stochastic Control for Economic Models. New York, NY, USA: McGraw-Hill Book Company. Second Edition, 2002.Google Scholar
Kendrick, D. A., Amman, H. M. and Tucci, M. P. (2014) Learning about learning in dynamic economic models. In: Schmedders, K., and Judd, K. (eds.), Handbook of Computational Economics. Handbooks in Economics, Volume 3, Chapter 1, pp. 1–35. North-Holland: Elsevier.Google Scholar
Kendrick, D. A., Mercado, P. R. and Amman, H. M. (2006a) Computational Economics (Supplementary chapters on the value function are provided by authors). Princeton and Oxford: Princeton University Press.Google Scholar
Kendrick, D. A., Tucci, M. P. and Amman, H. M. (2006b) DualI: Software for solving stochastic control problems. Working paper, Department of Economics, University of Texas, Austin, Texas, USA.Google Scholar
Kiefer, N. (1989) A value function arising in the economics of information. Journal of Economic Dynamics and Control 13, 201–223.CrossRefGoogle Scholar
Kiefer, N. and Nyarko, Y. (1989) Optimal control of an unknown linear process with learning. International Economic Review 30, 571–586.CrossRefGoogle Scholar
Levin, A., Wieland, V. and Williams, J. C. (2003) The performance of forecast-based monetary policy rules under model uncertainty. American Economic Review 93, 622–645.CrossRefGoogle Scholar
MacRae, E. C. (1972) Linear decision with experimentation. Annals of Economic and Social Measurement 1, 437–448.Google Scholar
MacRae, E. C. (1975) An adaptive learning role for multi-period decision problems. Econometrica 43, 893–906.CrossRefGoogle Scholar
Moscarini, G. and Smith, L. (2001) The optimal level of experimentation. Econometrica 69(6), 1629–1644.CrossRefGoogle Scholar
Prescott, E. C. (1972) The multi-period control problem under uncertainty. Econometrica 40, 1043–1058.CrossRefGoogle Scholar
Salmon, T. C. (2001) An evaluation of econometric models of adaptive learning. Econometrica 69(6), 1597–1628.CrossRefGoogle Scholar
Taylor, J. B. (1974) Asymptotic properties of multi-period control rules in the linear regression model. International Economic Review 15, 472–482.CrossRefGoogle Scholar
Tse, E. (1973) Further comments on adaptive stochastic control for a class of linear systems. IEEE Transactions on Automatic Control 18, 324–326.CrossRefGoogle Scholar
Tucci, M. P. (2004) The Rational Expectation Hypothesis, Time-varying Parameters and Adaptive Control. Dordrecht, The Netherlands: Springer.CrossRefGoogle Scholar
Tucci, M. P., Kendrick, D. A. and Amman, H. M. (2010) The parameter set in an adaptive control Monte Carlo experiment: Some considerations. Journal of Economic Dynamics and Control 34, 1531–1549.CrossRefGoogle Scholar
Wieland, V. (2000a) Learning by doing and the value of optimal experimentation. Journal of Economic Dynamics and Control 24, 501–534.CrossRefGoogle Scholar
Wieland, V. (2000b) Monetary policy, parameter uncertainty and optimal learning. Journal of Monetary Economics 46, 199–228.CrossRefGoogle Scholar
Willems, T. (2012) Essays on Optimal Experimentation. PhD thesis, Tinbergen Institute, University of Amsterdam, Amsterdam, The Netherlands.Google Scholar