skip to content
Numerical Methods for Stochastic Control Problems in Continuous Time Preview this item
ClosePreview this item
Checking...

Numerical Methods for Stochastic Control Problems in Continuous Time

Author: Harold J Kushner; Paul G Dupuis
Publisher: New York, NY : Springer US, 1992.
Series: Applications of Mathematics, Stochastic Modelling and Applied Probability, 24.
Edition/Format:   eBook : Document : EnglishView all editions and formats
Summary:
The book presents a comprehensive development of effective numerical methods for stochastic control problems in continuous time. The process models are diffusions, jump-diffusions or reflected diffusions of the type that occur in the majority of current applications. All the usual problem formulations are included, as well as those of more recent interest such as ergodic control, singular control and the types of  Read more...
Rating:

(not yet rated) 0 with reviews - Be the first.

Subjects
More like this

 

Find a copy online

Links to this item

Find a copy in the library

&AllPage.SpinnerRetrieving; Finding libraries that hold this item...

Details

Genre/Form: Electronic books
Additional Physical Format: Print version:
Material Type: Document, Internet resource
Document Type: Internet Resource, Computer File
All Authors / Contributors: Harold J Kushner; Paul G Dupuis
ISBN: 9781468404418 1468404415 0387978348 9780387978345 3540978348 9783540978343
OCLC Number: 851825515
Description: 1 online resource (x, 439 pages 40 illustrations).
Contents: 1 Review of Continuous Time Models --
1.1 Martingales and Martingale Inequalities --
1.2 Stochastic Integration --
1.3 Stochastic Differential Equations: Diffusions --
1.4 Reflected Diffusions --
1.5 Processes with Jumps --
2 Controlled Markov Chains --
2.1 Recursive Equations for the Cost --
2.2 Optimal Stopping Problems --
2.3 Discounted Cost --
2.4 Control to a Target Set and Contraction Mappings --
2.5 Finite Time Control Problems --
3 Dynamic Programming Equations --
3.1 Functionals of Uncontrolled Processes --
3.2 The Optimal Stopping Problem --
3.3 Control Until a Target Set Is Reached --
3.4 A Discounted Problem with a Target Set and Reflection --
3.5 Average Cost Per Unit Time --
4 The Markov Chain Approximation Method: Introduction --
4.1 The Markov Chain Approximation Method --
4.2 Continuous Time Interpolation and Approximating Cost Function --
4.3 A Continuous Time Markov Chain Interpolation --
4.4 A Random Walk Approximation to the Wiener Process --
4.5 A Deterministic Discounted Problem --
4.6 Deterministic Relaxed Controls --
5 Construction of the Approximating Markov Chain --
5.1 Finite Difference Type Approximations: One Dimensional Examples --
5.2 Numerical Simplifications and Alternatives for Example 4 --
5.3 The General Finite Difference Method --
5.4 A Direct Construction of the Approximating Markov Chain --
5.5 Variable Grids --
5.6 Jump Diffusion Processes --
5.7 Approximations for Reflecting Boundaries --
5.8 Dynamic Programming Equations --
6 Computational Methods for Controlled Markov Chains --
6.1 The Problem Formulation --
6.2 Classical Iterative Methods: Approximation in Policy and Value Space --
6.3 Error Bounds for Discounted Problems --
6.4 Accelerated Jacobi and Gauss-Seidel Methods --
6.5 Domain Decomposition and Implementation on Parallel Processors --
6.6 A State Aggregation Method --
6.7 Coarse Grid-Fine Grid Solutions --
6.8 A Multigrid Method --
6.9 Linear Programming Formulations and Constraints --
7 The Ergodic Cost Problem: Formulations and Algorithms --
7.1 The Control Problem for the Markov Chain: Formulation --
7.2 A Jacobi Type Iteration --
7.3 Approximation in Policy Space --
7.4 Numerical Methods for the Solution of (3.4) --
7.5 The Control Problem for the Approximating Markov Chain --
7.6 The Continuous Parameter Markov Chain Interpolation --
7.7 Computations for the Approximating Markov Chain --
7.8 Boundary Costs and Controls --
8 Heavy Traffic and Singular Control Problems: Examples and Markov Chain Approximations --
8.1 Motivating Examples --
8.2 The Heavy Traffic Problem: A Markov Chain Approximation --
8.3 Singular Control: A Markov Chain Approximation --
9 Weak Convergence and the Characterization of Processes --
9.1 Weak Convergence --
9.2 Criteria for Tightness in Dk [0,?) --
9.3 Characterization of Processes --
9.4 An Example --
9.5 Relaxed Controls --
10 Convergence Proofs --
10.1 Limit Theorems and Approximations of Relaxed Controls --
10.2 Existence of an Optimal Control: Absorbing Boundary --
10.3 Approximating the Optimal Control --
10.4 The Approximating Markov Chain: Weak Convergence --
10.5 Convergence of the Costs: Discounted Cost and Absorbing Boundary --
10.6 The Optimal Stopping Problem --
11 Convergence for Reflecting Boundaries, Singular Control and Ergodic Cost Problems --
11.1 The Reflecting Boundary Problem --
11.2 The Singular Control Problem --
11.3 The Ergodic Cost Problem --
12 Finite Time Problems and Nonlinear Filtering --
12.1 The Explicit Approximation Method: An Example --
12.2 The General Explicit Approximation Method --
12.3 The Implicit Approximation Method: An Example --
12.4 The General Implicit Approximation Method --
12.5 The Optimal Control Problem: Approximations and Dynamic Programming Equations --
12.6 Methods of Solution, Decomposition and Convergence --
12.7 Nonlinear Filtering --
13 Problems from the Calculus of Variations --
13.1 Problems of Interest --
13.2 Numerical Schemes and Convergence for the Finite Time Problem --
13.3 Problems with a Controlled Stopping Time --
13.4 Problems with a Discontinuous Running Cost --
14 The Viscosity Solution Approach to Proving Convergence of Numerical Schemes --
14.1 Definitions and Some Properties of Viscosity Solutions --
14.2 Numerical Schemes --
14.3 Proof of Convergence --
References --
List of Symbols.
Series Title: Applications of Mathematics, Stochastic Modelling and Applied Probability, 24.
Responsibility: by Harold J. Kushner, Paul G. Dupuis.

Abstract:

The book presents a comprehensive development of effective numerical methods for stochastic control problems in continuous time. The process models are diffusions, jump-diffusions or reflected diffusions of the type that occur in the majority of current applications. All the usual problem formulations are included, as well as those of more recent interest such as ergodic control, singular control and the types of reflected diffusions used as models of queuing networks. Convergence of the numerical approximations is proved via the efficient probabilistic methods of weak convergence theory. The methods also apply to the calculation of functionals of uncontrolled processes and for the appropriate to optimal nonlinear filters as well. Applications to complex deterministic problems are illustrated via application to a large class of problems from the calculus of variations. The general approach is known as the Markov Chain Approximation Method. Essentially all that is required of the approximations are some natural local consistency conditions. The approximations are consistent with standard methods of numerical analysis. The required background in stochastic processes is surveyed, there is an extensive development of methods of approximation, and a chapter is devoted to computational techniques. The book is written on two levels, that of practice (algorithms and applications), and that of the mathematical development. Thus the methods and use should be broadly accessible.

Reviews

User-contributed reviews
Retrieving GoodReads reviews...
Retrieving DOGObooks reviews...

Tags

Be the first.
Confirm this request

You may have already requested this item. Please select Ok if you would like to proceed with this request anyway.

Linked Data


Primary Entity

<http://www.worldcat.org/oclc/851825515> # Numerical Methods for Stochastic Control Problems in Continuous Time
    a schema:Book, schema:CreativeWork, schema:MediaObject ;
   library:oclcnum "851825515" ;
   library:placeOfPublication <http://id.loc.gov/vocabulary/countries/nyu> ;
   library:placeOfPublication <http://experiment.worldcat.org/entity/work/data/24228596#Place/new_york_ny> ; # New York, NY
   schema:about <http://experiment.worldcat.org/entity/work/data/24228596#Topic/systems_theory> ; # Systems theory
   schema:about <http://id.worldcat.org/fast/895600> ; # Distribution (Probability theory)
   schema:about <http://id.loc.gov/authorities/subjects/sh85093237> ; # Numerical analysis
   schema:about <http://dewey.info/class/519/e23/> ;
   schema:about <http://id.loc.gov/authorities/subjects/sh85082127> ; # Mathematical optimization
   schema:about <http://id.loc.gov/authorities/subjects/sh85082139> ; # Mathematics
   schema:about <http://id.worldcat.org/fast/1041273> ; # Numerical analysis
   schema:about <http://id.worldcat.org/fast/1012099> ; # Mathematical optimization
   schema:about <http://id.worldcat.org/fast/1012163> ; # Mathematics
   schema:bookFormat schema:EBook ;
   schema:contributor <http://viaf.org/viaf/9035984> ; # Paul G. Dupuis
   schema:creator <http://viaf.org/viaf/84606868> ; # Harold J. Kushner
   schema:datePublished "1992" ;
   schema:description "The book presents a comprehensive development of effective numerical methods for stochastic control problems in continuous time. The process models are diffusions, jump-diffusions or reflected diffusions of the type that occur in the majority of current applications. All the usual problem formulations are included, as well as those of more recent interest such as ergodic control, singular control and the types of reflected diffusions used as models of queuing networks. Convergence of the numerical approximations is proved via the efficient probabilistic methods of weak convergence theory. The methods also apply to the calculation of functionals of uncontrolled processes and for the appropriate to optimal nonlinear filters as well. Applications to complex deterministic problems are illustrated via application to a large class of problems from the calculus of variations. The general approach is known as the Markov Chain Approximation Method. Essentially all that is required of the approximations are some natural local consistency conditions. The approximations are consistent with standard methods of numerical analysis. The required background in stochastic processes is surveyed, there is an extensive development of methods of approximation, and a chapter is devoted to computational techniques. The book is written on two levels, that of practice (algorithms and applications), and that of the mathematical development. Thus the methods and use should be broadly accessible."@en ;
   schema:description "1 Review of Continuous Time Models -- 1.1 Martingales and Martingale Inequalities -- 1.2 Stochastic Integration -- 1.3 Stochastic Differential Equations: Diffusions -- 1.4 Reflected Diffusions -- 1.5 Processes with Jumps -- 2 Controlled Markov Chains -- 2.1 Recursive Equations for the Cost -- 2.2 Optimal Stopping Problems -- 2.3 Discounted Cost -- 2.4 Control to a Target Set and Contraction Mappings -- 2.5 Finite Time Control Problems -- 3 Dynamic Programming Equations -- 3.1 Functionals of Uncontrolled Processes -- 3.2 The Optimal Stopping Problem -- 3.3 Control Until a Target Set Is Reached -- 3.4 A Discounted Problem with a Target Set and Reflection -- 3.5 Average Cost Per Unit Time -- 4 The Markov Chain Approximation Method: Introduction -- 4.1 The Markov Chain Approximation Method -- 4.2 Continuous Time Interpolation and Approximating Cost Function -- 4.3 A Continuous Time Markov Chain Interpolation -- 4.4 A Random Walk Approximation to the Wiener Process -- 4.5 A Deterministic Discounted Problem -- 4.6 Deterministic Relaxed Controls -- 5 Construction of the Approximating Markov Chain -- 5.1 Finite Difference Type Approximations: One Dimensional Examples -- 5.2 Numerical Simplifications and Alternatives for Example 4 -- 5.3 The General Finite Difference Method -- 5.4 A Direct Construction of the Approximating Markov Chain -- 5.5 Variable Grids -- 5.6 Jump Diffusion Processes -- 5.7 Approximations for Reflecting Boundaries -- 5.8 Dynamic Programming Equations -- 6 Computational Methods for Controlled Markov Chains -- 6.1 The Problem Formulation -- 6.2 Classical Iterative Methods: Approximation in Policy and Value Space -- 6.3 Error Bounds for Discounted Problems -- 6.4 Accelerated Jacobi and Gauss-Seidel Methods -- 6.5 Domain Decomposition and Implementation on Parallel Processors -- 6.6 A State Aggregation Method -- 6.7 Coarse Grid-Fine Grid Solutions -- 6.8 A Multigrid Method -- 6.9 Linear Programming Formulations and Constraints -- 7 The Ergodic Cost Problem: Formulations and Algorithms -- 7.1 The Control Problem for the Markov Chain: Formulation -- 7.2 A Jacobi Type Iteration -- 7.3 Approximation in Policy Space -- 7.4 Numerical Methods for the Solution of (3.4) -- 7.5 The Control Problem for the Approximating Markov Chain -- 7.6 The Continuous Parameter Markov Chain Interpolation -- 7.7 Computations for the Approximating Markov Chain -- 7.8 Boundary Costs and Controls -- 8 Heavy Traffic and Singular Control Problems: Examples and Markov Chain Approximations -- 8.1 Motivating Examples -- 8.2 The Heavy Traffic Problem: A Markov Chain Approximation -- 8.3 Singular Control: A Markov Chain Approximation -- 9 Weak Convergence and the Characterization of Processes -- 9.1 Weak Convergence -- 9.2 Criteria for Tightness in Dk [0,?) -- 9.3 Characterization of Processes -- 9.4 An Example -- 9.5 Relaxed Controls -- 10 Convergence Proofs -- 10.1 Limit Theorems and Approximations of Relaxed Controls -- 10.2 Existence of an Optimal Control: Absorbing Boundary -- 10.3 Approximating the Optimal Control -- 10.4 The Approximating Markov Chain: Weak Convergence -- 10.5 Convergence of the Costs: Discounted Cost and Absorbing Boundary -- 10.6 The Optimal Stopping Problem -- 11 Convergence for Reflecting Boundaries, Singular Control and Ergodic Cost Problems -- 11.1 The Reflecting Boundary Problem -- 11.2 The Singular Control Problem -- 11.3 The Ergodic Cost Problem -- 12 Finite Time Problems and Nonlinear Filtering -- 12.1 The Explicit Approximation Method: An Example -- 12.2 The General Explicit Approximation Method -- 12.3 The Implicit Approximation Method: An Example -- 12.4 The General Implicit Approximation Method -- 12.5 The Optimal Control Problem: Approximations and Dynamic Programming Equations -- 12.6 Methods of Solution, Decomposition and Convergence -- 12.7 Nonlinear Filtering -- 13 Problems from the Calculus of Variations -- 13.1 Problems of Interest -- 13.2 Numerical Schemes and Convergence for the Finite Time Problem -- 13.3 Problems with a Controlled Stopping Time -- 13.4 Problems with a Discontinuous Running Cost -- 14 The Viscosity Solution Approach to Proving Convergence of Numerical Schemes -- 14.1 Definitions and Some Properties of Viscosity Solutions -- 14.2 Numerical Schemes -- 14.3 Proof of Convergence -- References -- List of Symbols."@en ;
   schema:exampleOfWork <http://worldcat.org/entity/work/id/24228596> ;
   schema:genre "Electronic books"@en ;
   schema:inLanguage "en" ;
   schema:isPartOf <http://experiment.worldcat.org/entity/work/data/24228596#Series/applications_of_mathematics_stochastic_modelling_and_applied_probability> ; # Applications of Mathematics, Stochastic Modelling and Applied Probability ;
   schema:isPartOf <http://worldcat.org/issn/0172-4568> ; # Applications of Mathematics, Stochastic Modelling and Applied Probability,
   schema:isSimilarTo <http://worldcat.org/entity/work/data/24228596#CreativeWork/> ;
   schema:name "Numerical Methods for Stochastic Control Problems in Continuous Time"@en ;
   schema:productID "851825515" ;
   schema:publication <http://www.worldcat.org/title/-/oclc/851825515#PublicationEvent/new_york_ny_springer_us_1992> ;
   schema:publisher <http://experiment.worldcat.org/entity/work/data/24228596#Agent/springer_us> ; # Springer US
   schema:url <http://public.eblib.com/choice/publicfullrecord.aspx?p=3082326> ;
   schema:url <https://link.springer.com/openurl?genre=book&isbn=978-0-387-97834-5> ;
   schema:url <http://dx.doi.org/10.1007/978-1-4684-0441-8> ;
   schema:url <http://link.springer.com/10.1007/978-1-4684-0441-8> ;
   schema:workExample <http://worldcat.org/isbn/9783540978343> ;
   schema:workExample <http://dx.doi.org/10.1007/978-1-4684-0441-8> ;
   schema:workExample <http://worldcat.org/isbn/9780387978345> ;
   schema:workExample <http://worldcat.org/isbn/9781468404418> ;
   wdrs:describedby <http://www.worldcat.org/title/-/oclc/851825515> ;
    .


Related Entities

<http://experiment.worldcat.org/entity/work/data/24228596#Series/applications_of_mathematics_stochastic_modelling_and_applied_probability> # Applications of Mathematics, Stochastic Modelling and Applied Probability ;
    a bgn:PublicationSeries ;
   schema:hasPart <http://www.worldcat.org/oclc/851825515> ; # Numerical Methods for Stochastic Control Problems in Continuous Time
   schema:name "Applications of Mathematics, Stochastic Modelling and Applied Probability ;" ;
    .

<http://id.loc.gov/authorities/subjects/sh85082127> # Mathematical optimization
    a schema:Intangible ;
   schema:name "Mathematical optimization"@en ;
    .

<http://id.loc.gov/authorities/subjects/sh85082139> # Mathematics
    a schema:Intangible ;
   schema:name "Mathematics"@en ;
    .

<http://id.loc.gov/authorities/subjects/sh85093237> # Numerical analysis
    a schema:Intangible ;
   schema:name "Numerical analysis"@en ;
    .

<http://id.worldcat.org/fast/1012099> # Mathematical optimization
    a schema:Intangible ;
   schema:name "Mathematical optimization"@en ;
    .

<http://id.worldcat.org/fast/1012163> # Mathematics
    a schema:Intangible ;
   schema:name "Mathematics"@en ;
    .

<http://id.worldcat.org/fast/1041273> # Numerical analysis
    a schema:Intangible ;
   schema:name "Numerical analysis"@en ;
    .

<http://id.worldcat.org/fast/895600> # Distribution (Probability theory)
    a schema:Intangible ;
   schema:name "Distribution (Probability theory)"@en ;
    .

<http://link.springer.com/10.1007/978-1-4684-0441-8>
   rdfs:comment "from Springer" ;
   rdfs:comment "(Unlimited Concurrent Users)" ;
    .

<http://viaf.org/viaf/84606868> # Harold J. Kushner
    a schema:Person ;
   schema:familyName "Kushner" ;
   schema:givenName "Harold J." ;
   schema:name "Harold J. Kushner" ;
    .

<http://viaf.org/viaf/9035984> # Paul G. Dupuis
    a schema:Person ;
   schema:familyName "Dupuis" ;
   schema:givenName "Paul G." ;
   schema:name "Paul G. Dupuis" ;
    .

<http://worldcat.org/entity/work/data/24228596#CreativeWork/>
    a schema:CreativeWork ;
   schema:description "Print version:" ;
   schema:isSimilarTo <http://www.worldcat.org/oclc/851825515> ; # Numerical Methods for Stochastic Control Problems in Continuous Time
    .

<http://worldcat.org/isbn/9780387978345>
    a schema:ProductModel ;
   schema:isbn "0387978348" ;
   schema:isbn "9780387978345" ;
    .

<http://worldcat.org/isbn/9781468404418>
    a schema:ProductModel ;
   schema:isbn "1468404415" ;
   schema:isbn "9781468404418" ;
    .

<http://worldcat.org/isbn/9783540978343>
    a schema:ProductModel ;
   schema:isbn "3540978348" ;
   schema:isbn "9783540978343" ;
    .

<http://worldcat.org/issn/0172-4568> # Applications of Mathematics, Stochastic Modelling and Applied Probability,
    a bgn:PublicationSeries ;
   schema:hasPart <http://www.worldcat.org/oclc/851825515> ; # Numerical Methods for Stochastic Control Problems in Continuous Time
   schema:issn "0172-4568" ;
   schema:name "Applications of Mathematics, Stochastic Modelling and Applied Probability," ;
    .


Content-negotiable representations

Close Window

Please sign in to WorldCat 

Don't have an account? You can easily create a free account.