skip to content
Examples in Markov Decision Processes. Preview this item
ClosePreview this item
Checking...

Examples in Markov Decision Processes.

Author: A B Piunovskiy
Publisher: Singapore : World Scientific Publishing Company, 2012.
Series: Imperial College Press optimization series.
Edition/Format:   eBook : Document : EnglishView all editions and formats
Summary:
This invaluable book provides approximately eighty examples illustrating the theory of controlled discrete-time Markov processes. Except for applications of the theory to real-life problems like stock exchange, queues, gambling, optimal search etc, the main attention is paid to counter-intuitive, unexpected properties of optimization problems. Such examples illustrate the importance of conditions imposed in the  Read more...
Rating:

(not yet rated) 0 with reviews - Be the first.

Subjects
More like this

 

Find a copy online

Links to this item

Find a copy in the library

&AllPage.SpinnerRetrieving; Finding libraries that hold this item...

Details

Genre/Form: Electronic books
Additional Physical Format: Print version:
Piunovskiy, A.B.
Examples in Markov Decision Processes.
Singapore : World Scientific Publishing Company, ©2012
Material Type: Document, Internet resource
Document Type: Internet Resource, Computer File
All Authors / Contributors: A B Piunovskiy
ISBN: 9781848167940 1848167946
OCLC Number: 830162389
Notes: 3.2.3 A non-optimal strategy for which v x solves the optimality equation.
Description: 1 online resource (308 pages).
Contents: Preface; 1. Finite-Horizon Models; 1.1 Preliminaries; 1.2 Model Description; 1.3 Dynamic Programming Approach; 1.4 Examples; 1.4.1 Non-transitivity of the correlation; 1.4.2 The more frequently used control is not better; 1.4.3 Voting; 1.4.4 The secretary problem; 1.4.5 Constrained optimization; 1.4.6 Equivalent Markov selectors in non-atomic MDPs; 1.4.7 Strongly equivalent Markov selectors in nonatomic MDPs; 1.4.8 Stock exchange; 1.4.9 Markov or non-Markov strategy? Randomized or not? When is the Bellman principle violated?; 1.4.10 Uniformly optimal, but not optimal strategy. 1.4.11 Martingales and the Bellman principle1.4.12 Conventions on expectation and infinities; 1.4.13 Nowhere-differentiable function vt(x); discontinuous function vt(x); 1.4.14 The non-measurable Bellman function; 1.4.15 No one strategy is uniformly -optimal; 1.4.16 Semi-continuous model; 2. Homogeneous Infinite-Horizon Models: Expected Total Loss; 2.1 Homogeneous Non-discounted Model; 2.2 Examples; 2.2.1 Mixed Strategies; 2.2.2 Multiple solutions to the optimality equation; 2.2.3 Finite model: multiple solutions to the optimality equation; conserving but not equalizing strategy. 2.2.4 The single conserving strategy is not equalizing and not optimal2.2.5 When strategy iteration is not successful; 2.2.6 When value iteration is not successful; 2.2.7 When value iteration is not successful: positive model I; 2.2.8 When value iteration is not successful: positive model II; 2.2.9 Value iteration and stability in optimal stopping problems; 2.2.10 A non-equalizing strategy is uniformly optimal; 2.2.11 A stationary uniformly -optimal selector does not exist (positive model); 2.2.12 A stationary uniformly -optimal selector does not exist (negative model). 2.2.13 Finite-action negative model where a stationary uniformly -optimal selector does not exist2.2.14 Nearly uniformly optimal selectors in negative models; 2.2.15 Semi-continuous models and the blackmailer's dilemma; 2.2.16 Not a semi-continuous model; 2.2.17 The Bellman function is non-measurable and no one strategy is uniformly -optimal; 2.2.18 A randomized strategy is better than any selector (finite action space); 2.2.19 The fluid approximation does not work; 2.2.20 The fluid approximation: refined model; 2.2.21 Occupation measures: phantom solutions. 2.2.22 Occupation measures in transient models2.2.23 Occupation measures and duality; 2.2.24 Occupation measures: compactness; 2.2.25 The bold strategy in gambling is not optimal (house limit); 2.2.26 The bold strategy in gambling is not optimal (inflation); 2.2.27 Search strategy for a moving target; 2.2.28 The three-way duel ("Truel"); 3. Homogeneous Infinite-Horizon Models: Discounted Loss; 3.1 Preliminaries; 3.2 Examples; 3.2.1 Phantom solutions of the optimality equation; 3.2.2 When value iteration is not successful: positive model.
Series Title: Imperial College Press optimization series.

Abstract:

This invaluable book provides approximately eighty examples illustrating the theory of controlled discrete-time Markov processes. Except for applications of the theory to real-life problems like stock exchange, queues, gambling, optimal search etc, the main attention is paid to counter-intuitive, unexpected properties of optimization problems. Such examples illustrate the importance of conditions imposed in the theorems on Markov Decision Processes. Many of the examples are based upon examples published earlier in journal articles or textbooks while several other examples are new. The aim was.

Reviews

User-contributed reviews
Retrieving GoodReads reviews...
Retrieving DOGObooks reviews...

Tags

Be the first.
Confirm this request

You may have already requested this item. Please select Ok if you would like to proceed with this request anyway.

Linked Data


Primary Entity

<http://www.worldcat.org/oclc/830162389> # Examples in Markov Decision Processes.
    a schema:Book, schema:CreativeWork, schema:MediaObject ;
   library:oclcnum "830162389" ;
   library:placeOfPublication <http://experiment.worldcat.org/entity/work/data/1167706822#Place/singapore> ; # Singapore
   library:placeOfPublication <http://id.loc.gov/vocabulary/countries/si> ;
   schema:about <http://dewey.info/class/519.233/> ;
   schema:about <http://experiment.worldcat.org/entity/work/data/1167706822#Topic/mathematics_probability_&_statistics_stochastic_processes> ; # MATHEMATICS--Probability & Statistics--Stochastic Processes
   schema:about <http://id.worldcat.org/fast/1010347> ; # Markov processes
   schema:bookFormat schema:EBook ;
   schema:creator <http://viaf.org/viaf/279191359> ; # A. B. Piunovskiy
   schema:datePublished "2012" ;
   schema:description "This invaluable book provides approximately eighty examples illustrating the theory of controlled discrete-time Markov processes. Except for applications of the theory to real-life problems like stock exchange, queues, gambling, optimal search etc, the main attention is paid to counter-intuitive, unexpected properties of optimization problems. Such examples illustrate the importance of conditions imposed in the theorems on Markov Decision Processes. Many of the examples are based upon examples published earlier in journal articles or textbooks while several other examples are new. The aim was."@en ;
   schema:description "Preface; 1. Finite-Horizon Models; 1.1 Preliminaries; 1.2 Model Description; 1.3 Dynamic Programming Approach; 1.4 Examples; 1.4.1 Non-transitivity of the correlation; 1.4.2 The more frequently used control is not better; 1.4.3 Voting; 1.4.4 The secretary problem; 1.4.5 Constrained optimization; 1.4.6 Equivalent Markov selectors in non-atomic MDPs; 1.4.7 Strongly equivalent Markov selectors in nonatomic MDPs; 1.4.8 Stock exchange; 1.4.9 Markov or non-Markov strategy? Randomized or not? When is the Bellman principle violated?; 1.4.10 Uniformly optimal, but not optimal strategy."@en ;
   schema:exampleOfWork <http://worldcat.org/entity/work/id/1167706822> ;
   schema:genre "Electronic books"@en ;
   schema:inLanguage "en" ;
   schema:isPartOf <http://experiment.worldcat.org/entity/work/data/1167706822#Series/imperial_college_press_optimization_series> ; # Imperial College Press optimization series.
   schema:isSimilarTo <http://worldcat.org/entity/work/data/1167706822#CreativeWork/examples_in_markov_decision_processes> ;
   schema:name "Examples in Markov Decision Processes."@en ;
   schema:productID "830162389" ;
   schema:publication <http://www.worldcat.org/title/-/oclc/830162389#PublicationEvent/singapore_world_scientific_publishing_company_2012> ;
   schema:publisher <http://experiment.worldcat.org/entity/work/data/1167706822#Agent/world_scientific_publishing_company> ; # World Scientific Publishing Company
   schema:url <http://ebookcentral.proquest.com/lib/ucm/detail.action?docID=1143323> ;
   schema:url <http://ebookcentral.proquest.com/lib/concordiaab-ebooks/detail.action?docID=1143323> ;
   schema:url <http://search.ebscohost.com/login.aspx?direct=true&scope=site&db=nlebk&db=nlabk&AN=545467> ;
   schema:url <http://search.ebscohost.com/login.aspx?direct=true&scope=site&db=e000xna&AN=545467> ;
   schema:url <http://public.eblib.com/choice/publicfullrecord.aspx?p=1143323> ;
   schema:url <http://ebookcentral.proquest.com/lib/gprc-ebooks/detail.action?docID=1143323> ;
   schema:url <http://ebookcentral.proquest.com/lib/rdc/detail.action?docID=1143323> ;
   schema:url <http://ebookcentral.proquest.com/lib/columbia/detail.action?docID=1143323> ;
   schema:url <http://ebookcentral.proquest.com/lib/macewan-ebooks/detail.action?docID=1143323> ;
   schema:workExample <http://worldcat.org/isbn/9781848167940> ;
   wdrs:describedby <http://www.worldcat.org/title/-/oclc/830162389> ;
    .


Related Entities

<http://ebookcentral.proquest.com/lib/concordiaab-ebooks/detail.action?docID=1143323>
   rdfs:comment "from Ebook Central Academic Complete" ;
   rdfs:comment "(Unlimited Concurrent Users)" ;
    .

<http://ebookcentral.proquest.com/lib/gprc-ebooks/detail.action?docID=1143323>
   rdfs:comment "from Ebook Central Academic Complete" ;
   rdfs:comment "(Unlimited Concurrent Users)" ;
    .

<http://ebookcentral.proquest.com/lib/macewan-ebooks/detail.action?docID=1143323>
   rdfs:comment "from Ebook Central Academic Complete" ;
   rdfs:comment "(Unlimited Concurrent Users)" ;
    .

<http://ebookcentral.proquest.com/lib/rdc/detail.action?docID=1143323>
   rdfs:comment "from Ebook Central Academic Complete" ;
   rdfs:comment "(Unlimited Concurrent Users)" ;
    .

<http://experiment.worldcat.org/entity/work/data/1167706822#Agent/world_scientific_publishing_company> # World Scientific Publishing Company
    a bgn:Agent ;
   schema:name "World Scientific Publishing Company" ;
    .

<http://experiment.worldcat.org/entity/work/data/1167706822#Series/imperial_college_press_optimization_series> # Imperial College Press optimization series.
    a bgn:PublicationSeries ;
   schema:hasPart <http://www.worldcat.org/oclc/830162389> ; # Examples in Markov Decision Processes.
   schema:name "Imperial College Press optimization series." ;
   schema:name "Imperial College Press Optimization Series ;" ;
    .

<http://experiment.worldcat.org/entity/work/data/1167706822#Topic/mathematics_probability_&_statistics_stochastic_processes> # MATHEMATICS--Probability & Statistics--Stochastic Processes
    a schema:Intangible ;
   schema:name "MATHEMATICS--Probability & Statistics--Stochastic Processes"@en ;
    .

<http://id.worldcat.org/fast/1010347> # Markov processes
    a schema:Intangible ;
   schema:name "Markov processes"@en ;
    .

<http://search.ebscohost.com/login.aspx?direct=true&scope=site&db=e000xna&AN=545467>
   rdfs:comment "from EBSCO Academic Collection" ;
   rdfs:comment "(Unlimited Concurrent Users)" ;
    .

<http://viaf.org/viaf/279191359> # A. B. Piunovskiy
    a schema:Person ;
   schema:familyName "Piunovskiy" ;
   schema:givenName "A. B." ;
   schema:name "A. B. Piunovskiy" ;
    .

<http://worldcat.org/entity/work/data/1167706822#CreativeWork/examples_in_markov_decision_processes>
    a schema:CreativeWork ;
   rdfs:label "Examples in Markov Decision Processes." ;
   schema:description "Print version:" ;
   schema:isSimilarTo <http://www.worldcat.org/oclc/830162389> ; # Examples in Markov Decision Processes.
    .

<http://worldcat.org/isbn/9781848167940>
    a schema:ProductModel ;
   schema:isbn "1848167946" ;
   schema:isbn "9781848167940" ;
    .


Content-negotiable representations

Close Window

Please sign in to WorldCat 

Don't have an account? You can easily create a free account.