skip to content
Markov decision processes in practice Preview this item
ClosePreview this item
Checking...

Markov decision processes in practice

Author: R J Boucherie; N M van Dijk
Publisher: Cham, Switzerland : Springer, [2017] ©2017
Series: International series in operations research & management science, v. 248.
Edition/Format:   eBook : Document : EnglishView all editions and formats
Summary:
This book presents classical Markov Decision Processes (MDP) for real-life applications and optimization. MDP allows users to develop and formally support approximate and simple decision rules, and this book showcases state-of-the-art applications in which MDP was key to the solution approach. The book is divided into six parts. Part 1 is devoted to the state-of-the-art theoretical foundation of MDP, including  Read more...
Rating:

(not yet rated) 0 with reviews - Be the first.

Subjects
More like this

 

Find a copy online

Links to this item

Find a copy in the library

&AllPage.SpinnerRetrieving; Finding libraries that hold this item...

Details

Genre/Form: Electronic books
Additional Physical Format: Print version:
Markov decision processes in practice.
Cham, Switzerland : Springer, [2017]
(OCoLC)959033672
Material Type: Document, Internet resource
Document Type: Internet Resource, Computer File
All Authors / Contributors: R J Boucherie; N M van Dijk
ISBN: 9783319477664 3319477668
OCLC Number: 975486905
Description: 1 online resource.
Contents: Foreword; Preface; Part I: General Theory; Part II: Healthcare; Part III: Transportation; Part IV: Production; Part V: Communications; Part VI: Financial Modeling; Summarizing; Acknowledgments; Contents; List of Contributors ; Part I General Theory; 1 One-Step Improvement Ideas and Computational Aspects; 1.1 Introduction; 1.2 The Average-Cost Markov Decision Model; 1.2.1 The Concept of Relative Values; 1.2.2 The Policy-Improvement Step; 1.2.3 The Odoni Bounds for Value Iteration; 1.3 Tailor-Made Policy-Iteration Algorithm; 1.3.1 A Queueing Control Problem with a Variable Service Rate. 1.4 One-Step Policy Improvement for Suboptimal Policies; 1.4.1 Dynamic Routing of Customers to Parallel Queues; 1.5 One-Stage-Look-Ahead Rule in Optimal Stopping; 1.5.1 Devil's Penny Problem; 1.5.2 A Game of Dropping Balls into Bins; 1.5.3 The Chow-Robbins Game; References; 2 Value Function Approximation in Complex Queueing Systems; 2.1 Introduction; 2.2 Difference Calculus for Markovian Birth-Death Systems; 2.3 Value Functions for Queueing Systems; 2.3.1 The M/Cox(r)/1 Queue; 2.3.2 Special Cases of the M/Cox(r)/1 Queue; 2.3.3 The M/M/s Queue; 2.3.4 The Blocking Costs in an M/M/s/s Queue. 2.3.5 Priority Queues; 2.4 Application: Routing to Parallel Queues; 2.5 Application: Dynamic Routing in Multiskill Call Centers; 2.6 Application: A Controlled Polling System; References; 3 Approximate Dynamic Programming by Practical Examples; 3.1 Introduction; 3.2 The Nomadic Trucker Example; 3.2.1 Problem Introduction; 3.2.2 MDP Model; 3.2.2.1 State; 3.2.2.2 Decision; 3.2.2.3 Costs; 3.2.2.4 New Information and Transition Function; 3.2.2.5 Solution; 3.2.3 Approximate Dynamic Programming; 3.2.3.1 Post-decision State; 3.2.3.2 Forward Dynamic Programming; 3.2.3.3 Value Function Approximation. 3.3 A Freight Consolidation Example; 3.3.1 Problem Introduction; 3.3.2 MDP Model; 3.3.2.1 State; 3.3.2.2 Decision; 3.3.2.3 Costs; 3.3.2.4 New Information and Transition Function; 3.3.2.5 Solution; 3.3.3 Approximate Dynamic Programming; 3.3.3.1 Post-decision State; 3.3.3.2 Forward Dynamic Programming; 3.3.3.3 Value Function Approximation; 3.4 A Healthcare Example; 3.4.1 Problem Introduction; 3.4.2 MDP Model; 3.4.2.1 State; 3.4.2.2 Decision; 3.4.2.3 Costs; 3.4.2.4 New Information and Transition Function; 3.4.2.5 Solution; 3.4.3 Approximate Dynamic Programming; 3.4.3.1 Post-decision State. 3.4.3.2 Forward Dynamic Programming; 3.4.3.3 Value Function Approximation; 3.5 What's More; 3.5.1 Policies; 3.5.2 Value Function Approximations; 3.5.3 Exploration vs Exploitation; Appendix; References; 4 Server Optimization of Infinite Queueing Systems; 4.1 Introduction; 4.2 Basic Definition and Notations; 4.3 Motivating Examples; 4.3.1 Optimization of a Queueing System with Two Different Servers; 4.3.2 Optimization of a Computational System with Power Saving Mode; 4.3.3 Structural Properties of These Motivating Examples; 4.4 Theoretical Background; 4.4.1 Subset Measures in Markov Chains.
Series Title: International series in operations research & management science, v. 248.
Responsibility: Richard J. Boucherie, Nico M. van Dijk.

Abstract:

This book presents classical Markov Decision Processes (MDP) for real-life applications and optimization. MDP allows users to develop and formally support approximate and simple decision rules, and  Read more...

Reviews

User-contributed reviews
Retrieving GoodReads reviews...
Retrieving DOGObooks reviews...

Tags

Be the first.
Confirm this request

You may have already requested this item. Please select Ok if you would like to proceed with this request anyway.

Linked Data


Primary Entity

<http://www.worldcat.org/oclc/975486905> # Markov decision processes in practice
    a schema:CreativeWork, schema:Book, schema:MediaObject ;
    library:oclcnum "975486905" ;
    library:placeOfPublication <http://id.loc.gov/vocabulary/countries/sz> ;
    schema:about <http://experiment.worldcat.org/entity/work/data/3882198227#Topic/markov_processes> ; # Markov processes
    schema:about <http://experiment.worldcat.org/entity/work/data/3882198227#Topic/mathematics_probability_&_statistics_general> ; # MATHEMATICS--Probability & Statistics--General
    schema:about <http://dewey.info/class/519.233/e23/> ;
    schema:about <http://experiment.worldcat.org/entity/work/data/3882198227#Topic/decision_making> ; # Decision making
    schema:about <http://experiment.worldcat.org/entity/work/data/3882198227#Topic/mathematics_applied> ; # MATHEMATICS--Applied
    schema:bookFormat schema:EBook ;
    schema:datePublished "2017" ;
    schema:description "This book presents classical Markov Decision Processes (MDP) for real-life applications and optimization. MDP allows users to develop and formally support approximate and simple decision rules, and this book showcases state-of-the-art applications in which MDP was key to the solution approach. The book is divided into six parts. Part 1 is devoted to the state-of-the-art theoretical foundation of MDP, including approximate methods such as policy improvement, successive approximation and infinite state spaces as well as an instructive chapter on Approximate Dynamic Programming. It then continues with five parts of specific and non-exhaustive application areas. Part 2 covers MDP healthcare applications, which includes different screening procedures, appointment scheduling, ambulance scheduling and blood management. Part 3 explores MDP modeling within transportation. This ranges from public to private transportation, from airports and traffic lights to car parking or charging your electric car. Part 4 contains three chapters that illustrates the structure of approximate policies for production or manufacturing structures. In Part 5, communications is highlighted as an important application area for MDP. It includes Gittins indices, down-to-earth call centers and wireless sensor networks. Finally Part 6 is dedicated to financial modeling, offering an instructive review to account for financial portfolios and derivatives under proportional transactional costs. The MDP applications in this book illustrate a variety of both standard and non-standard aspects of MDP modeling and its practical use. This book should appeal to readers for practitioning, academic research and educational purposes, with a background in, among others, operations research, mathematics, computer science, and industrial engineering."@en ;
    schema:description "Foreword; Preface; Part I: General Theory; Part II: Healthcare; Part III: Transportation; Part IV: Production; Part V: Communications; Part VI: Financial Modeling; Summarizing; Acknowledgments; Contents; List of Contributors ; Part I General Theory; 1 One-Step Improvement Ideas and Computational Aspects; 1.1 Introduction; 1.2 The Average-Cost Markov Decision Model; 1.2.1 The Concept of Relative Values; 1.2.2 The Policy-Improvement Step; 1.2.3 The Odoni Bounds for Value Iteration; 1.3 Tailor-Made Policy-Iteration Algorithm; 1.3.1 A Queueing Control Problem with a Variable Service Rate."@en ;
    schema:editor <http://experiment.worldcat.org/entity/work/data/3882198227#Person/boucherie_r_j_richard_j_1964> ; # Richard J. Boucherie
    schema:editor <http://experiment.worldcat.org/entity/work/data/3882198227#Person/dijk_n_m_van> ; # N. M. van Dijk
    schema:exampleOfWork <http://worldcat.org/entity/work/id/3882198227> ;
    schema:genre "Electronic books"@en ;
    schema:inLanguage "en" ;
    schema:isPartOf <http://experiment.worldcat.org/entity/work/data/3882198227#Series/international_series_in_operations_research_&_management_science> ; # International series in operations research & management science ;
    schema:isSimilarTo <http://www.worldcat.org/oclc/959033672> ;
    schema:name "Markov decision processes in practice"@en ;
    schema:productID "975486905" ;
    schema:url <https://0-link-springer-com.pugwash.lib.warwick.ac.uk/book/10.1007/978-3-319-47766-4> ;
    schema:url <http://dx.doi.org/10.1007/978-3-319-47766-4> ;
    schema:url <http://lib.myilibrary.com?id=1000843> ;
    schema:url <http://sfx.carli.illinois.edu/sfxwhe/sfx_local?genre=book&sid=Voyager:WHE&sfx.ignore_date_threshold=1&svc.fulltext=yes&rft.isbn=978-3-319-47766-4> ;
    schema:url <https://link.springer.com/openurl?genre=book&isbn=978-3-319-47764-0> ;
    schema:url <http://libproxy.uwinnipeg.ca/login?url=http://link.springer.com/10.1007/978-3-319-47766-4> ;
    schema:url <http://search.ebscohost.com/login.aspx?direct=true&scope=site&db=nlebk&db=nlabk&AN=1383916> ;
    schema:url <http://link.springer.com/10.1007/978-3-319-47766-4> ;
    schema:url <http://link.springer.com/openurl?genre=book&isbn=978-3-319-47764-0> ;
    schema:url <http://www.myilibrary.com?id=1000843> ;
    schema:url <https://grinnell.idm.oclc.org/login?url=http://link.springer.com/10.1007/978-3-319-47766-4> ;
    schema:url <http://public.eblib.com/choice/publicfullrecord.aspx?p=4822024> ;
    schema:workExample <http://worldcat.org/isbn/9783319477664> ;
    schema:workExample <http://dx.doi.org/10.1007/978-3-319-47766-4> ;
    wdrs:describedby <http://www.worldcat.org/title/-/oclc/975486905> ;
    .


Related Entities

<http://experiment.worldcat.org/entity/work/data/3882198227#Person/boucherie_r_j_richard_j_1964> # Richard J. Boucherie
    a schema:Person ;
    schema:birthDate "1964" ;
    schema:familyName "Boucherie" ;
    schema:givenName "Richard J." ;
    schema:givenName "R. J." ;
    schema:name "Richard J. Boucherie" ;
    .

<http://experiment.worldcat.org/entity/work/data/3882198227#Person/dijk_n_m_van> # N. M. van Dijk
    a schema:Person ;
    schema:familyName "Dijk" ;
    schema:givenName "N. M. van" ;
    schema:name "N. M. van Dijk" ;
    .

<http://experiment.worldcat.org/entity/work/data/3882198227#Series/international_series_in_operations_research_&_management_science> # International series in operations research & management science ;
    a bgn:PublicationSeries ;
    schema:hasPart <http://www.worldcat.org/oclc/975486905> ; # Markov decision processes in practice
    schema:name "International series in operations research & management science ;" ;
    .

<http://experiment.worldcat.org/entity/work/data/3882198227#Topic/mathematics_applied> # MATHEMATICS--Applied
    a schema:Intangible ;
    schema:name "MATHEMATICS--Applied"@en ;
    .

<http://experiment.worldcat.org/entity/work/data/3882198227#Topic/mathematics_probability_&_statistics_general> # MATHEMATICS--Probability & Statistics--General
    a schema:Intangible ;
    schema:name "MATHEMATICS--Probability & Statistics--General"@en ;
    .

<http://lib.myilibrary.com?id=1000843>
    rdfs:comment "Connect to MyiLibrary resource." ;
    .

<http://worldcat.org/isbn/9783319477664>
    a schema:ProductModel ;
    schema:isbn "3319477668" ;
    schema:isbn "9783319477664" ;
    .

<http://www.worldcat.org/oclc/959033672>
    a schema:CreativeWork ;
    rdfs:label "Markov decision processes in practice." ;
    schema:description "Print version:" ;
    schema:isSimilarTo <http://www.worldcat.org/oclc/975486905> ; # Markov decision processes in practice
    .


Content-negotiable representations

Close Window

Please sign in to WorldCat 

Don't have an account? You can easily create a free account.