Skip to content
Cart

Your Cart

×

You have 0 items in your cart.

Register Sign in Wishlist

Partially Observed Markov Decision Processes
From Filtering to Controlled Sensing

£59.99

textbook
  • Date Published: March 2016
  • availability: In stock
  • format: Hardback
  • isbn: 9781107134607

£ 59.99
Hardback

Add to cart Add to wishlist

Other available formats:
eBook


Request inspection copy

Lecturers may request a copy of this title for inspection

Description
Product filter button
Description
Contents
Resources
Courses
About the Authors
  • Covering formulation, algorithms, and structural results, and linking theory to real-world applications in controlled sensing (including social learning, adaptive radars and sequential detection), this book focuses on the conceptual foundations of partially observed Markov decision processes (POMDPs). It emphasizes structural results in stochastic dynamic programming, enabling graduate students and researchers in engineering, operations research, and economics to understand the underlying unifying themes without getting weighed down by mathematical technicalities. Bringing together research from across the literature, the book provides an introduction to nonlinear filtering followed by a systematic development of stochastic dynamic programming, lattice programming and reinforcement learning for POMDPs. Questions addressed in the book include: when does a POMDP have a threshold optimal policy? When are myopic policies optimal? How do local and global decision makers interact in adaptive decision making in multi-agent social learning where there is herding and data incest? And how can sophisticated radars and sensors adapt their sensing in real time?

    • Links theory to real-world applications in controlled sensing
    • Brings together results from across the literature and from multiple different disciplines
    • Includes several new topics, such as reciprocal processes, geometric ergodicity of the optimal filter, data incest and social learning
    Read more

    Customer reviews

    Not yet reviewed

    Be the first to review

    Review was not posted due to profanity

    ×

    , create a review

    (If you're not , sign out)

    Please enter the right captcha value
    Please enter a star rating.
    Your review must be a minimum of 12 words.

    How do you rate this item?

    ×

    Product details

    • Date Published: March 2016
    • format: Hardback
    • isbn: 9781107134607
    • length: 488 pages
    • dimensions: 254 x 180 x 25 mm
    • weight: 1.15kg
    • contains: 47 b/w illus. 5 tables
    • availability: In stock
  • Table of Contents

    Preface
    1. Introduction
    Part I. Stochastic Models and Bayesian Filtering:
    2. Stochastic state-space models
    3. Optimal filtering
    4. Algorithms for maximum likelihood parameter estimation
    5. Multi-agent sensing: social learning and data incest
    Part II. Partially Observed Markov Decision Processes. Models and Algorithms:
    6. Fully observed Markov decision processes
    7. Partially observed Markov decision processes (POMDPs)
    8. POMDPs in controlled sensing and sensor scheduling
    Part III. Partially Observed Markov Decision Processes:
    9. Structural results for Markov decision processes
    10. Structural results for optimal filters
    11. Monotonicity of value function for POMPDs
    12. Structural results for stopping time POMPDs
    13. Stopping time POMPDs for quickest change detection
    14. Myopic policy bounds for POMPDs and sensitivity to model parameters
    Part IV. Stochastic Approximation and Reinforcement Learning:
    15. Stochastic optimization and gradient estimation
    16. Reinforcement learning
    17. Stochastic approximation algorithms: examples
    18. Summary of algorithms for solving POMPDs
    Appendix A. Short primer on stochastic simulation
    Appendix B. Continuous-time HMM filters
    Appendix C. Markov processes
    Appendix D. Some limit theorems
    Bibliography
    Index.

  • Resources for

    Partially Observed Markov Decision Processes

    Vikram Krishnamurthy

    General Resources

    Welcome to the resources site

    Here you will find free-of-charge online materials to accompany this book. The range of materials we provide across our academic and higher education titles are an integral part of the book package whether you are a student, instructor, researcher or professional.

    Find resources associated with this title

    Type Name Unlocked * Format Size

    Showing of

    Back to top

    *This title has one or more locked files and access is given only to lecturers adopting the textbook for their class. We need to enforce this strictly so that solutions are not made available to students. To gain access to locked resources you either need first to sign in or register for an account.


    These resources are provided free of charge by Cambridge University Press with permission of the author of the corresponding work, but are subject to copyright. You are permitted to view, print and download these resources for your own personal use only, provided any copyright lines on the resources are not removed or altered in any way. Any other use, including but not limited to distribution of the resources in modified form, or via electronic or other media, is strictly prohibited unless you have permission from the author of the corresponding work and provided you give appropriate acknowledgement of the source.

    If you are having problems accessing these resources please email lecturers@cambridge.org

  • Author

    Vikram Krishnamurthy, Cornell University/Cornell Tech
    Vikram Krishnamurthy is a Professor and Canada Research Chair in Statistical Signal Processing at the University of British Columbia, Vancouver. His research contributions focus on nonlinear filtering, stochastic approximation algorithms and POMDPs. Dr Krishnamurthy is a Fellow of the Institute of Electrical and Electronics Engineers (IEEE) and served as a distinguished lecturer for the IEEE Signal Processing Society. In 2013, he received an honorary doctorate from KTH, Royal Institute of Technology, Sweden.

Sign In

Please sign in to access your account

Cancel

Not already registered? Create an account now. ×

Sorry, this resource is locked

Please register or sign in to request access. If you are having problems accessing these resources please email lecturers@cambridge.org

Register Sign in
Please note that this file is password protected. You will be asked to input your password on the next screen.

» Proceed

You are now leaving the Cambridge University Press website. Your eBook purchase and download will be completed by our partner www.ebooks.com. Please see the permission section of the www.ebooks.com catalogue page for details of the print & copy limits on our eBooks.

Continue ×

Continue ×

Continue ×

Find content that relates to you

Are you sure you want to delete your account?

This cannot be undone.

Cancel

Thank you for your feedback which will help us improve our service.

If you requested a response, we will make sure to get back to you shortly.

×
Please fill in the required fields in your feedback submission.
×