Partially observable Markov decision process
Generalization of a Markov decision process
📊 Rating
2 news mentions · 👍 0 likes · 👎 0 dislikes
📌 Topics
- Artificial Intelligence (2)
- Computational Theory (1)
- Formal Methods (1)
- Machine Learning (1)
- Robotics (1)
🏷️ Keywords
POMDP (2) · Markov Decision Processes (2) · arXiv (2) · Reachability Value (1) · Computational Complexity (1) · Robotics (1) · Algorithm Design (1) · Deep Reinforcement Learning (1) · Lexpop Framework (1) · Finite-State Controllers (1) · Neural Networks (1)
📖 Key Information
# Partially Observable Markov Decision Process (POMDP)
A **Partially Observable Markov Decision Process (POMDP)** is a mathematical framework for modeling decision-making under uncertainty. It serves as a generalization of the **Markov Decision Process (MDP)**.
### Core Concept
In a standard MDP, the agent is assumed to have full knowledge of the current environment state. In contrast, a POMDP models scenarios where the system dynamics are governed by an MDP, but the agent **cannot directly observe** the underlying state. Instead, the agent must infer the state based on incomplete or noisy sensory data.
### Key Components and Mechanisms
To navigate the environment effectively, a POMDP agent utilizes two primary models:
* **System Dynamics:** The underlying MDP that governs state transitions and rewards.
* **Sensor Model:** A probability distribution that defines the likelihood of receiving specific observations given the current underlying state.
### Policy and Decision-Making
The fundamental difference between an MDP and a POMDP lies in the derivation of the optimal action:
* **MDP Policy:** A function that maps the **direct state** to an action: $\pi(s) \to a$.
* **POMDP Policy:** Because the state is hidden, the policy maps the **history of observations** or a **belief state** (a probability distribution over all possible states) to an action: $\pi(b) \to a$.
### Applications
POMDPs are essential in fields where information is imperfect, such as robotics (navigating with noisy sensors), autonomous vehicle control, machine maintenance, and cognitive science.
📰 Related News (2)
-
🇺🇸 Computing the Reachability Value of Posterior-Deterministic POMDPs
arXiv:2602.07473v1 Announce Type: new Abstract: Partially observable Markov decision processes (POMDPs) are a fundamental model for sequential decisi...
-
🇺🇸 Finite-State Controllers for (Hidden-Model) POMDPs using Deep Reinforcement Learning
arXiv:2602.08734v1 Announce Type: new Abstract: Solving partially observable Markov decision processes (POMDPs) requires computing policies under imp...
🔗 Entity Intersection Graph
People and organizations frequently mentioned alongside Partially observable Markov decision process:
- 🌐 Markov decision process (2 shared articles)
- 🌐 Neural network (1 shared articles)
- 🌐 Computational complexity (1 shared articles)
- 🌐 Robotics (1 shared articles)