Safe-Support Q-Learning: Learning without Unsafe Exploration

Most safe RL research accepts that some unsafe exploration is unavoidable and builds penalty or constraint mechanisms to limit damage after the fact. This work's contribution is the claim that the behavior policy can be structurally confined to a safe support region before any environment interaction, which is a different problem formulation, not just a tighter constraint.

The timing here is notable. Published the same day as 'Biased Dreams: Limitations to Epistemic Uncertainty Quantification in Latent Space Models,' that paper identifies a specific failure mode where uncertainty signals meant to guide safe exploration are corrupted by attractor bias in learned representations. Safe-Support Q-Learning sidesteps that class of problem entirely by not relying on uncertainty estimates to police exploration at all. The two papers are effectively addressing the same deployment risk from opposite directions: one exposing how uncertainty-guided safety breaks down, the other proposing a framework that does not depend on uncertainty guidance in the first place. Together they reinforce a growing concern that uncertainty quantification alone is not a reliable foundation for safe RL in real deployments.

The critical test is whether the predefined safe region assumption holds in domains with partially observable or dynamically shifting safety boundaries, such as robotic manipulation with moving obstacles. If the authors or follow-on work demonstrate benchmark results in those settings without requiring a static safety specification, the approach has genuine deployment reach; otherwise it remains constrained to environments where safe regions can be fully enumerated in advance.