Abstract: Offline reinforcement learning leverages static, pre-collected historical datasets to learn optimal policies, effectively avoiding the high costs and potential risks caused by online interaction with the environment. It demonstrates broad application prospects in safety-critical fields such as robotics, autonomous driving, and healthcare. However, during the learning process, offline reinforcement learning methods typically suffer from unstable policy learning due to extrapolation errors, and the generalization ability of the learned policy is limited by conservative constraints. To address this core issue, this paper proposes an offline Actor-Critic method based on Q-value Bellman error regularization (ACBER). ACBER explicitly introduces a dynamic Bellman error constraint mechanism during the policy improvement phase, adaptively adjusting policy updates based on the reliability of Q-value estimation. Specifically, when the Q-value estimation exhibits significant bias, ACBER automatically suppresses the policy’s excessive exploration toward uncertain regions, thereby effectively stabilizing the policy learning process and reducing the misguiding impact of policy evaluation errors on policy updates. Experimental results on the D4RL benchmark show that, compared with mainstream baselines, ACBER achieves performance improvements ranging from 2.81% to 36.94% on the Gym-MuJoCo locomotion control tasks, and exhibits superior suboptimal trajectory stitching capability on the antmaze sparse-reward navigation tasks, thereby validating its effectiveness in enhancing policy generalization.