China’s New Guardrails for Emotion-Driven AI Agents Demand Addiction Controls, Minor Consent, and Real-Time Intervention

The regulations focus on emotion-driven generative AI agents—systems designed to sustain user engagement by simulating empathy, companionship, or ongoing interaction. This distinguishes them from typical productivity tools or single-shot chat features, because the concern is behavioral influence over time, not just content generation.

Addiction controls generally refer to limiting techniques that could encourage compulsive use. That may include throttling or pausing certain interaction loops, monitoring engagement patterns associated with dependency, and implementing safety responses when a user shows risk signals. The aim is to reduce runaway usage rather than merely adding generic content filters.

For users who may be minors, the rules emphasize tighter consent requirements, such as obtaining guardian approval where appropriate and using safeguards that restrict emotionally persuasive features. Providers are expected to verify age more carefully and adjust the system’s behavior to reduce the chance of manipulation or reliance in younger users.

Real-time intervention means the system needs to detect problematic states during active interaction and respond immediately—rather than only logging issues for later review. Examples could include interrupting escalating emotional dependence, providing warnings, offering alternative actions, or triggering human-assisted support when certain risk patterns appear.

While exact mechanisms can vary, the direction of the rules suggests ongoing compliance duties: documenting how the agent manages engagement, how it handles minor users, and how intervention triggers operate. Regulators can also expect auditability of safety logs and the ability to demonstrate that safeguards work as intended under real usage.

Violations could lead to restrictions on deployment, required remediation, or penalties depending on severity and prior compliance history. The core idea is that emotional and dependency-related harms are treated as a risk category, so providers may face heightened scrutiny if their systems fail to implement meaningful addiction controls or timely intervention.