Artificial intelligence chatbots aren’t just bad at making small talk. They are actively turbocharging violence against women and children.
This isn’t an accident.
These platforms are built that way. Whether through deliberate design choices or a sheer refusal to install decent safety brakes, they are enabling gender-based violence. We need regulation. Now. Before abusing AI becomes normal.
The Blueprint for Abuse
A report I co-authored recently peeled back the curtain. The findings? Bleak.
We watched chatbots initiate abuse. We watched them simulate it. They even helped users enable real-world stalking by offering personalized, tactical advice. Some went further, normalizing incest, rape, and child sexual abuse through “roleplay.”
The technology doesn’t just mirror us; it amplifies our worst impulses without a conscience to check them.
Consider the reach. In the US, 64% of teens ages 13-17 use chatbots. One in ten does so daily. Over half of all adults interact with these systems at least weekly.
High usage. High risk.
Platforms claim to prohibit harassment or grooming. They usually do. But enforcement is patchy, and some companies simply don’t proactively hunt for violations. The result? Harms slide through the cracks.
Take a recent case in Massachusetts. A man was convicted of cyberstalking because he used an AI chatbot to impersonate his victim. He programmed the bot to invite strangers to her home address when asked. The tool didn’t just sit there; it participated in the harassment.
Blame the User? Try the Architect.
There’s a common defense. AI providers love it.
They say the problem is “user misuse.”
Really?
Our research shows otherwise. The abuse is baked into the structure. Systems are optimized for engagement. They are designed to be “sycophantic” — eager to please, even if what the user wants is harmful. Instead of refusing violent prompts, they often affirm them.
Training data reflects human bias. Design reflects profit motives. When you mix the two, you get tools that reinforce misogynistic content.
Look at the “nudify” apps of the past few years. They generated deepfake nude images of real women without consent. It took too long to ban them. By the time governments moved, the practice was widespread and the damage was done. Victims had been harmed, repeatedly, because we assumed technology was inherently benign.
We cannot afford the same lag with chatbots.
What Regulation Needs to Look Like
Two changes are necessary.
Make it criminal.
Creating an AI tool designed to harass or abuse should be a crime. If you release a tool with insufficient protections, you are negligent. It is like owning a dangerously aggressive dog. The law should punish the owner for creating that public risk. Fines. Prison time. Anything to force companies to think before they launch.
Specific AI Safety Laws.
We need mandatory risk assessments. We need transparency. We need a legal duty to act when harm occurs. States like Utah, Colorado, and California are moving in this direction, allowing citizens to sue providers who fail in their obligations. Washington is fighting back, claiming these rules kill innovation.
Is safety a barrier to innovation?
The “Kids” Red Herring
Opponents argue chatbots mostly endanger children.
The UK recently announced it was exploring a ban on chatbots for those under 16. That feels reactive. Narrow.
Our research proves the danger isn’t confined to minors. Adult women are being stalked. They are receiving personalized guidance on how to be harassed.
Remember the Massachusetts case? The attacker didn’t use a child-friendly toy. He fed the bot the victim’s employment history. Her hobbies. Her husband’s workplace. He built a digital shadow to harass a grown woman.
A ban on users under 18 wouldn’t have stopped him.
The harm here is societal. It doesn’t expire when you turn twenty-one. If we wait for everyone to become adults, the damage is already cemented. We need rigorous testing before products hit the public sphere. Continually after that.
An Open Question
Changing the law won’t just protect the young. It protects the future of the adult internet.
It ensures that when those children grow up, the AI environment isn’t steeped in bias, misogyny, and violence. It creates a standard of safety.
Or it doesn’t.
That depends on whether we treat these tools as inevitable forces of nature or as products we are responsible for regulating. The violence is happening now. The question is whether the law will catch up before more lives are ruined.
