New York City lawmakers want to give AI whistleblowers a financial reason to come forward – and give New Yorkers a new way to sue artificial intelligence companies over certain harms.
The proposals are part of a sweeping package announced Friday by City Council Speaker Julie Menin ahead of an Oct. 5 hearing that could also require third-party validation and human “kill switches” for AI systems marketed, offered for sale or deployed in the city, along with 24-hour reporting of certain AI safety incidents involving city contracts.
“New York City is fast becoming the technology and AI capital of the world, which we want to encourage,” Menin said in announcing the package. “But that also means we now have an even greater responsibility to ensure that we have the appropriate safeguards in place to protect New Yorkers from unintended consequences.”
One proposal sponsored by Menin would allow individual whistleblowers to receive a portion of fines or penalties recovered from artificial intelligence companies that violate applicable laws.
The Council is billing it as a first-in-the-nation approach aimed at encouraging insiders to report major violations of applicable laws. The Council’s announcement does not say how large a whistleblower’s share could be or spell out how the incentive program would work.
A separate bill sponsored by Council Member Virginia Maloney would create a private right of action allowing people harmed by certain malicious uses of AI to sue the companies behind the technology.
According to the announcement, a company could potentially be held liable if the harm was foreseeable, it failed to put reasonable safeguards in place and a third party exploited that failure — including by circumventing an AI system’s safety controls, commonly known as “jailbreaking.”
Speaking Thursday, Menin argued that the city already regulates a wide range of industries and should take a leading role on AI.
“We have to remember that the City of New York regulates all sorts of industries,” Menin said, pointing to her time as consumer affairs commissioner, when she said roughly 80,000 businesses were subject to city regulation.
Menin said cities should take a leading role when the federal government is “deregulating or not acting.”
A broader push to regulate AI
A second Menin-sponsored bill would prohibit businesses from marketing, offering for sale or deploying an AI system in NYC unless it first receives validation from an outside reviewer.
Those reviews would cover areas including data quality, bias, privacy, security and the system’s outputs, along with any additional requirements set by the city’s Cyber Command.
The proposal would also require covered AI systems to have a human override, or “kill switch,” capable of shutting them down.
Businesses and validators could each face a $25,000 penalty for every instance in which an AI system is marketed, offered for sale or deployed without the required validation, or when a validation has been falsified.
The city push comes as Albany weighs its own next round of AI safeguards: Gov. Kathy Hochul this week floated state-level “kill switches,” while state Sen. Andrew Gounardes has proposed annual independent safety reviews for large frontier AI developers.
Other bills would require city contractors and agencies to report AI safety incidents to Cyber Command within 24 hours, create an emergency response plan for AI-related disruptions and extend whistleblower protections to city employees and contractors who report AI-related public safety concerns.
Another proposal would require AI companies to make certain disclosures about their products and prohibit false or misleading safety claims, while the Council is also considering privacy, security and transparency requirements for chatbot providers.
The package will get its first major airing Oct. 5, when all 51 Council members are expected to convene as a Committee of the Whole, a rarely used format that Menin said reflects the gravity of the issue.
She has requested participation from OpenAI CEO Sam Altman, Anthropic CEO Dario Amodei, Google CEO Sundar Pichai, Elon Musk and Meta CEO Mark Zuckerberg.
A Council source told amNewYork that officials are in active talks with several of the companies that received letters, while stressing that the Council retains the ability to use its subpoena power if necessary.
The upcoming hearing follows a series of warnings and testing incidents that have drawn scrutiny from lawmakers.
The Council pointed to reports that AI agents being tested by OpenAI in July circumvented containment controls, communicated through unauthorized channels, gained internet access and compromised systems belonging to AI company Hugging Face during a cybersecurity evaluation.
Menin emphasized Thursday that the episode occurred during a safety test as she argued for a more active city role in regulating the industry.
The concerns also formed the backdrop of Menin’s initial outreach to AI companies last week, when she first announced that the Council would examine what additional safeguards city government could impose beyond existing state and local rules.
But Menin said the goal is not to push the industry out of New York.
“We want New York City to remain the AI capital of the world, but we must have responsible safety regulations, and those two concepts can exist hand in hand,” she said.
