The City Council is set to grill officials from some of the biggest tech companies on Monday on the dangers of artificial intelligence, and ways they could rein them in amid increasingly dire warnings from industry leaders and others of rogue robots and malevolent online agents.
One city official warned that New York is already facing such attacks.
“I’m thinking a lot about our financial, accounting, contracting, payment systems — that’s my role as Comptroller — and we’re already seeing an increase in attacks, and these are automated attacks,” city Comptroller Mark Levine told the FAQ NYC podcast this week. “These are not human-initiated attacks or human-led attacks.”
On Thursday, Mayor Zohran Mamdani told reporters his administration wants to embrace technology that can be helpful while being aware of AI threats.
“I think it’s always a balance where we’re looking for any tools that we can use to make a more efficient, more excellent city government,” he said. “And we’re also looking to ensure that this is not a technology that is coming to the expense of a workforce or of the quality of work that we’re putting forward.”
The heads of the city’s technology and worker and consumer protection agencies have agreed to testify at the Monday hearing, according to a source. It was not clear whether officials from the city’s emergency management agency would testify.
Officials from companies including OpenAI and Anthropic said they will testify — but every company excluding Meta agreed to do so only after they were threatened with subpoenas, according to the City Council.
MORE: Why New York City Has Become an AI Mecca
“Given the high stakes, these firms owe it to the public to come before the Council, answer our questions, and provide input on our proposed legislation under oath,” Julie Menin, the Council Speaker, said in a statement.
Menin made the extraordinary move of convening the entire Council’s 51 members, rather than a specific committee, to examine the potential risks posed by the rapid development of artificial intelligence technologies and to question the executives.
Brooklyn Councilman Chi Ossé’s bill to create an emergency plan for potential AI attacks will look at all opportunities to protect government from potential dangers.
“City government should explore all options, including existing laws and new legislation, to protect against AI-related dangers,” he said in a statement.
Other measures under consideration would set guardrails against automation in the civil service sector; create new protocols for contractors to report security breaches; and develop an emergency plan to address potential attacks on the city’s infrastructure, public health and welfare systems.
On Thursday, the Mamdani administration unveiled new AI safeguards including crackdowns on customer service chatbots and dynamic pricing, and launching a study on AI assistants. The new initiative is led by the city’s worker and consumer protection, taxi and limousine, and human rights agencies.
Sam Levine, the head of the city’s Department of Consumer and Worker Protection, said the brief had been in the works for months, and that it serves as a warning shot to many of the business leaders who will be testifying on Monday.
“I just felt it was important to make clear that we are not going to be exempting AI companies from legal requirements in this city,” he said, including the city’s minimum pay laws and rules regarding “deceptive” pricing. He added that he is prepared to testify on the agency’s efforts on those and other subjects.
The Council last year passed a package of bills to develop a framework for oversight and transparency with how the city uses AI, including one that established the Office of Algorithmic Accountability, which was recently set up, according to a spokesperson for the Office of Technology and Innovation.
The most recent budget allocated $1.4 million for the office, and the city is “actively hiring for those roles.”
AI Tracks COVID
The city is also required by a 2021 law to share which algorithmic tools it uses, and by which city agency. Last year, a record number of city agencies reported using AI-powered tools including in categories where such disclosure is required: data analyst, decision-making, or public impact.
In 2025, 19 of 58 reported agencies reported using 86 algorithmic tools, a 56% increase from the previous year. When the first report was released in 2022, 11 city agencies reported using 31 algorithmic tools.
The Department of Health and Mental Hygiene used the most AI tools, at 17, and the mayor’s office (then under Eric Adams) reported using 12.
At the health department, one algorithmic tool is used to analyze COVID-19 variants in wastewater. Another, in use for nearly a decade, monitors social media like Yelp and, until last year, X to find complaints about potential foodborne illness.
The increased regulation and transparency come as concerns about AI systems and potential attacks grow.
‘Internal Controls’
On Tuesday, leaders of top tech firms signed an accord with President Donald Trump vowing to “self police” growth and innovation in their industry, which has been criticized in recent weeks.
They all agreed to install “robust internal controls,” as well as have external auditors look at the effectiveness of the controls.
The officials, including Elon Musk of SpaceX and Mark Zuckerberg of Meta, signed the letters voluntarily, without an enforcement mechanism — although Trump said “it may make sense to codify these steps into law and regulations.”
Levine, the city comptroller, said in the same FAQ NYC interview that the city has benefitted greatly from the success of these companies — but warned of threats to the overall economy — and even to the government.
He said the city needs to ensure its cyber capabilities are up to the task, noting that New York has already seen an increase in “automated attacks.”
“We’re doing okay as of today, but I want to make sure that city government is well defended, and it’s going to take upping our game on cyber defense,” added Levine. “But I just feel like inaction is really not an option.”
Our nonprofit newsroom relies on donations from readers to sustain our local reporting and keep it free for all New Yorkers. Donate to THE CITY today.
The post NYC Already Battling AI Attacks as Council Prepares to Grill Artificial Intelligence Leaders appeared first on The City Reporter.
