Flock Uses Overseas Gig Workers to Build its Surveillance AI
This article was produced with support from WIRED.
Flock, the automatic license plate reader (ALPR) and AI-powered camera company, uses overseas workers from Upwork to train its machine learning algorithms, with training material telling workers how to review and categorize footage including images people and vehicles in the U.S., according to material reviewed by 404 Media that was accidentally exposed by the company.
The findings bring up questions about who exactly has access to footage collected by Flock surveillance cameras and where people reviewing the footage may be based. Flock has become a pervasive technology in the U.S., with its cameras present in thousands of communities that cops use everyday to investigate things like car jackings. Local police have also performed numerous lookups for ICE in the system.
Companies that use AI or machine learning regularly turn to overseas workers to train their algorithms, often because the labor is cheaper than hiring domestically. But the nature of Flock’s business—creating a surveillance system that constantly monitors U.S. residents’ movements—means that footage might be more sensitive than other AI training jobs.
💡
Do you work at Flock or know more about the company? I would love to hear from you. Using a non-work device, you can message me securely on Signal at joseph.404 or send me an email at joseph@404media.co.
Flock’s cameras continuously scan the license plate, color, brand, and model of all vehicles that drive by. Law enforcement are then able to search cameras nationwide to see where else a vehicle has driven. Authorities typically dig through this data without a warrant, leading the American Civil Liberties Union (ACLU) and Electronic Frontier Foundation (EFF) to recently sue a city blanketed in nearly 500 Flock cameras.
Broadly, Flock uses AI or machine learning to automatically detect license plates, vehicles, and people, including what clothes they are wearing, from camera footage. A Flock patent also mentions cameras detecting “race.”
Screenshots from the exposed material. Redactions by 404 Media.
Multiple tipsters pointed 404 Media to an exposed online panel which showed various metrics associated with Flock’s AI training.
This post is for subscribers only
Become a member to get access to all content
Subscribe now
Flock Uses Overseas Gig Workers to Build its Surveillance AI
Flock accidentally exposed training materials and a panel which tracked what its AI annotators were working on. It showed that Flock, which has cameras in thousands of U.S.Joseph Cox (404 Media)
ACLU and EFF Sue a City Blanketed With Flock Surveillance Cameras
Lawyers from the American Civil Liberties Union (ACLU) and Electronic Frontier Foundation (EFF) sued the city of San Jose, California over its deployment of Flock’s license plate-reading surveillance cameras, claiming that the city’s nearly 500 cameras create a pervasive database of residents movements in a surveillance network that is essentially impossible to avoid.The lawsuit was filed on behalf of the Services, Immigrant Rights & Education Network and Council on American-Islamic Relations, California, and claims that the surveillance is a violation of California’s constitution and its privacy laws. The lawsuit seeks to require police to get a warrant in order to search Flock’s license plate system. The lawsuit is one of the highest profile cases challenging Flock; a similar lawsuit in Norfolk, Virginia seeks to get Flock’s network shut down in that city altogether.
“San Jose’s ALPR [automatic license plate reader] program stands apart in its invasiveness,” ACLU of Northern California and EFF lawyers wrote in the lawsuit. “While many California agencies run ALPR systems, few retain the locations of drivers for an entire year like San Jose. Further, it is difficult for most residents of San Jose to get to work, pick up their kids, or obtain medical care without driving, and the City has blanketed its roads with nearly 500 ALPRs.”
The lawsuit argues that San Jose’s Flock cameras “are an invasive mass surveillance technology” that “collect[s] driver locations en masse.”
“Most drivers are unaware that San Jose’s Police Department is tracking their locations and do not know all that their saved location data can reveal about their private lives and activities,” it adds. The city of San Jose currently has at least 474 ALPR cameras, up from 149 at the end of 2023; according to data from the city, more than 2.6 million vehicles were tracked using Flock in the month of October alone. The lawsuit states that Flock ALPRs are stationed all over the city, including “around highly sensitive locations including clinics, immigration centers, and places of worship. For example, three ALPR cameras are positioned on the roads directly outside an immigration law firm.”
Andrew Crocker, surveillance litigation director for the EFF, told 404 Media in a phone call that “it’s fair to say that anyone driving in San Jose is likely to have their license plates captured many times a day. That pervasiveness is important.”
DeFlock's map of San Jose's ALPRs
A zoomed in look at San Jose
A search of DeFlock, a crowdsourced map of ALPR deployments around the country, shows hundreds of cameras in San Jose spaced essentially every few blocks around the city. The map is not exhaustive.The lawsuit argues that warrantless searches of these cameras are illegal under the California constitution’s search and seizure clause, which Crocker said “has been interpreted to be even stronger than the Fourth Amendment,” as well as other California privacy laws. The case is part of a broader backlash against Flock as it expands around the United States. 404 Media’s reporting has shown that the company collects millions of records from around the country, and that it has made its national database of car locations available to local cops who have in turn worked with ICE. Some of those searches have violated California and Illinois law, and have led to reforms from the company. Crocker said that many of these problems will be solved if police simply need to get a warrant to search the system.
“Our legal theory and the remedy we’re seeking is quite simple. We think they need a warrant to search these databases,” he said. “The warrant requirement is massive and should help in terms of preventing these searches because they will have to be approved by a judge.” The case in Norfolk is ongoing. San Jose Police Department and Flock did not immediately respond to a request for comment.
Flock Removes States From National Lookup Tool After ICE and Abortion Searches Revealed
Following 404 Media’s reporting and in light of new legislation, automatic license plate reader (ALPR) company Flock has stopped agencies reaching into cameras in California, Illinois, and Virginia.Joseph Cox (404 Media)
🔥 Peter 🌍✊anti-fascist reshared this.