It isn't just San Francisco with misgivings about facial recognition technology.

This week, the city's board of supervisors approved an ordinance restricting the use of facial recognition and other surveillance tech, a category that includes products such as Amazon's (AMZN - Get Report) Rekognition software and Microsoft's (MSFT - Get Report) Face API, among others. Under the new ordinance, any municipal agency using surveillance tech must describe exactly how it will be used and seek approval from the board, thereby creating a record of how the surveillance tech is deployed. 

Amazon shares were up 1.69% to $1,871.15 on Wednesday and are up about 25% year to date.

It's not known how much Amazon makes from selling Rekognition, but it's one instance of the company's embrace of often lucrative deals in the law enforcement, military and government sectors -- making it an outlier among other big tech firms, such as Alphabet (GOOGL - Get Report) and Microsoft, that have disavowed certain uses of their technologies. Amazon has a $600 million cloud computing deal with the CIA, for instance, and is competing with Microsoft on a $10 billion cloud contract with the Pentagon.  Reached for comment, an Amazon spokesperson declined to specify how many agencies use Rekognition, saying that it can't divulge customer information without permission. 

The use of facial recognition technology, particularly in law enforcement, has drawn condemnation by civil liberties groups for its potential to intensify unequal policing and invade privacy more generally. A January MIT study, for example, found widespread errors in Rekognition's classification system by race; an earlier MIT study found similar discrepancies with facial recognition software by Microsoft, IBM (IBM - Get Report) and others.

It turns out that some of Amazon's own shareholders share many of these concerns.

A proposal to impose a moratorium on Amazon's sales of Rekognition is up for a vote at its annual investor meeting on May 22, and a group of investors advocating for the proposal suggest that the San Francisco ban is only the tip of the iceberg.

"The business case for this is going to get harder to make as municipalities become more concerned about the potential risks," said Mary Beth Gallagher of the Tri-State Coalition for Responsible Investment, one of the proponents of the shareholder proposal. "There aren't really clear parameters on how it should and shouldn't be used, and under what conditions it's appropriate. I think this is a very serious potential risk, and something that we don't think should move forward without more foresight and assessment of that risk."

According to Gallagher, a lack of clear guidelines around facial recognition in policing and elsewhere will make Amazon vulnerable to lawsuits and costly litigation, causing financial harm to the company and its shareholders.

But there's another potential, perhaps even greater risk: Eroding consumer trust.

Especially relative to other tech companies, Amazon still enjoys a strong reputation right now: A survey by Georgetown University and NYU last year found that Amazon was the most-trusted institution in the U.S., second only to the military. And while some of that goodwill may have been eroded by concerns about various business practices and the way Amazon handled its HQ2 search and decision-making, controversies over Rekognition could imperil that reputation further, as well as cause collateral damage to other crucial products Amazon sells, according to the shareholders backing the ban.

"The increasing controversy surrounding Rekognition highlights the fragile relationship of trust between the Company and its consumers, employees, and the public at large. A number of the Company's products -- Alexa, Ring, and Eero -- could face a spillover effect if Amazon's ability to maintain the trust of customers is breached by concerns about privacy and surveillance," the proposal said.

While the proposal may not pass, forcing the board to engage with risks involving facial recognition, and sending a signal to the company that more diligence is needed, will be a step in the right direction, Gallagher added.

"We're doing our best to educate investors that this is about long-term shareholder value," Gallagher said.

Amazon, Alphabet and Microsoft are holdings in Jim Cramer's Action Alerts PLUS Charitable Trust Portfolio. Want to be alerted before Cramer buys or sells these stocks? Learn more now.