Apple has packed an interesting new accessibility feature into the latest beta of iOS: a system that detects the presence of and distance to people in the view of the iPhone’s camera, so blind users can social distance effectively, among many other things.
The feature emerged from Apple’s ARKit, for which the company developed “people occlusion,” which detects people’s shapes and lets virtual items pass in front of and behind them. The accessibility team realized that this, combined with the accurate distance measurements provided by the lidar units on the iPhone 12 Pro and Pro Max, could be an extremely useful tool for anyone with a visual impairment.
Of course during the pandemic one immediately thinks of the idea of keeping six feet away from other people. But knowing where others are and how far away is a basic visual task that we use all the time to plan where we walk, which line we get in at the store, whether to cross the street and so on.
The new feature, which will be part of the Magnifier app, uses the lidar and wide-angle camera of the Pro and Pro Max, giving feedback to the user in a variety of ways.
The lidar in the iPhone 12 Pro shows up in this infrared video. Each dot reports back the precise distance of what it reflects off of.
First, it tells the user whether there are people in view at all. If someone is there, it will then say how far away the closest person is in feet or meters, updating regularly as they approach or move further away. The sound corresponds in stereo to the direction the person is in the camera’s view.
Second, it allows the user to set tones corresponding to certain distances. For example, if they set the distance at six feet, they’ll hear one tone if a person is more than six feet away, another if they’re inside that range. After all, not everyone wants a constant feed of exact distances if all they care about is staying two paces away.
The third feature, perhaps extra useful for folks who have both visual and hearing impairments, is a haptic pulse that goes faster as a person gets closer.
Last is a visual feature for people who need a little help discerning the world around them, an arrow that points to the detected person on the screen. Blindness is a spectrum, after all, and any number of vision problems could make a person want a bit of help in that regard.
The system requires a decent image on the wide-angle camera, so it won’t work in pitch darkness. And while the restriction of the feature to the high end of the iPhone line reduces the reach somewhat, the constantly increasing utility of such a device as a sort of vision prosthetic likely makes the investment in the hardware more palatable to people who need it.
This is far from the first tool like this — many phones and dedicated devices have features for finding objects and people, but it’s not often that it comes baked in as a standard feature.
People detection should be available to iPhone 12 Pro and Pro Max running the iOS 14.2 release candidate that was just made available today. Details will presumably appear soon on Apple’s dedicated iPhone accessibility site.
Things usually move pretty slowly for the US Supreme Court, with cases sometimes taking years to make their way through to a ruling. But these days it’s moving so quickly that the newest justice didn’t even have time to participate in the first two crucial voting-related rulings after her confirmation. The breakneck pace reveals that the nation’s highest court is already shaping the 2020 election—and shows how it might do even more after November 3.
In just two weeks, the court has issued five orders on voting cases, all focusing on one central question: Who decides exactly how we count votes?
On Friday afternoon, President Donald Trump tweeted about the latest Supreme Court decision, on North Carolina’s extension for counting mail-in ballots to account for delays in the postal service. The justices rejected a Republican attempt to block the extension, and the state can now count mail ballots until November 12 as long as they were sent by Election Day. The extension is designed as a fix to unprecedented mail delivery delays.
“This decision is CRAZY and so bad for our Country,” Trump tweeted. “Can you imagine what will happen during that nine day period. The Election should END on November 3rd.”
Trump’s tweet goes to the heart of the political and legal case he’s been making all year. But there are problems with his argument: it goes against every election in American history, it has no legal basis, and it’s a part of his politicized disinformation campaign about election security, which is regularly contradicted by the federal government’s own top election security officials.
There is nothing nefarious to “imagine.” Counting ballots after Election Day happens in literally every national election. It has never been the case that every vote is counted on election night. The “results” you typically hear on that night are actually news media predictions with zero legal weight. Officially certified results come in days or weeks later as all ballots are counted.
It’s the counting of ballots that’s at issue before the Supreme Court. Dealing with a worsening pandemic, record-breaking mail-in voting, and a US Postal Service faltering under a recent Trump donor installed as its leader, some states have attempted to deal with the problems by extending the time for valid ballots to be counted.
Democrats and court liberals have supported expanded voting rules in hopes of adapting to this unprecedented election challenge. Republicans and court conservatives, meanwhile, have generally opposed rule changes enacted by state legislatures even when they’re pushed by state election officials.
The rulings so far
Here’s how the Supreme Court has ruled in recent cases that will define the 2020 election.
Pennsylvania, October 20: The Republicans lost a swing-state battle last week. The state supreme court had ruled that mail ballots could be received three days past Election Day, after the Postal Service said that delivery delays risked disenfranchisement around the state. Who made the ruling is a key theme here: If extensions come from the states, they tend to succeed before the Supreme Court; if they come from the federal government, they fail.
Alabama, October 22: A statewide ban on curbside voting—in which disabled folks drive up to a polling place and drop off their ballots—was allowed to stand. The ban originally came from the Alabama secretary of state, who was in dispute with a federal court over whether the ban violated the Americans with Disabilities Act. The state prevailed here over dissent from the Supreme Court’s liberals.
Wisconsin, October 26: As we recently reported, the Supreme Court declined to extend the deadline for counting of mail-in votes in Wisconsin, a victory for Republicans who brought the legal challenge. This particular extension order originally came from a federal judge in September, a crucial point that the conservatives on the court all agreed on: federal courts shouldn’t micromanage state-run elections.
Pennsylvania, October 28: The US Supreme Court declined a Republican request to expedite a review of Pennsylvania’s mail-in ballot deadlines—the case it had ruled on the previous week. But the issue is not gone for good: conservative justices left the door open to the possibility of revisiting the case after the election, and Pennsylvania officials are segregating ballots received after Election Day in case of just such a legal battle. If the vote is close in Pennsylvania, you can bet this will rear its head once again.
North Carolina, October 29: Democrats won a similar case a few days later in a 5-3 decision, with Chief Justice Roberts joining the more liberal justices in allowing North Carolina to receive and count votes up to nine days after Election Day. This extension, from three days to nine days, came from the state’s board of elections. That, to Roberts, ultimately made the difference.
The near future
“I think this will end up in the Supreme Court,” Trump predicted last month.
Amy Coney Barrett hasn’t participated in any of the five major voting rulings, but she’s sure to be involved in the future, and will certainly play a role in any legal dispute after Election Day. President Trump has made it clear that’s where sees the fight going after the polls are closed.
If results are close enough for ballots counted after polls close to make a difference to the outcome, it’s a virtual guarantee that lawsuits will be filed in short order—as long as one side sees an advantage in doing so. However, there is increasing evidence that Democrats have sent their mail-in ballots in earlier than Republicans. If that’s the case in some key swing states, the entire premise could get flipped.
If it feels as if the US Supreme Court can play an enormous role in the outcome of the upcoming election, that’s true. But everything depends on what happens at the ballot box. A landslide one way or the other could make most legal challenges inconsequential: there’s no need to fight over 10,000 votes if the gap is 500,000.
But if the margin around the country and in key swing states gets close enough, the next move is clear.
Four years ago, Bin Xie was happy to sing the praises of WeChat. The IT manager from Houston had seen his pro-Trump blog, Chinese Voice of America, go viral on the app.
Today, Xie stands firmly behind the president, but his relationship with the platform that fueled his rise has soured. The shift didn’t happen when Trump announced that he would ban the app, though: it came in 2019, when Xie’s account was temporarily suspended after he shared the results of Hong Kong’s district elections in a WeChat group with the note, “The pro-China candidates totally lost.”
For Xie, who had long been tired of writing in purposefully bungled Chinese to confuse the platform’s censors (“like a kindergartener,” he says), this was the final straw. He started encouraging his followers to leave for alternative apps.
And he was far from alone. For years, many Chinese-American WeChat users have become increasingly disillusioned with the platform’s opaque censorship and surveillance practices. While some have turned to alternatives, like Telegram, WhatsApp, and Line, most found that WeChat’s popularity meant it was impossible to leave.
WeChat “is so important to the Chinese-American community,” says Steven Chen, who writes a popular liberal-leaning WeChat blog and helps nonprofit organizations use the platform. “But more importantly,” he adds, “we actually have to use it to communicate with our parents … the elder[ly] people in China basically only have WeChat.”
This level of nuance was lost when President Trump issued an executive order in early August that would have banned WeChat (as well as the Chinese-owned video-sharing platform TikTok) within 45 days on grounds of national security. While many Chinese-Americans actually agreed that WeChat deserved more scrutiny, few believed that Trump’s ban—seen as both another attack on Chinese-Americans and an example of the administration’s blunt force approach to US-China relations—was the right way to go about it.
‘A virtual Chinatown’
Since its creation in 2011, WeChathas become the undisputed messaging app of choice in China. With its 1.2 billion monthly active users, it is the world’s fifth-largest social network.
For the service’s owner, Tencent, it has been a huge success, essentially acting as its own mobile operating system. It has an app store that caters to all of its users’ digital needs, combining the social features of Facebook profiles, time lines, and groups; the payment/shopping features of Venmo, Paypal, and Amazon; the geolocation and mapping functions of Google Maps; and, in the age of covid-19, even a health code program that predicts your likelihood of infection, which then determines your ability to leave your home, visit stores and restaurants, or travel.
In the US, WeChat’s user base is much smaller, numbering in the “single-digit millions,” according to Tencent America. They are mostly first-generation Chinese immigrants or others with strong ties to China, who mainly use the app for social activity and information sharing.
Many of these immigrants are more comfortable conversing in Chinese than English, and Chinese is the main language in use on the app. Steven Chen is concerned that this has made WeChat into a “virtual Chinatown,” keeping “isolated first-generation immigrants from mainland China from the rest of the country and the broader range of political views,” as he wrote in a Medium post in 2018.
The limits are exacerbated by the censorship that, Chen says, everyone knows to occur on the platform. The issue is one that WeChat users—like all Chinese internet users—regularly navigate. (While American WeChat users aren’t necessarily subject to the same levels of Chinese internet policing, it is dramatically easier to create a blog through the Chinese arm of the app, which means that most content is still subject to the Chinese Communist Party’s rules.) Most people don’t have that much to worry about, says Chen, because “they’re not trying to overthrow the government.” But he acknowledges that he is “really careful” when publishing articles, and that he has had them removed in the past. So have Xie and three other blog owners I interviewed.
Online mobilization
At the center of these first-generation immigrants’ experiences on WeChat are its groups. They can be created by anyone but are limited to 500 members. Users can join an unlimited number of them and choose how their name is displayed in each one.
In the beginning, groups were mostly nonpolitical, reflecting the fact that Chinese-Americans have historically been one of the least politically active demographicsin the United States. Butthis began to change in 2014, driven by two specific events.
The first was a proposition in California called SCA-5 that planned to restore affirmative action in university admissions. The move to allow race, gender, and ethnicity to be considered in these decisions was intended to ensure that more nonwhite students entered the University of California system, and a field poll conducted that year showed that Asian-Americans actually supported affirmative action at a rate of 69%.
But first-generation Chinese-American parents—who were less supportive of affirmative action—panicked as rumors on WeChat and ethnic media suggested that the bill would result in racial quotas damaging the educational prospects of their children. They used WeChat to mobilize demonstrations and protests, many for the first time, and the bill was withdrawn under pressure, which the new activists considered a victory.
In November of the same year, Peter Liang, a Chinese-American police officer in New York City, shot and killed a 28-year-old Black man, Akai Gurley. While white officers in controversial shootings had not been indicted—including Darren Wilson for the death of Michael Brown in Ferguson, Missouri, and Daniel Pantaleo for the death of Eric Garner in Staten Island, New York—Liang became the first NYPD officer charged for a shooting in over 10 years. He was indicted and later convicted.
First-generation Chinese-Americans organized en masse via WeChat, believing that Liang had been unfairly scapegoated for the more frequent crimes of white officers. In the end, Liang was sentenced to five years of probation and 800 hours of community service.
In the beginning, WeChat groups were mostly nonpolitical, reflecting the fact that Chinese-Americans have historically been one of the least politically active demographicsin the United States. But by the time the 2016 US presidential election took place, it captivated audiences on WeChat just as it did the English-language media.
The community’s interest in political participation grew. By the time the 2016 US presidential election took place, it captivated audiences on WeChat just as it did in the English-language media.
And among those who benefited from the political activity was Xie. Chinese Voice of America was proudly pro-Trump, repeating right-wing talking points that, often, had already been debunked on English-language fact-checking sites. One article, titled “Banning pork has quietly begun across the United States,” typified how CVA tailored the messaging from right-wing publications to cater to the specific concerns of his audience. (Pork is an important part of the middle-class Chinese diet.)
In an interview I conducted with Xie in 2017, a few months after Trump had taken office, he described how WeChat helped his messages go viral. “If I publish it on WeChat, I’ll get thousands of hits,” he said. “If readers see something on their topic [of interest], they are going to spread it quickly to all their groups”—a much easier process than if he published on a website.
But Xie and his friends didn’t just publish articles and then sit back; they also actively engaged their readers, and their opponents, in vicious partisan debates that often dominated even the most nonpolitical groups. Their coordination made it seem as if most Chinese-Americans supported Trump. “The pro-Trump side was definitely louder,” recalls Ling Luo, a prominent Democratic activist who now leads a pro-Biden affinity group for Chinese-Americans; she ran her own WeChat blog, but she admits that in 2016, the Democratic side was not as prepared for the partisan fights that would take place in WeChat groups.
Chen says he had never seen politics become as divisive for the community as they did during the 2016 campaign. “In previous years,” he says, “of course people supported different presidents,” but that did not mean that “people stop talking to each other,” or that they gave up friendships that had spanned continents, as they did now.
At first, he attributed this to Trump himself, but when I pressed him further, he recognized that the app itself was a factor. “WeChat probably played a bigger role … and intensified the difference between the people,” he says. “It’s not as easy to use email or phone to fight.”
Two sides
If 2016 revealed strong divisions in the Chinese-American community, at least the most ferocious political debates still focused on supporting or opposing the candidates. But this year, some users say the arguments hinge on something more existential: whether one is pro-China or pro-America.
Both sides accuse each other of being “red guards,” referring to the youth militia groups weaponized during the Cultural Revolution to attack intellectuals and other “class enemies.” The insult implies that someone is a brainwashed ideologue doing another’s bidding.
The Pro-China side might also use the more serious label “traitors to the Chinese race” (反华分子), while the pro-America side calls its opponents “CCP spies.” Both of these accusations carry serious weight, given China’s increased demand for loyalty from Chinese abroad, on the one hand, and the US government’s increased concern about Chinese espionage.
One woman, who I’ll call Jan to protect her from potential retaliation, recalls an incident that provoked accusations of being anti-Chinese.
Some time after Trump announced his ban, a member in one of her groups remarked, “WeChat is not innocent,” and suggested that people move to a more secure app, like Telegram. Another group member immediately jumped in, labeling him a traitor and accusing him of “moving people from a popular app to an app that nobody uses … destroying the grassroots movement.”
The escalation was immediate and dizzying. Pro-CCP users “always have the moral high ground,” she said, “sowing doubts” about the motives of others.
She kicked the second member out of her group, but still, Jan has been haunted by a lingering question: Are these just typical internet trolls who happen to be pro-China, or are they part of something more sinister—a targeted attack aimed at dividing the Chinese diaspora?
Over the past few months, she’s been comparing notes with friends across the country who have had similar experiences. “We spent a lot of time cross-referencing,” she said. Many shared her experiences, with accounts posting the same kinds of divisive messages and using the same language across multiple groups. They also used the same avatars with the same pseudonyms, which they had not bothered to change between groups.
Jan has become paranoid about CCP internet operatives, who are already notorious within China’s firewalled internet. There, they are known as the “50 cent army,” because of the apocryphal 50 cents that they make for every pro-China post. Besides, the CCP is known for its long-standing strategy of using its diaspora communities to help the motherland.
So, Jan wondered, was it really so strange to think that the CCP was targeting people of Chinese descent in the United States?
“In recent years, the Chinese government has stepped up moves to influence the diaspora communities around the world to advance Beijing’s interests, and the use of Chinese tech is a key component of this influence operation,” says Yaqiu Wang, a China analyst with Human Rights Watch. “One of the biggest victims of China’s authoritarian tech expanding abroad has been the Chinese diaspora.”
Jan has been thinking about leaving WeChat, or at least ceasing to express even the faintest of political opinions (including, ironically, suggestions to leave WeChat).
But regardless of whether she leaves, she is afraid that the damage has already been done. She’s aware of the US government’s increased scrutiny of Chinese-Americans, which is not limited to the FBI but also includes the Department of Justice’s China Initiative. She is also afraid that she has been connected to potential CCP operatives just by virtue of being in the same WeChat groups. When it comes to Chinese-Americans, she says, the FBI “cannot distinguish between victims, collaborators, and masterminds.”
Indeed, even before the latest wave of discrimination and hate crimes against Chinese-Americans during the coronavirus pandemic, and before Trump’s stubborn characterization of the disease as the “China virus” or “Kung flu,” anti-China sentiment in the United States had been growing. Christopher Wray, the director of the FBI, has called China “the greatest long-term threat to our nation’s information and intellectual property,” saying that a “whole-of-society” response from the United States is required to fight it.
These kinds of remarks, civil rights advocates say, are already resulting in racial profiling, especially of scientists of Chinese descent.
Backfiring ban
In late August, a group of WeChat users sued the Trump administration on First Amendment grounds. On September 20, the day the ban would have gone into effect, a judge in California’s Northern District Court granted the apps a preliminary reprieve. Since then, the ban has been making its way through the courts. The next decision is not expected until after the election, which might change everything anyway.
Instead of pushing users away from WeChat, the threatened ban did the opposite. On August 6, when Trump issued his executive order, there was a spike in downloads of alternative apps such as Line, Telegram, and WhatsApp, according to data provided by the mobile apps insight company Apptopia.
But it also led to a rush of downloads of WeChat itself. This bump was even more pronounced and prolonged around September 20, when the ban was scheduled to go into effect.
It’s unclear from the data, though, whether or not anyone has deleted WeChat.
For his part, Xie now splits his time between apps. “Everybody’s just like me,” he says with a chuckle. “Spend some time in WeChat, some time in Telegram, some time in Line … And, in fact, we enjoy better [the] replacements,” he adds, finding it freeing not to worry about group size limits or euphemisms and other creative ways to avoid censorship.
But if WeChat was a “virtual Chinatown” before, it’s possible these shifts might end up exacerbating political divides. Before, at least, WeChat users could easily come across other Chinese-Americans with different opinions in the same groups. Now Xie, for example, runs a WhatsApp group for people censored by WeChat, while another woman invited me to a Telegram group that was decidedly pro-Trump.
For Chen, the increased potential for unity is a reason for him to stay on WeChat. He could choose “to get out of the virtual Chinatown,” he says, but then he’d be leaving WeChat to other people. So even though he doesn’t think WeChat is a good long-term solution, he hasn’t abandoned it, because he wants “to fight to make [WeChat] a better place.”
Unless you’re a physicist or an engineer, there really isn’t much reason for you to know about partial differential equations. I know. After years of poring over them in undergrad while studying mechanical engineering, I’ve never used them since in the real world.
But partial differential equations, or PDEs, are also kind of magical. They’re a category of math equations that are really good at describing change over space and time, and thus very handy for describing the physical phenomena in our universe. They can be used to model everything from planetary orbits to plate tectonics to the air turbulence that disturbs a flight, which in turn allows us to do practical things like predict seismic activity and design safe planes.
The catch is PDEs are notoriously hard to solve. And here, the meaning of “solve” is perhaps best illustrated by an example. Say you are trying to simulate air turbulence to test a new plane design. There is a known PDE called Navier-Stokes that is used to describe the motion of any fluid. “Solving” Navier-Stokes allows you to take a snapshot of the air’s motion (a.k.a. wind conditions) at any point in time and model how it will continue to move, or how it was moving before.
These calculations are highly complex and computationally intensive, which is why disciplines that use a lot of PDEs often rely on supercomputers to do the math. It’s also why the AI field has taken a special interest in these equations. If we could use deep learning to speed up the process of solving them, it could do a whole lot of good for scientific inquiry and engineering.
Now researchers at Caltech have introduced a new deep-learning technique for solving PDEs that is dramatically more accurate than deep-learning methods developed previously. It’s also much more generalizable, capable of solving entire families of PDEs—such as the Navier-Stokes equation for any type of fluid—without needing retraining. Finally, it is 1,000 times faster than traditional mathematical formulas, which would ease our reliance on supercomputers and increase our computational capacity to model even bigger problems. That’s right. Bring it on.
Hammer time
Before we dive into how the researchers did this, let’s first appreciate the results. In the gif below, you can see an impressive demonstration. The first column shows two snapshots of a fluid’s motion; the second shows how the fluid continued to move in real life; and the third shows how the neural network predicted the fluid would move. It basically looks identical to the second.
The first thing to understand here is that neural networks are fundamentally function approximators. (Say what?) When they’re training on a data set of paired inputs and outputs, they’re actually calculating the function, or series of math operations, that will transpose one into the other. Think about building a cat detector. You’re training the neural network by feeding it lots of images of cats and things that are not cats (the inputs) and labeling each group with a 1 or 0, respectively (the outputs). The neural network then looks for the best function that can convert each image of a cat into a 1 and each image of everything else into a 0. That’s how it can look at a new image and tell you whether or not it’s a cat. It’s using the function it found to calculate its answer—and if its training was good, it’ll get it right most of the time.
Conveniently, this function approximation process is what we need to solve a PDE. We’re ultimately trying to find a function that best describes, say, the motion of air particles over physical space and time.
Now here’s the crux of the paper. Neural networks are usually trained to approximate functions between inputs and outputs defined in Euclidean space, your classic graph with x, y, and z axes. But this time, the researchers decided to define the inputs and outputs in Fourier space, which is a special type of graph for plotting wave frequencies. The intuition that they drew upon from work in other fields is that something like the motion of air can actually be described as a combination of wave frequencies, says Anima Anandkumar, a Caltech professor who oversaw the research alongside her colleagues, professors Andrew Stuart and Kaushik Bhattacharya. The general direction of the wind at a macro level is like a low frequency with very long, lethargic waves, while the little eddies that form at the micro level are like high frequencies with very short and rapid ones.
Why does this matter? Because it’s far easier to approximate a Fourier function in Fourier space than to wrangle with PDEs in Euclidean space, which greatly simplifies the neural network’s job. Cue major accuracy and efficiency gains: in addition to its huge speed advantage over traditional methods, their technique achieves a 30% lower error rate when solving Navier-Stokes than previous deep-learning methods.
The whole thing is extremely clever, and also makes the method more generalizable. Previous deep-learning methods had to be trained separately for every type of fluid, whereas this one only needs to be trained once to handle all of them, as confirmed by the researchers’ experiments. Though they haven’t yet tried extending this to other examples, it should also be able to handle every earth composition when solving PDEs related to seismic activity, or every material type when solving PDEs related to thermal conductivity.
Super-simulation
The professors and their PhD students didn’t do this research just for the theoretical fun of it. They want to bring AI to more scientific disciplines. It was through talking to various collaborators in climate science, seismology, and materials science that Anandkumar first decided to tackle the PDE challenge with her colleagues and students. They’re now working to put their method into practice with other researchers at Caltech and the Lawrence Berkeley National Laboratory.
One research topic Anandkumar is particularly excited about: climate change. Navier-Stokes isn’t just good at modeling air turbulence; it’s also used to model weather patterns. “Having good, fine-grained weather predictions on a global scale is such a challenging problem,” she says, “and even on the biggest supercomputers, we can’t do it at a global scale today. So if we can use these methods to speed up the entire pipeline, that would be tremendously impactful.”
There are also many, many more applications, she adds. “In that sense, the sky’s the limit, since we have a general way to speed up all these applications.”
Want more leads and sales from your YouTube channel? Wondering how to use different types of videos for different objectives? To explore YouTube content strategy, I interview Sunny Lenarduzzi on the Social Media Marketing Podcast. Sunny is a YouTube marketing expert who helps entrepreneurs next-level their awareness online. Her YouTube channel has 500,000 subscribers. Her […]
The Uniswap community is voting on its second-ever governance proposal to distribute 5 million governance tokens to users who interacted with the DEX via a third-party platform.
