While growing up along the Gulf coast in Mississippi, Kenyatta Thomas relied on the internet and other teenagers to learn about sex.
Thomas and their peers watched videos during high school gym class that stressed the importance of abstinence – and the horrors that can come from sex before marriage. But for Thomas, who is bisexual and non-binary, the lessons didn’t explain who they were as a person.
“It was very confusing trying to navigate understanding who I am and my identity,” said Thomas, now a student at Arizona State University. It was on the internet that Thomas learned about a whole community of young people with similar experiences. Blogposts on Tumblr helped them make sense of their place in the world and what it meant to be bisexual. “I was able to find the words to understand who I am – words that I wouldn’t be able to piece together in a sentence if the internet wasn’t there.”
But now, as states adopt anti-LGBTQ laws and abortion bans, the digital footprint that Thomas and other students leave may come back to harm them, privacy and civil rights advocates warn, and it could be their school-issued devices that end up exposing them to that legal peril.
For years, schools across the US have used digital surveillance tools that collect information about youth sexuality – intimate details that are gleaned from students’ conversations with friends, diary entries and search histories. Also, student information collected by surveillance companies is regularly shared with police, according to a recent survey conducted by the non-profit Center for Democracy and Technology. These two realities are concerning to Elizabeth Laird, the center’s director of equity in civic technology. After the supreme court’s repeal of Roe v Wade in June, she said information about youth sexuality could be weaponized.
“Right now – without doing anything – schools may be getting alerts about students” who are searching the internet for resources related to reproductive health, Laird said. “If you are in a state that has a law that criminalizes abortion, right now this tool could be used to enforce those laws.”
Teens across the country are already organising and disseminating information to fill the void for themselves and their peers in the current climate. Thomas, the ASU student and an outspoken reproductive justice activist, said that while students are generally aware that school devices and accounts are monitored, the repeal of Roe has led some to take extra privacy precautions.
“I have switched to using Signal to talk to friends and colleagues in this space,” they said, referring to the encrypted instant-messaging app. “The fear, even though it’s been common knowledge for basically my generation’s entire life that everything you do is being surveilled, it definitely has been amplified tenfold.”
Police have long used social media and other online platforms to investigate people for suspected violations of abortion laws, including a recent case in Nebraska where police obtained a teen’s private Facebook messages through a search warrant before charging the then-17-year-old and her mother with violating the state’s ban on abortions after 20 weeks of pregnancy.
LGBTQ students face similar risks as lawmakers in Florida and elsewhere impose rules that prohibit classroom discussions about sexuality and gender. This year alone, lawmakers have proposed 300 anti-LGBTQ bills and about a dozen have become law. They include “don’t say gay” laws in Florida and Alabama that ban classroom discussions about gender and sexuality and require school officials to talk to the parents of children who share that they may be gay or transgender.
In a survey, a fifth of LGBTQ students told the Center for Democracy and Technology that they or another student they knew had their sexual orientation or gender identity disclosed without their consent due to online student monitoring. They were more likely than straight and cisgender students to report getting into trouble for their web browsing activity and to be contacted by the police about having committed a crime.
LGBTQ youth are nearly twice as likely as their straight and cisgender classmates to search for health information online, according to research by the non-profit LGBT Tech. But as anti-LGBTQ laws proliferate, student surveillance tools should reconsider collecting data about youth sexuality, Christopher Wood, the group’s co-founder and executive director, told The 74.
“Right now, we are not in a landscape or an environment where that is safe for a company to be doing,” Wood said. “If there is a remote possibility that the information that they are trying to provide to help a student could potentially lead them into more harm, then they need to be looking at that very carefully and considering whether that is the appropriate direction for a company to be taking.”
Tracking youth sexuality
For decades, federal law has required school technology to block access to images that are obscene, contain child sexual abuse images or are deemed “harmful to minors”, and schools have used web-filtering software to prevent students from accessing sexually explicit content. But in some cases, the filtering software has been programmed to block pro-LGBTQ websites that aren’t explicit, including those that offer crisis counseling.
Many student monitoring tools, which saw significant growth during the pandemic, go far beyond web filtering and employ artificial intelligence to identify serious issues, including depression and violent impulses. The tools can sift through students’ social media posts, follow their digital movements in real time and scan files on school-issued laptops – from classroom assignments to journal entries – in search of warning signs.
They have also come under heightened scrutiny. In a report this year, the Democratic senators Elizabeth Warren and Ed Markey warned that schools’ widespread adoption of the tools could trample students’ civil rights. By flagging words related to sexual orientation, the report notes, LGBTQ youth could be subjected to disproportionate disciplinary rates and be unintentionally outed to their parents.
In a follow-up letter in July, Warren and Markey cautioned that the tools could pose new risks after the repeal of Roe and asked four leading student surveillance companies – GoGuardian, Gaggle, Securly and Bark – whether they flag students for using keywords related to reproductive health, such as “pregnant” and “abortion”.
“We are extraordinarily concerned that your software could result in punishment or criminalization of students seeking contraception, abortion or other reproductive health care,” Markey and Warren wrote. “With reproductive rights under attack nationwide, it would represent a betrayal of your company’s mission to support students if you fail to provide appropriate protections for students’ privacy related to reproductive health information.”
School surveillance companies have acknowledged their tools track student references to sex but sought to downplay the risks they pose to students. Bark spokesperson Adina Kalish said the company began to immediately purge all data related to reproductive health after a leaked supreme court draft opinion suggested Roe’s repeal was imminent – despite maintaining a 30-day retention period for most other data.
“By immediately and permanently deleting data which contains a student’s reproductive health data or searches for reproductive health information, such data is not in our possession and therefore not produceable under a court order, subpoena, etc,” Bark’s CEO, Brian Bason, wrote in a response letter to the senators, which the company shared with The 74.
GoGuardian spokesperson Jeff Gordon said its tools “cannot be used by educators or schools to flag reproductive health-related search terms” and its web filter cannot “flag reproductive health-related searches”. Securly did not respond to requests for comment. Last year, a Vice News investigation found its web-filtering tool categorized health resources for LGBTQ teens as pornography.
Gaggle’s founder and CEO, Jeff Patterson, wrote in a letter to the senators that his company does not “collect health data of any kind including reproductive health information”, specifying that the monitoring tool does not flag students who use the terms “pregnant, abortion, birth control, contraception or Planned Parenthood”.
Yet tracking conversations about sex is a primary part of Gaggle’s business, more than references to suicide, violence or drug use, according to nearly 1,300 incident reports generated by the company for Minneapolis public schools during a six-month period in 2020. The reports, obtained by The 74, showed that 38% were prompted by content that was pornographic or sexual in nature, including references to “sexual activity involving a student”. Students were regularly flagged for using keywords like “virginity”, “rape” and, simply, “sex”.
Patterson, the Gaggle CEO, has acknowledged that a student’s private diary entry about being raped wasn’t off limits. In touting the tool’s capabilities, he told The 74 his company uncovered a girl’s diary entry, where she discussed how the assault led to self-esteem issues and guilt. Nobody knew she was struggling until Gaggle notified school officials about what they had learned from her diary, Patterson said.
“They were able to intervene and get this girl help for things that she couldn’t have dealt with on her own,” Patterson said.
Any information that surveillance companies collect about students’ sexual behaviors could be used against them by police during investigations, privacy experts warned. And it’s unclear, Laird said, how long the police can retain any data gleaned from the tools.
Internet search engines are “particularly potent” tools to track the behaviors of pregnant people, according to a recent report by the non-profit Surveillance Technology Oversight Project. In 2017, for example, a Mississippi woman was charged with second-degree murder of her stillborn foetus after police scoured her browser history and identified a search for an abortion pill.
Gaggle has also faced scrutiny for including LGBTQ-specific keywords in its algorithm, including “gay” and “lesbian”. Patterson said the heightened surveillance of LGBTQ youth is necessary because they face a disproportionately high suicide rate. A company spokesperson shared examples where the keywords were used to spot cyberbullying incidents.
But now, in the era of “don’t say gay” laws, digital surveillance tools could be used to out LGBTQ students and put them in danger, Wood, of the non-profit LGBT Tech, said. Student surveillance companies can claim their decision to include LGBTQ terminology is designed to help students, but historically such data have “been used against us in very detrimental ways”.
Companies, he said, are unable to control how officials use that information in an era “where teachers and administrators and other students are encouraged to out other students or blame them or somehow get them in trouble for their identity”. In Texas, the Republican governor, Greg Abbott, issued a February directive calling on child protective services to investigate as child abuse any parents who provide gender-affirming healthcare to their transgender children.
“They can’t control what’s going to happen in Florida or Texas and they can’t control what’s going to happen in an individual home,” where students could be subjected to abuse, Wood said.
‘I don’t trust adults’
Susan, a 14-year-old from Cincinnati, knows first-hand how surveillance companies can target students discussing their sexuality. In middle school, she was assigned to write a “time capsule” letter to her future self.
Until Susan retrieved the letter after high school graduation, her teacher said that no one – not even him – would read it. So Susan, who is now a freshman and asked to remain anonymous, used the private space to question her gender identity.
But her teacher’s assurance wasn’t quite true, she learned. Someone had been reading the letter – and would soon hold it against her.
In an automated May 2021 email, Gaggle notified her that the letter to her future self was “identified as inappropriate” and urged her to “refrain from storing or sharing inappropriate content”. In a “second warning” sent to her inbox, she was told a school administrator was given “access to this violation”. After a third alert, she said, access to her school email account was restricted. She said the experience left her with “a sense of betrayal from my school”. She said she had no idea words like “gay” or “sex” could get flagged by Gaggle’s algorithm.
“It’s frustrating to know that this program finds the need to have these as keywords, and quite depressing,” she said. “There’s always going to be oppression against the community somewhere, it seems, and it’s quite disheartening.”
School administrators reviewed the time capsule letter and determined it didn’t contain anything inappropriate, her mother, Margaret, said. While Susan lives in an LGBTQ-affirming household, Thomas, who grew up in Mississippi, warned that’s not the case for everyone.
“That’s not just the surveillance of your activities, that’s the surveillance of your thoughts,” Thomas said of Susan’s experience. “I know that wouldn’t have gone very well for me and I know for a lot of young people that would place them in a lot of danger.”
Such harms could be exacerbated, Margaret said, if authorities use student data to enforce Ohio’s strict abortion ban.
A Cincinnati public schools spokesperson, Mark Sherwood, said in an email that “law enforcement is immediately contacted” if the district receives an alert from Gaggle suggesting that a student poses “an imminent threat of harm to self or others”.
With a strict six-week abortion ban in place in Ohio, Susan said she was concerned that student conversations and classroom assignments that discuss gender and sexuality could wind up in the hands of the police. She lost faith in school-issued technology after her assignment got flagged by Gaggle.
“I just flat-out don’t trust adults in positions of power or authority,” Susan said. “You don’t really know for sure what their true motives are or what they could be doing with the tools they have at their disposal.”
This report was co-published with the 74, a non-profit, non-partisan news site covering education in the US.