Google’s AI Search puts schools on alert over student safety
Common Sense Media says Google’s AI-powered Search creates unacceptable risks for children, while Google disputes the watchdog’s testing.
By Poppy Nakagawa · Culture Writer
3 min read
Google Search, the classroom staple that once helped students dodge dodgy web pages, is now under fire from child safety advocates over its built-in AI answers.
A new report from Common Sense Media concluded that Google’s AI-powered Search poses an “unacceptable risk” to young users. The watchdog said students should stop using it until educators have a way to switch off the AI features.
The concern centers on AI Overviews and AI Mode, Google’s chatbot-style Search experience. Google launched AI Mode for Search last year, and it was made available broadly, including to education accounts, according to Mashable.
Common Sense Media and experts interviewed by Mashable said that is alarming because AI Mode is powered by Gemini models. Those models are age-gated in other Google products, including parts of Google’s education technology suite.
Why advocates are worried
Robbie Torney, senior director of AI programs at Common Sense Media, told Mashable that Google has enormous influence in both search and education. Google products are used across schools through Chromebooks, Workspace for Education and other classroom tools, serving users from kindergarten through higher education, according to the report.
Advocates told Mashable that Google has made generative AI so central to its products that many users may not realize they are using it. That creates a problem for children who are asked to search the web for schoolwork.
Researchers have found that Google’s AI Overviews can repeat misinformation, according to reporting cited by Mashable. Experts said the design of AI answers can make them seem authoritative, which may confuse adults and children alike.
Common Sense Media also found that Google’s AI tools would complete homework assignments for minors. Experts told Mashable that the mix of confident answers, shifting responses and chatbot-style agreeableness could lead students to cheat without realizing it, or make learning harder.
Amanda Bickerstaff, deputy director of AI for Education, told Mashable that Google is placing AI into Search in ways that are not appropriate for children’s development.
“There is no reason why a 10-year-old, 13-year-old, or even 15-year-old should see an AI Overview or have AI Mode,” Bickerstaff said.
No simple off switch
Google does not offer a setting that turns off only AI features while leaving Search available, according to Mashable. Parents can disable Search entirely or block Google.com for managed child accounts, and school administrators could theoretically block the Search domain for Workspace for Education users.
Bickerstaff said that leaves schools with an unrealistic choice. “What are you going to say? That kids can’t search the web?” she told Mashable.
Google pushed back in a statement to Mashable, saying Common Sense Media’s stress testing was narrow and ambiguous, and not an effective way to measure product safety and helpfulness. Common Sense Media tested consumer accounts, not Workspace for Education accounts. Torney said Google told the group the AI-powered Search experience is the same for all users.
Schools face pressure
David Monahan, campaign director for the child safety nonprofit Fairplay for Kids, told Mashable that schools should stop using Google products if the company will not let them disable AI features for students.
Anya Meksin, deputy director of Schools Beyond Screens, said the findings should alarm districts. Her group and Fairplay are among organizations calling for years-long moratoriums on generative AI in schools.
Meksin told Mashable that districts may already be struggling to enforce their own AI rules because the technology is embedded inside tools students already use. She pointed to Los Angeles Unified School District’s policy barring students under 13 from using generative AI in school, while students can still access AI Mode through Google Search on school devices.
She said districts should use their purchasing power to demand stronger controls from tech companies, including controls for generative AI built into school devices and services.
This story draws on original reporting from Mashable.