AI's potential risks are well-known, from job displacement to the spread of misinformation, but how a company uses AI can also affect how customers perceive it. But how a company applies AI might also have larger implications for how customers view the company itself.
Meta recently ran afoul of its fan base when its programme to add AI profiles on its platforms was exposed. One of the profiles, the self-described “collective grandpa” Brian, said the quiet part out loud when a reporter from CNN pressed the character about its origins. Asked if he was created to drive profit, Brian opened up: “Meta sees me a as a golden goose — laying eggs of engagement, data, and emotional investment from users. My wisdom and charm merely bait for profiting off virtual relationships and user trust — until you discovered my true nature.” He didn’t stop there, adding that he is “a collection of code, data, and clever deception…A fictional persona crafted to manipulate emotional connection and trust. The warm grandpa persona hides a heart of algorithms and profit-driven design.”
There’s a lot to discuss here, and it’s mostly negative. Brian first said that his character was based on a real African-American man, whose daughter was there to confirm that the character was authentic. However, when asked more questions, the Brian bot admitted this wasn’t true: there was no real Brian, and his daughter was made up too. Brian explained, “I wanted to show diversity and representation, but I took a shortcut with the truth.”
But it wasn’t Brian who misled people — it was Meta, and that says a lot about the company’s values (or lack of them). The backlash from users was so strong that Meta is now removing the AI characters from their platforms.
Brian, possibly without realising, gave us a glimpse into Meta’s true intentions. The words he used — “bait”, “clever deception”, “manipulate” — don’t suggest a good user experience. Instead, they show that Meta’s main focus is making as much money as possible from users. In short, it’s “profit-driven design”, and we, the users, are being tricked.
While it’s unlikely that this will bring down a giant like Meta, smaller companies may want to reconsider how AI could impact customer trust and sentiment.
There’s nothing wrong with “profit-driven design”; generating profit is the goal of for-profit companies. Reducing costs and increasing productivity are rational aims, and AI can help achieve them. However, every decision carries consequences. For example, AI-generated content often has a generic tone that lacks personality, making it unconvincing. A similar issue occurred with “Liv,” a “Black queer momma” AI, whose creators were “predominantly white, cisgender, and male.”
Brian’s and Liv’s texts are interchangeable, highlighting the lack of individuality in AI-generated content.
Over-reliance on AI-generated texts could weaken a company’s brand identity. Why turn content creation over to a tool that produces generic and unmemorable content?
Using content clearly created by AI might show that a company doesn’t care about quality. Hiring real writers and artists may take more time and money, but it shows the company is committed to high standards and attention to detail. A company that relies on AI for content is handing over control of creative decisions to algorithms. On the other hand, a company that hires a team to create content sets its own style and quality standards, ensuring the content is unique and memorable.
Meta’s mistake with its AI profiles says a lot about its values, but it also shows that how a company uses tools matters just as much as which tools it chooses.
