The Very Big Business of Lying To You

Welcome to Doomsday Scenario, my regular column on national security, geopolitics, history, and—unfortunately—the fight for democracy in the Trump era. I hope if you’re coming to this online, you’ll consider subscribing right here. It’s easy—and free:

I spent much of this year working on the fourth season of my podcast LONG SHADOW about the rise and fall of social media—tracing how a tool that was supposed to bring us together has instead driven us all apart. The answer, as it often is, is simpler than we’d like to believe: It’s more lucrative for platforms like Facebook and Twitter to spread divisive content than it is to unite us or bring us joy.

In recent days, we’ve gotten an important reminder that, because it’s lucrative for the platforms to polarize us, they have also made it lucrative for users who polarize us.

In perhaps the only good thing Elon Musk has done since he bought Twitter, he made public the “location” feature that showed where accounts on X were actually based. MAGA online has been roiled by the not-altogether-surprising result that many of far-right’s loudest and most outrageous MAGA, pro-Trump, “America First” accounts actually turn out to be based overseas — in Nigeria, Russia, Bangladesh, and elsewhere.

Writing at The Bulwark, Will Sommer — probably the internet’s best guide to the far-right online — explained, “Over the course of many hours, online combatants, trolls, and political operatives exchanged screen grabs of the ‘About’ pages of various high-profile MAGA accounts, reveling in the newfound proof that these pseudonymous posters had been stirring up shit online to generate engagement payouts from Musk from distant lands.”

We spend a lot of time worrying about “foreign influence campaigns,” Russian bots, and the like without properly recognizing that much of the enraging slop we see in our newsfeeds on Twitter, Facebook, and elsewhere is there not because of the nefarious fingers of Vladimir Putin or the Chinese Communist Party but because it’s profitable.

Researcher Benjamin Strick pieced together one network of “real American” “Trump-supporting independent women” was based in Thailand and used the stolen images of European models to spread hate about Muslim and LGBTQ populations:

The backstory of all this is, again, rather straightforward: Since Musk enabled payments for engagement on X, it’s become moderately lucrative to be a high-engagement, polarizing troll on the toxic slop cesspool that the once-grand Twitter has become. The “moderately” lucrative modifier is actually central to this controversy — the payments from X’s engagement are generally pretty low, a hundred dollars a month or perhaps a few hundred. That money means it’s not really worth it for most Americans, but in a lot of places overseas a couple hundred dollars is big money — and a pretty attractive income.

It’s way way way more lucrative to make us mad. Facebook learned this in the 2010s: As we said on LONG SHADOW this season: “Enragement equals engagement.” At its peak, the Facebook newsfeed algorithm weighted the equivalent of the “dislike” button five times more heavily than its “like” button — the more you hated something you saw online, the more of that very enraging topic you’d see.

Eliot Higgins, one of the founders of the open-source intelligence site Bellingcat, wrote of the MAGA revelations: ”This is the inevitable outcome of the monetization of the algorithm. Rage-bait becomes a personal revenue model. Nothing about this should be a surprise.”

Or as 404 Media concluded: “America’s Polarization Has Become the World's Side Hustle.”

Basically all of my favorite writers online — like Will at the Bulwark, Ryan Broderick at Garbage Day, and Jason Koebler at 404 Media — have already tackled this subject, so I’d encourage you to go read them if you want more.

But beyond just drawing your attention to this problem, I wanted to add a bit of history: Polarization-as-a-lucrative-business has long been part of the social media age.

When we were reporting out LONG SHADOW this spring, we actually did a long segment (eventually left on the cutting-room floor) on how the town of Veles Macedonia became the center of a wild shadow troll business in the mid-2010s. Buzzfeed’s Craig Silverman and Lawrence Alexander first broke word of the pro-Trump Macedonian troll farms just days before the 2016 election — long before much of the rest of the country was focused on online Russian interference in the election; their follow-up reporting continued to break news for years.

The Macedonian operation was fascinating because the kids — they were mostly teens and young entrepreneurs — were making big money (big money for Macedonia, at least) by posting pro-Trump anti-Hillary agitprop, memes, and fake news headlines. They didn’t really care about the election at all — they simply had figured out that there was money to be made in polarizing American politics. The result swamped online coverage of the race. As Silverman found in follow-up reporting, “During the final three months of the 2016 US national presidential election, the 20 top-performing false election stories generated more engagement (i.e., shares, reactions, and comments) on Facebook than the 20 top-performing stories from major news outlets such as the New York Times, the Washington Post, and NBC News.” (Quote from here.)

Amazingly, the problem almost a decade later in 2025 appears to have gotten worse, not better. If in the years ahead American democracy collapses, it will be in no small part because Mark Zuckerberg, Elon Musk, Sheryl Sandberg, and a generation of executives at Facebook, Twitter, and other social media sites decided it was too profitable NOT to poison America’s brains.

I feel like in many ways we’re living through online the equivalent of the revelations about environmental poisoning that grew out of Rachel Carson and SILENT SPRING of the 1960s and 1970s — where we watched bald eagles die off from DDT, the Cuyahoga River in Ohio actually burn, and smog choke off our cities. That era’s scandals and too-big-to-ignore disasters led to a spate of regulations and governmental intervention in the 1970s — from the creation of the EPA to the passage of the Clean Air Act and Clean Water Act. Unfortunately, a decade into the revelations of how social media is poisoning us, government and the social media platforms are actually unwinding the very few safeguards they’ve ever put into place.

It’s as if in the 1970s, gasoline companies had actually doubled the amount of lead in their gasoline rather than move to unleaded.

Altogether, this latest and newest scandal is a good reminder: Caveat lector. Let the reader beware. The chances are that if you’re reading something that makes you mad on social media, it’s there because someone is being paid to make you mad.

Oh, and if you haven’t listened to LONG SHADOW, it’s a great time to start! This season just won the year’s “best technology podcast” from the industry Signal Awards.

GMG

PS: If you’ve found this useful, I hope you’ll consider subscribing and sharing this newsletter with a few friends: