On Mastodon

[pseudomonas]

I think Mastodon will wind up being far far far easier for the Troll Factory to astroturf, once it's big enough for them to bother with in earnest.

There's no-one with overview of the network (by design) so it'd be almost impossible to use network analysis methods to discover inauthentic accounts systematically promoting whatever narrative. It's basically an open door. Especially given that the troll factory can easily create their own nodes in as large numbers as they wish.

Also, I assume some nodes are going to be run by people who have skewed attitudes towards what (non-illegal-in-their-jurisdiction) hate speech etc they allow? I don't know what happens when Node A allows something, Node B blocks Node A, Node C doesn't, and I'm watching two-thirds of a conversation from Node B (between someone from Node B  and someone from Node C which I can view, and someone from Node A which I can't)?

ISTM that there's a high chance nodes who want to avoid weirdness of that sort end up choosing between "We're going to block any node that hasn't blocked node A" vs "We'll block hatespeech on our own nodes but we're not going to cut off entire nodes just because they're run by two volunteers, one of whom is ill, and haven't got the resources to moderate adequately." At which point the network is basically bifurcated. Repeat for various values of Node A with varying levels of awfulness. Obviously morally a node which is actively cheerleading nazis is worse than a node that just run by someone without the resources to moderate away the nazis, but from the outside they look the same. For some nodes, that will be an important difference; for others it won't.

Is there even a way to find out if [prospective node] has a way of funding (or recruiting volunteers for) a moderation/abuse reports team that will scale with their userbase? I get the idea that a lot of mastodon nodes are one or two people who have access to a server and who (at least at the time they set the thing up) had a bit of spare time.

But even if they have the resources to deal with obvious abuse, detecting coordinated inauthentic activity is a different kind of problem. It's vast numbers of accounts who look like perfectly normal, reasonable people to get readers, and every so often, boost the visibility of a political argument that they want to become part of mainstream discourse in whatever their target society is.

Twitter's team to deal with coordinated (troll-factory scale!) inauthentic activity is pretty huge and also pretty well paid (their field overlaps with mine so I see their job ads from time to time; they're hiring pretty senior AI expertise). The fact that they're not doing anything like an adequate job isn't to say that things wouldn't be far far far worse if they weren't trying.

But Mastodon has no theoretical mechanism to detect that, no team whose job it is to stop that, and no money whatsoever going into it. And it's not just that I think most nodes aren't going to be good at it, I think most nodes are not even going to try. So all you need to do if you're the troll factory, is get your accounts onto nodes that aren't trying, but which other nodes are still federating. Next election, in the bag.

[ETA: I'm not saying that Mastodon doesn't have any points where it's better than Twitter or whatever comparator. I'm saying that its federated nature makes it peculiarly vulnerable to certain attacks, and I can't see how these vulnerabilities can easily be mitigated]

Comments

[ewt]

Thoughts:

1) Mastodon instances are, for the most part, way smaller than Twitter. Playing whack-a-mole with spammers and Nazis isn't fun, but finding people who want to do it doesn't turn out to be impossible. I'm not sure AI is a better solution than (relatively) large numbers of people who care and have skin in the game within their own communities.

2) Users have a lot more control over their home timelines than on Twitter. The ability to turn off boosts is great on days when the network is all up in arms about something. Similarly, the lack of trending topics is something of a calming influence.

3) Users timelines aren't dictated by an algorithm (that rewards engagement for advertising purposes etc). There are still outrage cycles, but they feel different in ways I struggle to articulate.

4) If users aren't happy with the moderation policies on their home instance, they can switch to another one (and I think importing followers/followees is automated now).

No, we cannot stop Nazis creating instances of their own. But putting them mostly in their own echo chamber means that it's possible for the rest of us to participate widely on Mastodon with much less hassle and risk. I am not sure that bifurcation is such a problem in that context.

[ewt]

Also, turning off new account creation, or moderating or making it invite-only, does seem to be at least somewhat helpful in avoiding coordinated inauthentic activity.

[pseudomonas]

Yes, that would definitely help. But if you're still federating content from instances that don't do that, it doesn't make any difference. The adversary will pick the path of least resistance here.

[pseudomonas]

so, for 1, I think we're talking at cross-purposes.

If I was given a timeline from a troll farm account, I probably wouldn't be able to tell that it was inauthentic content. Certainly I wouldn't be able to tell that any given tweet was. It's only by analysing patterns across hundreds of thousands of accounts that one could tell. And I don't see how you can ask volunteers to do that.

https://www.opendemocracy.net/en/odr/troll-factories-kyrgyzstan/ and https://www.rferl.org/a/russian-troll-factory-hacking/31076160.html https://www.diis.dk/en/my-life-as-a-troll-lyudmila-savchuk-s-story are the sort of thing I'm talking about.

[pseudomonas]

with regard to 4; users don't in general *know* that they're being exposed to inauthentic content; so they're not (in general) going to be able to judge which places are better able to keep it from them.

[ewx]

Usenet notably failed to deal with its troll and spam issues and its distributed nature was part of that. (Being first and being unable to escape its own technological history probably didn't help either.)

[jack]

I still don't really understand how federating works, and don't know if mastodon will ever get big, but I have mixed feelings about what if it did.

I see your logic.

But I also feel like a big problem of the current big centralised platforms is that they massively lean into "a giant mosh pit of everyone on the service all watching a small number of celebrities" which isn't that great for users, and means that any kind of moderation needs to "cope" with people interacting with "basically everyone, unless you almost completely opt out". If most ways of interacting are "people you've specifically friended" and "randoms on the same smallish/moderated instance", I think the problem's much smaller. Still potentially big, in that even "friend-of-friend only or backchannel only friend requests" can be a vector for misinformation and abuse, but possibly susceptible to "choosing an instance which tries to cope, or is big enough to have a paid team".

But I guess, I should understand how it DOES work first...