@jdp23 I don't see how partial relays would be possible in atproto. say some catastrophic event happens where people were dead set on splitting off from bsky the corporation. assume it's truly the top priority and nothing else goes until it happens. assume further still this is some unimaginable proportion of the userbase acting in concert - hell, say 25% want to go all at once. best case scenario for making an independent relay.

you create a new relay, migrate data to new PDSes, get that new relay to crawl the PDSes, so far so good. Now what tho? everyone on the new relay is invisible to everyone on the old relay and vice versa. you are back to 0 appviews and 0 feed generators because they all are listening to the main relay. every single appview and feed generator now needs to choose to listen to the new relay. but why would they? you're still responsible as an appview or feed generator for the content you distribute, and you don't know who this new relay is. that's assuming there's no ill will in such a massive split.

so you set up a new basic set of appviews and feed generators. do they also listen to the main relay? do you mirror the old relay in the new relay? do you let the old relay crawl the pdses too? if so, what was the point of the split? now you need to redesign all the existing appviews and feed generators in flight to deduplicate records, which is possible since they're content addressed, but i would doubt they're designed to handle multiple relays because none have existed before now.

what about DIDs? most of the existing infrastructure is designed to just use PLC, which is just a lookup table that bsky also owns. shoot. but we're saved by magic here, because remember there is no acrimony in this enormous network redefining split! So say bsky the corporation is kind enough to keep letting people register DIDs with PLC. we didn't quite make the clean break we were after, but hey it's only the fundamental ability to exist on the network that we were unable to leave behind, and we'll always be reliant on bsky's goodwill for that until someone makes a DID method that works and then we redesign all the appviews and feed generators again.

So now after all that... we're still invisible to most people on the main relay?! oh right because bsky the corporation also provides the default feeds, and despite the high numbers claimed in the press releases, alternate feeds are actually only sparsely used and as a rule very simple hashtag/account feeds because doing anything else is ridiculously expensive. Bluesky the appview is provided by bluesky the corporation, and that's what's actually fetching and hydrating the feeds for us anyway, so even if the feed generators swap over, we'd still be invisible to everyone still on bsky the app. More magic! bsky the appview chooses to crawl and hydrate our posts. We're pretty far from our initial intention of a clean break, but what choice do we have? Now we're partially viewable, some of the time, on some non-default feeds, and there's no way at all to tell within the interface which those are. All it took was totally redesigning most of the network and an enormous amount of goodwill.

What about labels? What about all the automated content moderation bsky the appview does like scanning images and etc? Who moderates? How? Who's paying for all this anyway? The new relay is bound to be extremely expensive - either it's too small and you don't have the critical mass to make any of the above happen, or it's very large and you run into exactly the same problems of scale that necessitate bsky the corporation to need seed funding and eventually make a revenue model on. Where on fedi people pay for servers and donate to their instance because it's a visible part of their experience with moderators they know and like, now all that labor is diffused among a bunch of anonymous service providers - this is by design! It was supposed to depersonalize the network and make it so everyone is just an interchangeable part that you can shop around between. What keeps people donating to the new PDSes, the new relay, the new appviews, the new feed generators? How would they even know how to do that?Meanwhile the network is continuing to tack on features with some combination of bsky corporation fiat, behind the scenes server magic, and so on, so the best we can hope for is partial compatibility and an always-inferior experience.

And that's just to get to 2 relays. what about 3? Remember how much people complained about how hard it was to find an instance? That's absolutely nothing to the combinatoric complexity of PDS * relay * feed generator * app view. How on earth will anyone know how to follow and talk to their friends? To see your friend's post, if they are not on the main relay, you need to get just the right combination of parameters. Even in this perfect scenario with unlimited resources, attention, goodwill, and organization, we couldn't even manage to make a clean break and still have to be reliant on bsky for basically the entire stack, at least partially.

So maybe some small, closed group could make subnetworks, and that is lovely! i'm glad that tech is out there. There's no such thing as privacy on those networks unless they redesign indigo, but hey it's a start! But that looks nothing like the interoperable paradise that's on the label.

In reality we don't get perfect conditions though, and so we'll get stuck at step one: new relay, zero appviews, zero feed generators, zero visibility, and zero people. Again I don't think alternate relays are possible with atproto -- if they were, then there would be no reason to invest $13 million dollars in bluesky.

#atproto #bsky #bluesky #fediverse

I see it somewhat differently. Many apps don't care about most of the data on the broader Bluesky network, so could have their own partial-network Relays as well as AppView. There's already one AppView that doesn't use the Bluesky Relay at all, jus goes directly to PDSs. Or, you could have AppViews that access both closed-subnetwork Relays and (optionally, if they want to plug into Bluesky's chokepointed world) Bluesky's big relay. A PDS, partial-network Relay, and AppView together (and maybe Labelers and Feed Generators) are an instance-equivalent; federations of instance-equivalents can have Relays as well.

I don't think it's inherently any more complex from a user perspective than the Fediverse's situation, where you have to choose an instance and an app and apps only work with specific instances (although I don't also think it's inherently any less complex either).

So maybe some small, closed group could make subnetworks, and that is lovely!

Just as it's lovely on the ActivityPub Fediverse! But when it comes to the full-network view, ActivityPub is implemented today also has a chokepoint: it favors big instances who are connected broadly (and have the disk space and processing power to maintain a whole-network index). That's one of the reasons Meta looked at the ActivityPub Fediveres and say "yum". Of course ActivityPub doesn't have to be that way, The Website League is taking a different approach ... it'll be intersting to see if it takes less than six years for somebody to try something analogous on AT.

That looks nothing like the interoperable paradise that's on the label.

And it looks nothing like what Bluesky is today, so we'll see if it happens.

(Although, I don't think Bluesky's saying they'll have interoperability, their focus is more in terms of identity: you can use your AT identity (on Bluesky, self-hosted, or eventually maybe hosted by a provider) to access other AT apps. Of course there's no reason in principle you couldn't take that in the Fediverse as well, the current software just hasn't been designed that way.)

@jonny

@jdp23 oh it's definitely true you can make apps on it that have some limited view of the network, that's useful, but that's not what they're promising and not what is being invested in. the critical part is that in that arrangement it becomes almost equivalent to nostr with a bilayer network, app views and pdses, without having any sort of server-to-server protocol: so they could just be mutually unintelligible islands. which is fine, but not what is being promised. and I think when most people think of account portability they think of "if i don't like this, i can take my data and go to something similar in kind" and there just isn't that with respect to the bluesky that people are familiar with now.

i could see the combined pds/relay/appview (which is also what bluesky currently is to 99.9999% of people). it's still substantially different from the AP fedi (which as usual i will remind others who i have not spoken about this with before and might be wandering by, i do not think is close to ideal either) in ways that are good -- i can be on multiple at the same time with the same identity, and also in ways that are bad - there's nothing in-protocol that anyone anywhere else can do to see us. As a pds, i can ask a relay to follow me, but i can't ask a relay to follow someone else. so without being on the main relay all interoperability between appview-like-things is lost in a way that's less recoverable than AP fedi. that is, unless everything is obligately indexed by the main relay as the point of interop, and we're back in chokepoint territory. and the series of mutually non-intelligible apps built on the social graph is exactly the business model that i think will be lucrative and drive them to shore up dependency on the relay to keep revenues coming. we haven't even talked about the way that advertising is going to play out in that kind of system, lest we not forget that privacy is impossible in the system too.

There will probably be some ad-hoc relay-to-relay protocol at some point, and one of the strong points of atproto is the fact that things can be relayed without needing to do auth back to the origin every time, but we're already seeing the limits of that with that lightweight firehose that people are having to rely on: sure we can make it cheap, but you have to throw away all the authentication parts. It's sort of this unhappy middle ground between true p2p and AP federation - AP federation works because it clumps resources and communities together and makes it manageable, but atproto is not really fit to be truly p2p either, unless you throw away most of the federation architecture and just use the pds and lexicon system, which are the most interesting parts to me.

ActivityPub is implemented today also has a chokepoint: it favors big instances who are connected broadly (and have the disk space and processing power to maintain a whole-network index).

I'm working on this and it's actually not that hard to overcome. backfilling and context expansion took like a week of concerted effort, and in the process i saw a bunch of places where efficiency could be improved. you don't need a whole-network index, and that's exactly because it's possible to get your instance to go exploring in a way that the relay will never be able to.

Agreed that account portability doesn't get people what they're hoping for: if you're banned from the Bluesky Relay and AppView, you essentially vanish from Bluesky and any other apps using that Relay. You've still got the data (which is better than being on a fediverse server that just vanishes before you've backed things up), and at some point in the future there might be other ways you can use it (which is approximately the same as having a backup of your fediverse data, there currently isn't any way to import it into another server but their might be).

And agreed that it the instance/federation-like structure I sketched is different from fedi in ways that are good and bad. Also right now everything there is public, and I kind of handwaved around that ... in principle you could do something with network-level firewalls that limit visibility to specific relays / appviews, but nobody's done it yet as far as I know so there may be gotchas.

In general I think of Bluesky and their investors interest in and intended uses for their protocol very much like Meta and Flipboard et al's interest and intended uses for AP. So yeah they're looking at how to exploit it, and the network they construct will be optimized for exploitation. But that's not the only way to use the protocols.

(Also AP has been in a weird situation for a bunch of years, the implementations to date have fixed around a certain way of using AP while more creative possibilities have languished ... that might be changing but then again I'm also hearing a lot of developers very frustrated with the protocol, which I consistently hear is very hard to program against, and ready to look at something different. So, it's hard to know how things will evolve.)

Your work on backfiling and context expansion is great and I hope it lands in Glitch soon (and maybe even Mastodon proper, although I'm not holding my breath) ... but does it address global search and hashtags? In principle a federated search could avoid any central(ish) index but I'm not sure that's feasible from a performance perspective. The fediverse's current solution on that front is activitypub relays, not great; Mastodon's direction is the Fediscovery stuff, which is architecturally somewhat similar to AT Relays, TBD how that works.

@jonny

@jdp23 yeah, activitypub the whole protocol is a pain in the ass to work with. now that i've been doing it hands on with the masto implementation and i've been doing like several years of work writing data modeling tooling, i don't think that AP per se is the thing that can/should be rescued, but i do think that there are transitional paths ahead. a big thing is that just like the tooling for dealing with linked data objects of any kind is absolutely shit and has important impedance mismatches with typical OOP object systems.

like i said lexicons and PDSes as data and schema structures are the most interesting parts to me technically, and while i find the bsky team's code structure ... impenetrable ... other implementations are more readable to me and i can see why it would be a lot of fun to develop for. basically the "people are less personally invested in the infrastructure and don't expect to have any degree of personal control over it and by the way everything is public" situation sounds like a fun playground. lexicons/xrpc seems like something to learn from and iterate on, atm not being a linked system i think gives them a much easier onramp as like familiar json-schema+json-like blobs, and CIDs are a way better system than DNS-backed URIs for object distribution if you can assume resolution, but a lower "ceiling" because they can't compose naturally (and if i dare utter the cursed phrase, are 'closed world').

it's sort of a similar tension to the one that spawned JSON-LD from the XML RDF family, "this is weird and unfamiliar and highly academic and need to get it to where the web developers are at" but then it didn't quite get there and i think ends up being more awkward than a syntax that can serialize down to JSON. one of the ways i feel about lexicons too is that it might be on the other side of the sweet spot, too web dev like, not sprawly and languagelike enough. but that's something where time will definitely tell and i don't make strong predictions.

but i've typed too many letters today about standards and schema systems in particular today so i'm gonna stop and think about other things, we'll pick this up again in the future i am sure (or tomorrow if you have more thoughts i'm just a little lettered out this evening, a pleasure to chat as always)