debugging journal

READING ORDER: top down!!!

%at=2024-11-15T01:41:24.349Z

current issue, relay errors on new post

though the posts do appear on frontend after a reload, but it seems it's shelling out to pds on getProfileFeed?

relay:

its so weird that it'd fail on something that it knows in the switch case..

	switch rec := op.Record.(type) {
	case *bsky.FeedPost:
		if err := ix.handleRecordCreateFeedPost(ctx, evt.User, op.Rkey, *op.RecCid, rec); err != nil {
			return nil, err
		}

added some more debugging and it's nil wtf

2024-11-14T22:46:37.647-0300 ERROR indexer indexer/indexer.go:100 failed to handle repo op {"err": "handle recordCreate: unrecognized record type (creation): app.bsky.feed.post (actually of type %!s(<nil>), record=<nil>)"}


weird weird weird!!!

feedgen shenanigans part 0 #

%at=2024-11-15T01:49:44.621Z

the Following feed has my own test user's post, but errors out because of the unable to find feed error since i don't have a feedgen, i think that's going to be my focus next

feedgen shenanigans part 1 #

%at=2024-11-15T21:02:28.085Z

using jaz's golang feedgen https://github.com/ericvolp12/go-bsky-feed-generator

i was able to spin something up that could sorta reproduce the followers feed here https://github.com/lun-4/atproto/tree/luna.gluesky/luna/feedgen2

still wip, currently learning how to link it all together in the appview

feedgen shenanigans part 2 #

%at=2024-11-15T23:46:21.520Z

configured a feedgen. took a bit of a woozy because there's url validation on the responses given by appview, and since i didn't configure CDN url it generated an invalid feedgen url (undefined/img/avatar/did:plc..../). i had to create a small shim that extracts ImageProcessingServer into its own configurable process because it seems that the appview default logic (which is to spin its own image processing server) is just broken. maybe. idk. got something that works and looks more and more like real production bsky. i shouldn't have tested the feedgen with an avatar lmao

feedgen soruce code for Followers feed

• https://github.com/lun-4/atproto/tree/luna.gluesky/luna/feedgen2
• main code for followers logic in https://github.com/lun-4/atproto/blob/luna.gluesky/luna/feedgen2/pkg/feeds/luna/firehose.go
  • syncs from firehose at cursor=0 and stores all follow relationships and post at-urls in sqlite
  • this is done so that i don't need to create a dependency on the dataplane

notes for upstreaming:

• the cdn shim requires changes to which types are exported out of the bsky package
• goat was modified so that its Default vars can be overriden by env var.
  • so that i can override to my did:plc server without having to patch source code
  • would be welcome to PR that to indigo very soon. sounds very useful esp. in a future beyond plc.directory

%at=2024-11-19T05:02:04.492Z

haven't setup jetstream or ozone over the weekend, but i want to. after that is burning this all down and redoing everything with proper public documented steps

started upstreaming at least One(tm) patch: let ATP_PLC_HOST override default plc host <a href="/wiki/_/tags/823.html">#823</a> https://github.com/bluesky-social/indigo/pull/823

ozone yipee #

%at=2024-11-21T04:13:09.060Z

got ozone running, no patches needed. tough luck if you typo the OZONE_PUBLIC_URL var tho. account seems to remain broken, might fix itself if i edit the plc record manually, but i cba'd to do it, spun up second account instead

• https://github.com/lun-4/atproto/blob/luna.gluesky/luna/ozone/run.sh

also got pdsls running with minimal patches needed so i can also debug records easily (thx @futanari.tentacle.expert)

• not in gluesky repo since it's not really env vars that i set but patches to frontend source code, but search for plc.directory and bsky and you'll get the places to patch

soon: jetstream! then burning all of this down and spinning it back up with step-by-step documents.

jet streams #

%at=2024-11-21T23:10:10.727Z

easy at first

• end result at https://github.com/lun-4/atproto/blob/luna.gluesky/luna/jetstream/run.sh

...well, save for this:

so, uh

export JETSTREAM_LIVENESS_TTL=9000000000s

lol, lmao, etc

soon: spin everything again while writing step-by-step docs so that i know what to upstream later