FeedFabric docsGitHub

FeedFabric

An open-source feed engine with ranking you can tune: timelines with fan-out, a learned For You ranker, ranking profiles, and content safety that runs on your own servers.

What it is

FeedFabric is the back end of a social feed, written in C# on .NET 10. Members post, comment, like, repost and follow; FeedFabric builds two feeds from that (Following, newest first from the people you follow, and For You, ranked) and keeps both safe with checks on every post, comment and photo. It has no accounts of its own: your identity provider signs people in, and FeedFabric trusts its tokens.

You bring the front end. These pages explain how each part works, then walk through building a member-facing UI on the REST API: feeds with cursor paging, post cards, posting with photos, and the reading signals (impressions, dwell, opens, Not interested) that the ranker learns from.

The moving parts

FeedFabric's architectureYour identity provider issues tokens to your app and the admin panel, which call FeedFabric.Api over REST. The API reads and writes Redis, PostgreSQL and blob storage. FeedFabric.Worker.Events publishes the Postgres outbox to Kafka, consumes it, and updates Redis timelines, scans photos with ClamAV and writes blobs. The ml folder trains the ranker from the feed log in Postgres and publishes ONNX models to blob storage. The simulator uses the same services against the same Postgres and Redis.Identity providerAuth0, Entra ID, Keycloak…Your appyour member UIAdmin panelweb/admin · staff onlySimulator1,000 simulated membersml/train_ranker.pyLightGBM → ONNXFeedFabric.Api/userfeed /follows/favourites /media/admin /ranking (staff)text moderationFor You rankingfeed log and signalsFeedFabric.Worker.Eventsoutbox → Kafkafan-out · engagementphoto checks · embeddingsState and infrastructureRedistimelines, For You sessions, signalsPostgreSQLposts, users, FeedLog, outboxBlob storagephotos and ONNX models (disk or S3)Kafkaposts · engagement · media topicsClamAVmalware scan (switchable)tokenRESTRESTreads,writesconsumesFeedLogONNXsame services, same Postgres and Redis
Your app only ever talks to the API. Everything slow or fan-shaped (timelines, counters, photo checks, embeddings) happens in the worker, driven by events the API writes to Postgres in the same transaction as the change.
PartWhereWhat it does
APIsrc/FeedFabric.ApiThe REST API your app calls. Checks tokens, creates users on first sight, moderates text, serves and ranks feeds, records reading signals.
Applicationsrc/FeedFabric.ApplicationThe domain: EF Core entities, services, ranking, safety, storage. Shared by the API, worker and simulator.
Workersrc/FeedFabric.Worker.EventsPublishes the outbox to Kafka, then consumes it: fan-out to Redis timelines, engagement counters, photo checks, text embeddings.
Simulatorsrc/FeedFabric.SimulatorSeeds 1,000 users, backfills 14 days of conversations, then posts, reads and engages live, so there's something to rank.
Admin panelweb/adminVue 3 app for staff: dashboard, review queue, reports, users, posts, Feed studio, system settings.
FabricUIweb/fabric-uiThe Vue 3 components and design tokens the admin panel (and this site) are built with. Use them for your own front end.
MLml/Python (uv): trains the For You ranker from the feed log, downloads and fine-tunes the safety models.

Stores: PostgreSQL 17 is the source of truth (posts, users, follows, likes, the feed log, the outbox). Redis 7.4 holds what's fast and rebuildable: timelines, For You sessions, affinity and topic scores, impression and dwell counters, embeddings. Kafka 4 carries events between them.Blob storage (a local folder in development, any S3-compatible service elsewhere) holds photos and the models, which all run on CPU with ONNX Runtime.

A post, start to finish

  1. Your appPOST /userfeed/posts with the member's token, and any photo IDs that passed their checks.
  2. Text checksRules, then the toxicity model. A rule break or threat is refused (422); worse abuse is held; insults aimed at someone are flagged.
  3. One transactionThe post, its tags and mentions, and a PostCreated outbox row are saved together.
  4. Outbox → KafkaThe worker polls the outbox every half second and publishes to feedfabric.posts.
  5. Fan-outThe worker adds the post ID to the author's and every follower's Redis timeline, newest 500 kept.
  6. FeedsFollowers see it on their next Following page; For You can pick it as a candidate.

Details: Posting and safety and Following and fan-out.

Reading For You

  1. CandidatesUp to four sources: who you follow, popular on your topics, liked by people like you, close in meaning to your likes. Plus your favourites.
  2. FiltersHidden posts, your own, old or held ones, and the profile's tag and media filters drop out.
  3. ScoreThe trained model weighted by the profile's goals, the built-in formula, a custom formula, or newest first.
  4. AdjustFreshness, boosts (favourites, tags, authors) and a penalty for posts seen in the last day.
  5. ArrangeVariety rules spread authors and topics, mix in accounts you don't follow and add exploration slots.
  6. SessionThe order is kept in Redis for 15 minutes; your app pages through it with an offset cursor.

Every post served is logged with the signals it was scored on. What the reader does next (sees it, opens it, likes it, reads it for ten seconds, scrolls past) labels that row, and ml/train_ranker.py learns from those labels. Details: For You ranking and Feed log and training.

What a front end does

  • Gets a token from your identity provider and sends it as Authorization: Bearer. In Development, X-Dev-User: <id> stands in.
  • Reads GET /userfeed/home and GET /userfeed/foryou a page at a time, passing back nextCursor.
  • Renders posts: reposts, held posts, tags, photos with short-lived URLs, and the ranking reasons on For You posts.
  • Likes, reposts, comments, follows and favourites with idempotent PUT/DELETE calls.
  • Uploads photos straight to blob storage through signed URLs, then posts once the worker has passed them.
  • Reports what the member actually reads: impressions, dwell, opens, Not interested and Show more. Without these, For You has nothing to learn from.
  • Handles errors by their stable code, never by the English detail.

Where to go next

Run it locallyDocker, make targets, ports, the simulator and training a model.How For You ranksCandidates, scoring, variety, sessions and cursors.FormulasEvery number behind a post's place, and how to write your own.Build a UIA member app step by step, in Vue, React or plain JavaScript.Deploy itImages, infrastructure, secrets, migrations, models and Cloudflare.API referenceEvery endpoint, its parameters and the errors it returns.

Edit this page on GitHub

FeedFabric is MIT licensed and open source on GitHub. These pages are built fromweb/docs; the repository's own docs/ folder has the full configuration, data model and error lists. Found something wrong? Fix it oropen an issue.

Search the docs