Social News Feed

Designing a Twitter/Facebook-style personalized feed.

Learning Objectives

Real-World Examples

Requirements

• Personalized, ranked timeline per user
• Low-latency reads; freshness with eventual consistency
• Infinite scroll; pagination and caching

High-Level Design

• Write path: store posts, edges (follows), engagement signals
• Fanout-on-write for small users; fanout-on-read for large/fat edges
• Ranking service using offline features + online signals
• Cache hot timelines; background refresh

Capacity & Sizing

• Follower distribution heavy-tail: isolate mega accounts
• Posts/sec during peak; timeline storage per user per day
• Cache budget per user; eviction policy impacts hit ratio

Key Components

• Write Service, Edge store, Timeline store
• Ranking service (features + online scoring)
• Cache (edge and app), Priority queues for recompute

Non-functional Requirements

• P95 timeline fetch < 200ms
• Freshness: new posts reflected within seconds
• High availability for read path

Data Model

• posts (post_id PK, author_id, ts, body)
• edges (follower_id, followee_id)
• timeline (user_id, post_id, score, ts)
• engagement (post_id, user_id, like, comment, share, ts)

APIs

• Publish: POST /api/posts with body { body }
• Timeline: GET /api/timeline?user=:id&cursor=...
• Engagement: POST /api/posts/:id/like, POST /api/posts/:id/comment

Hot Path

1. Read timeline: candidate fetch → rank → serve from cache
2. Publish: write → fanout (hybrid) → cache invalidate

Read Flow

1. Fetch candidate posts (precomputed or on-demand)
2. Rank with recency + engagement features
3. Serve from cache; hydrate missing details lazily

Ranking Strategy

• Offline: feature generation (author quality, interaction graph)
• Online: recency boost, personalization, diversity constraints
• A/B testing of ranking weights and models

Scaling

• Hybrid fanout model based on follower count thresholds
• Cache per-user timelines; invalidate on new posts/engagement
• Async recompute with priority queues for heavy users

Caching & TTL

• Short TTL cache per user (seconds) with jitter to avoid thundering herd
• Background warm for active users; invalidate on new posts

Trade-offs

• Storage for precomputed timelines vs read-time cost
• Staleness windows vs freshness guarantees
• Cold-start ranking for new users

Failure Modes & Mitigations

• Ranking outage → serve recency-only fallback
• Cache miss storms → stagger refresh and add jitter
• Skew from mega influencers → isolate and throttle

Observability

• SLIs: p95 read latency, cache hit ratio, ranking error rate
• Event tracing from publish to fanout to timeline read
• Dashboards: queue depths, fanout lag, cache evictions

Implementation Notes

Best Practices

Common Pitfalls