Rust async stream processing: walking a 4 GB event log without blocking the runtime
The Log
Mon, Oct 26
We ingest a rolling 4 GB event log across hundreds of thousands of lines, and the first cut blocked a worker on every 8 MB page fetch. Here is the async stream version that keeps the runtime hot end to end — patch, review comments and all.
62 like
19 love
28 insight
Delete this post? You can undo for 30 seconds.
Pulled the N+1 out of the dashboard post list: 140ms → 18ms
Shipped a small but satisfying refactor today: pulled the N+1 out of the dashboard post list by eager-loading category + tags in a single query. Page went from 140ms to 18ms. The boring wins are the best wins.
24 like
6 love
3 insight
Delete this post? You can undo for 30 seconds.
Sat, Oct 24
chunkById vs lazyById for a 500k-row export on a busy primary?
Best practice for chunking a 500k-row export?
lazy() seems to hold the connection hostage in worker queues — anyone benchmarked chunkById vs lazyById on a busy primary?
8 like
1 love
3 insight
Delete this post? You can undo for 30 seconds.
That's the root post. There's nothing before it.