Handle millions of records efficiently?

Handle millions of records efficiently

What interviewers test

• Memory pressure awareness
• Streaming & batching

❌ Bad (Loads everything)

var users = db.Users.ToList(); // Memory explosion

✅ Streaming (Best)

await foreach (var user in db.Users.AsAsyncEnumerable())
{

Process(user);

}

✅ Batching

const int batchSize = 1000;
for (int i = 0; i < total; i += batchSize)
{
var batch = GetBatch(i, batchSize);

ProcessBatch(batch);

}

Key principles

• Never load everything
• Prefer streams
• Control memory explicitly