You're given a large text file that doesn't fit in a single machine. How will you count the number of words in the file. Distributed Word Count
In Interview Camp's Live Session today, we discussed Distributed Processing - a common interview question. When does this come up? Let's say you're done coding an algorithm problem. A common follow-up is - what if this input data doesn't fit in memory? How will you solve this problem now?
In last week's session, we had covered Scaling an algorithm within the same machine. In this session, we covered processing the algorithm on multiple machines - a distributed system.
This involves splitting the large file into chunks and processing it in multiple machines. There's a lot more details we covered, including designing a distributed processing system.
We hope you found these useful. To access these sessions live, please sign up at Interview Camp