Recently, I had a conversation with my TL about selecting a subset of items from a large list of items (100s, 1000s?). The problem we were running into was that displaying all of the items at once was hanging the browser. Specifically, the widget we were using would completely hang IE7 when there is more than 200 items in the widget. I commented that this was a showstopper since we officially support IE7 and that we had to solve this, and the TL just responded with "If IE7 cannot handle such a large number of items, then we can't support this request." At first, I was annoyed at his flippant attitude since this basically meant we weren't going to be supporting a feature that seemed very likely (e.g., supporting a large list of items that needed to be selected), and he was the TL. He was supposed to help me solve this problem! Couldn't we do paging? Maybe we could do some fancy smancy scrolling + AJAX to get data in multiple sections! We could NOT just throw up our hands and say "Oops, this is a problem. Sucks for you."
After adamently arguing for a while, I realized that my TL wasn't really saying what I thought he was saying - which was "Give up." Instead, he was telling me to think outside the box of what the user is actually trying to do. Specifically, it doesn't seem likely that the user would ever want to be shown 100s of items at once so that he can make a selection. If he's looking at 100s of items and only wants to select a few, there must be some sort of criteria that he is using that we could probably leverage to help make his selection easier. By limiting his selection of items based on some filtering criteria, not only do we make the user's life easier (no need to see 100s of items when he only is looking for 3 items), we also completely sidestep the browser constraints when showing 100s of items! Looking at it this way definitely makes much more sense than insisting that "we must support showing 100s of items at once via
some fancy widget!"
I think the main "caveat" of going with this type of solution instead of the "fancy widget" solution is that each developer will now need to figure out how to create a useful filter for the user for each time there is a large data set. There is no way to find a "silver bullet" filter that will correctly (usefully?) filter large data sets in all cases. The "fancy widget" solution would definitely solve the problem with all large data sets since it'll always be able to (somehow) show all of the data, but upon thinking about user goals just a little bit more, it just doesn't make sense to build a fancy widget that does exactly what we think it needs to do, but completely misses the user goal's of why he is looking at this data set in the first case.
