Follow up on #347
To facilitate working with datasets that contain many unique categories, I think we should truncate/hide some of them so that the tables are still easy to read. I don't think it is terribly uncommon for a dataset to include categories in the 100s, e.g. all countries in the world:
This immediately make the table pretty hard to read, even for a simple table like this one (and view in the terminal would suffer similarly if not more). It would be help readability to truncate/hide the displayed categories when there are many values:
As a response to the points raised against truncation, I don't think it matters much that there is some arbitrariness in how the threshold/cutoff for truncation is picked since this is based on what is visually easier to read. What is more important is that we set a threshold that is in the ballpark of what is useful for someone reading these tables (and not off with an order of magnitude). E.g. whether we choose to show 20 or 21 categories is arbitrary, but since either choice fixes the issue with displaying say 1,000 categories, we shouldn't let the inability to perfectly motivate whether 20 or 21 is the exact right cutoff point, prevent us from picking one of them and implementing a useful cutoff that will improve the current situation.
In the previous implementation of this functionality, I truncated the number of values based on the total number of characters to display, which involved computed which categories to hide and show. I'm open to other solutions and think it would be great if less code was involved, but I couldn't think of a cleaner way. Some caveats are mentioned in this comment, including for the display from view in the terminal.
Another suggestion in that PR was to use a separate page for additional values.
Follow up on #347
To facilitate working with datasets that contain many unique categories, I think we should truncate/hide some of them so that the tables are still easy to read. I don't think it is terribly uncommon for a dataset to include categories in the 100s, e.g. all countries in the world:
This immediately make the table pretty hard to read, even for a simple table like this one (and
viewin the terminal would suffer similarly if not more). It would be help readability to truncate/hide the displayed categories when there are many values:As a response to the points raised against truncation, I don't think it matters much that there is some arbitrariness in how the threshold/cutoff for truncation is picked since this is based on what is visually easier to read. What is more important is that we set a threshold that is in the ballpark of what is useful for someone reading these tables (and not off with an order of magnitude). E.g. whether we choose to show 20 or 21 categories is arbitrary, but since either choice fixes the issue with displaying say 1,000 categories, we shouldn't let the inability to perfectly motivate whether 20 or 21 is the exact right cutoff point, prevent us from picking one of them and implementing a useful cutoff that will improve the current situation.
In the previous implementation of this functionality, I truncated the number of values based on the total number of characters to display, which involved computed which categories to hide and show. I'm open to other solutions and think it would be great if less code was involved, but I couldn't think of a cleaner way. Some caveats are mentioned in this comment, including for the display from
viewin the terminal.Another suggestion in that PR was to use a separate page for additional values.