Unexpected Power: Applying the Pigeonhole Principle in Machine Learning
The pigeonhole principle in mathematics is an overstated observation. We have more pigeons than holes; thus, at least one hole must contain more than one pigeon. This is, however, a seemingly contradictory concept. In computer science, artificial intelligence (AI), and machine learning (ML), pigeonholes are surprisingly valuable in practice. For instance, error prediction alongside the row/cell counting problem and its variations in any domain of computer science and teaching a computer to learn—these are all examples of pigeons in a cloakroom. The Pigeonhole Principle Restated Stated more formally, the pigeonhole principle declares: If n houses have n + 1 or more objects, at least one house has more than one object. Sure, it seems to be common sense, but when it comes to data, and more specifically, classification algorithms and the limitations of various models, it implies an explanation for why errors occur and the representation structure of ML data. Example in Clas...