A Commonplace Book

When it was proclaimed that the Library contained all books, the first impression was one of extravagant happiness. All men felt themselves to be the masters of an intact and secret treasure. There was no personal or world problem whose eloquent solution did not exist in some hexagon.

... As was natural, this inordinate hope was followed by an excessive depression. The certitude that some shelf in some hexagon held precious books and that these precious books were inaccessible, seemed almost intolerable.

The vast accumulations of knowledge--or at least of information--deposited by the nineteenth century have been responsible for an equally vast ignorance. When there is so much to be known, when there are so many fields of knowledge in which the same words are used with different meanings, when every one knows a little about a great many things, it becomes increasingly difficult for anyone to know whether he knows what he is talking about or not. And when we do not know, or when we do not know enough, we tend always to substitute emotions for thoughts.

The old library systems -- the LoC system the Dewey Decimal System -- imposed categories, which of course, decayed as all categories do. This fiction that every book is in one category required this elaboration of numbers and cross indexing, but, basically, the categories that the librarians worked out helped whittle or, rather, placed walls between the areas of thought. There should be no walls between areas of thought. There should be zones with multiple inter-connectivities across them.

Knowledge can be public,yet undiscovered,if independently created fragments are logically related but never retrieved, brought together, and interpreted. Information retrieval, although essential for assembling such fragments, is always problematic.The search process, like a scientific theory, can be criticized and improved, but can never be verified as capable of retrieving all information relevant to a problem or theory. This essential incompleteness of search and retrieval therefore makes possible, and plausible, the existence of undiscovered public knowledge.

The world of published knowledge certainly contains more than any one person can know and indeed contains more than the aggregate of what all persons know.... public knowledge may remain undiscovered solely because, like scattered pieces of a puzzle, the logically related parts that entail such knowledge have never all become known to any one person. The difficulties of information retrieval have delayed or prevented assembling the components.

Use of stored data [by military intelligence analysts] is intensively interactive; "information retrieval" is an inadequate and even misleading metaphor. The analyst is continually interacting with units of stored data as though they were pieces selected from a thousand scrambled jigsaw puzzles. Relevant patterns, not relevant documents, are sought. Imagine for a moment that the scientific community adopted the culture, attitudes, and metaphors of the intelligence community with respect to recorded information. Scientists might then take seriously the idea that new knowledge is to be gained from the library as well as from the laboratory, through processes of correlation, synthesis, and exploration of the literature.

So instead of knowledge coming together into a great whole, knowledge has been broken up into tens of thousands of isolated corporations or specialist groups. It's meant two things. It's meant, first of all, that society loses all sense of direction, because if everything is separated into little groups that don't really talk to each otherin an honest manner, except to negotiate between each another for power, then there is no possibility to have any kind of directed conversation about society. The second thing is, of course, that it has been very, very bad for each of those areas. The fact of the matter is that sewers run next to autoroutes and hearts lie next to lungs.

The identification of relevant knowledge which has in some way been "forgotten", "overlooked" or "ignored", so that it may be linked to other knowledge to create something new, is one of the major challenges for systems of information and documentation.

We now have access to more knowledge than ever, though anxiety about being overwhelmed is nothing new: Seneca and the Book of Ecclesiastes both fretted there were too many books being produced. But the abundance we face now is different by nature. It not only is abundant, it spills out of epistemological containers.

Dark archives are the repositories of human knowledge to which we no longer have operational access. They are the documents that have been lost, even though they still exist and the records that hold information we don't realize is there.

Mr. Rumsfeld has become, in a way, our patron saint.

You may recall that in the wake of the decision to conduct a retaliatory invasion of Iraq in 2002, Mr. Rumsfeld infamously tried to explain the problems around planning for war. "There are known knowns. These are things we know that we know. There are known unknowns. That is to say, there are things that we know we don't know. But there are also unknown unknowns. There are things we don't know we don't know."

Known knowns. Known unknowns. Unknown unknowns.

If you think about that formulation, you'll see that there is an unspoken fourth quadrant. These are the unknown knowns: the things we don't know that we know.

Thirty years after [Swanson] published his essay, we no longer have to rely on human contrivances alone. Now, with the ubiquity of the internet and the rise of machine learning, a new kind of solution is beginning to take shape. The infrastructure of the web, built to link one resource to the next, was the beginning. The next wave of information systems promises to more deeply establish links between people, ideas, and artifacts that have, so far, remained out of reach--by drawing connections between information and objects that have come unmoored from context and history.

Books represent a search for meaning in a disordered and empty world. In those endless piles, in the thousands of magazines and clippings resides the evidence of a search for order, a thirst for a knowledge which might make sense of the random, unpredictable everyday.

Using just the language in millions of old scientific papers, a machine learning algorithm was able to make completely new scientific discoveries.... [Researchers] used an algorithm [to] sift through scientific papers for connections humans had missed... Using only word associations, the algorithm was able to provide candidates for future thermoelectric materials, some of which may be better than those we currently use.

Good information that's obscured is useless; bad information that's readily available is nefarious.