The parts themselves need to be connected and interacting together in a tumultuous dance. (Location 206)
small changes cascade through this network, feedback occurs in the complex system, and there is even a sensitive dependence on the initial state of this system. (Location 207)
but there is nothing happening inside it: the networks of biology—the circulatory system, metabolic networks, the mass of firing neurons, and more—are all quiet. (Location 210)
Autocorrect, which we often deride as being hopelessly stupid for its failures, is actually incredibly advanced, relying on petabytes of data (a petabyte is a million gigabytes) and complex probability models. (Location 220)
But when we fail to have a complete understanding, we fall short in a specific way: we encounter unexpected outcomes. (Location 245)
In fact, this set of rules—developed over decades—is so complex that perhaps only a handful of individuals alive even understand it anymore. (Location 247)
When an outcome is unexpected, it means that we don’t have the level of understanding necessary to see how it occurred. (Location 252)
Abstraction is essentially the process of hiding unnecessary details of some part of a system while still retaining the ability to interact with it in a productive way. (Location 296)
Abstraction allows someone to build one technology on top of another, using what someone else has created without having to dwell on its internal details. (Location 305)
Unfortunately, in the Entanglement, abstraction can—and increasingly will—break down. Portions of systems that were intended to be shielded from each other increasingly collide in unexpected ways. (Location 310)
But how are the decisions made on how to trade? By pouring huge amounts of data into still other programs, ones that fit vast numbers of parameters in an effort to squeeze meaning from incredible complexity. (Location 314)
ways, causing a trillion dollars in lost value for a short period of time. Complex though they are, these systems do not exist in a vacuum. In addition to being part of a larger ecosystem of technology that determines when each specific equity or commodity should be traded, our financial systems are also regulated by a large set of laws and rules. (Location 319)
In the Entanglement, things collide across the many levels of abstraction, interacting in ways we can’t imagine. This web of interactions produces what’s known in complexity science as emergence, where the interactions at one level end up creating unanticipated phenomena at another. (Location 337)
When those tiny little details deep within the system rise up like miniature demiurges and ruin some other portion of the technological system that we have constructed, we can no longer rely on understanding only part of the system. (Location 343)
it’s another thing entirely when no one truly understands that gadget. While many of us continue to convince ourselves that experts can save us from this massive complexity—that they have the understanding that we lack—that moment has passed. (Location 350)
A system developed by researchers to communicate is not ideal for a high volume of smooth and secure commercial transactions. (Location 365)
Happily, the system does work. But beneath the user interfaces of a website lurks a bizarre and complicated structure. Sometimes we even see this mess directly, as users, when we see warnings about security certificates. Things work, but they are far from pretty. (Location 368)
What about software, which undergirds the modern technological systems of every aspect of our lives? One common way to measure the complexity of software is through the number of lines of code it takes to write a program. (Location 400)
To begin with, the most obvious reasons we end up with increasingly complex systems over time are the twin forces of accretion and interaction: adding more parts to something over time, and adding more connections between those parts. (Location 416)
The rules and regulations dictating the procedure for the bridge’s renovation involved forty-seven permits from nineteen different governmental entities—everything from environmental impact statements to historical surveys. (Location 524)
it’s too easy to build and connect systems together. (Location 544)
However, if a problem in the electrical grid renders a large portion of the United States without power, that’s an extremely costly failure. (Location 547)
takes more resources to construct the infrastructure for a banking system than for a chat program. (Location 550)
Technology ultimately connects, interacts, and converges. And when it does, it acts as a further force moving us toward complication. (Location 566)
Systems we build to reflect the world end up being complicated, because the world itself is complicated. (Location 598)
whether in making sure you never miss an appointment, or in building a self-driving vehicle that not only won’t get lost but also won’t injure anyone—things suddenly become a good deal more complicated. (Location 601)
the exceptions that nonetheless have to be dealt with, otherwise our technologies will fail. Edge cases range from the problem of the leap year to how to program database software to handle people’s names that have an apostrophe in them. (Location 603)
As Peter Norvig, Google’s director of research, put it, “What constitutes a language is not an eternal ideal form, represented by the settings of a small number of parameters, but rather is the contingent outcome of complex processes.” (Location 638)
Google Translate’s results can be imprecise, though interesting in their own way. But scientists have made great strides. (Location 647)
Exceptions must be cherished, rather than discarded, for exceptions or rare instances contain a large amount of information. The sophisticated machine learning techniques used in linguistics—employing probability and a large array of parameters rather than principled rules—are increasingly being used in numerous other areas, both in science and outside it, from criminal detection to medicine, as well as in the insurance industry. (Location 664)
In the end, the law turns out to look like a fractal: no matter how much you zoom in on such a shape, there is always more unevenness, more detail to observe. (Location 675)
if a firm becomes too big and its failure is expected to cause a cascading shock, it must be divided up or shrunk. (Location 702)
In other words, there are modules, parts of a system that are tightly interconnected and reasonably self-contained. (Location 710)
These modules are still intimately connected to the rest of the system, whether through other parts of the body or through chemical signals—I do not recommend trying to remove your heart—but they are relatively distinct and can be understood, at least to some degree, by themselves. (Location 712)
Unfortunately, understanding individual modules—or building them to begin with—doesn’t always yield the kinds of expected behaviors we might hope for. (Location 718)
As our systems become more complex over time, a gap begins to grow between the structure of these complex systems and what our brains can handle. Whether it’s the entirety of the Internet or other large pieces of infrastructure, understanding the whole is no longer even close to possible. (Location 735)
While this is a true statement, it completely ignores the fact that software is complex and can fail in many different ways. (Location 754)
The people responsible for ensuring the safety of the Therac-25 misunderstood technological complexity, with lethal consequences. (Location 757)
As the writer Scott Rosenberg notes, the space between machine counting and human counting is an area where we make adjustments in computer code, but it’s also where errors and bugs originate. (Location 766)
The fact that we don’t count from zero, but our computers do, is emblematic of the larger rift between human thought patterns and how large systems are constructed and how they operate. (Location 769)
Recursion is a computer science term that means, essentially, self-reference: it describes a section of computer code that refers back to itself. (Location 785)
Within the philosophy of technology, there is a growing interest in the philosophical implications of software: How should we think about our computational creations? (Location 875)
One thing computers are good at is seeing the underlying story in the data, so [a data center engineer] took the information we gather in the course of our daily operations and ran it through a model to help make sense of complex interactions that his team—being mere mortals—may not otherwise have noticed.” (Emphasis mine.) (Location 900)
One computational realm, evolutionary computation, allows software to “evolve” solutions to problems, while remaining agnostic as to what shape the eventual solution will take. (Location 910)
I have on my shelf three books that have the phrase “The Last Man Who Knew Everything” as their title or subtitle. One is an edited volume about Athanasius Kircher, a German Jesuit priest who lived during the seventeenth century. (Location 945)
build models of the world and new technological systems at the frontiers of what we know, we have had to learn “more and more about less and less”—to specialize in specific domains. (Location 985)
There are ways of trying to overcome this predicament: for example, perhaps it’s time to bring back generalists and polymaths, inviting them to flourish anew in this modern era—a possibility we will reexamine later in this book. (Location 1018)
secure in the belief that they are constructed on a logical foundation, until they confront us with unexpected behavior: the bugs and glitches that send major systems such as global finance into a tailspin. (Location 1022)
This unpredictability and fragility is actually a hallmark of the complex systems that we build. While complicated systems are often incredibly robust to shocks that are anticipated—that is, ones they have been designed for—their complexity can be a liability in the face of the unanticipated. (Location 1069)
known as highly optimized tolerance. (Location 1073)
This deeper problem in calculation—the flaw in the algorithm—was noticed because of an anomaly (in hindsight, not a particularly subtle one): the index was going down while the market was going up in value. (Location 1148)
Through this failure, the designers at Microsoft became better aware of how their program could interact with the raw, unfiltered id of the Internet and result in such hateful output, a type of output that they likely were not even aware was possible. (Location 1157)
Chaos Monkey’s function is simple: it unexpectedly takes Netflix systems out of service. Only by seeing how the vast Netflix system responds to these intentional failures can its engineers make it robust enough to withstand the unexpectedness that the messy real world might throw at it. (Location 1163)
Naturalists who examine the natural history around us have long been comfortable with the miscellaneous. Sometimes they detect an order in the living habits and behavior of the animals and plants they are observing, but there is also a purpose to their observations even without some sort of theoretical order—it allows them to understand and record the details, even if they don’t yet have a complete mental framework for every living thing they see. (Location 1178)
recognize that it is important to know the details, and sometimes even the names, of these different individual pieces, whether or not we know how they fit together. (Location 1184)
Like Fairfax, Newton cataloged observations, but he also uncovered a set of universal principles, often mathematically described, that govern our physical world. (Location 1214)
On the other hand, biologists, as a rule, have a greater comfort with diversity and bundles of facts, even if they are left unexplained by any single sweeping theory. (Location 1240)
For example, the use of mathematics to abstract away details at a grand level is found everywhere in physics, but less often in biology. (Location 1246)
First, biological systems are generally more complicated than those in physics. In physics, the components are often identical—think of a system of nothing but gas particles, for example, or a single monolithic material, like a diamond. Beyond that, the types of interactions can often be uniform throughout an entire system, such as satellites orbiting a planet. (Location 1259)
Second, biological systems are distinct from many physical systems in that they have a history. Living things evolve over time. While the objects of physics clearly do not emerge from thin air—astrophysicists even talk about the evolution of stars—biological systems are especially subject to evolutionary pressures; in fact, that is one of their defining features. (Location 1268)
modifying a system in small ways to adapt to a new environment. (Location 1274)
the honey locust might eventually lose its thorns. If this trait is truly useless, then producing thorns is a wasteful expenditure of energy for the honey locust. Over evolutionary time, the thorn trait will be swept away by the success (Location 1296)
Technologies can appear robust until they are confronted with some minor disturbance, causing a catastrophe. (Location 1301)
We need “field biologists” to catalog and study details and portions of our complex systems, including their failures and bugs. (Location 1327)
But biologists do much more than simply learn from these glitches. To better understand how to think like a biologist, we must look at how they conduct their work more generally. (Location 1330)
“The most exciting phrase to hear in science, the one that heralds new discoveries, is not ‘Eureka’ but ‘That’s funny . . .’” Penicillin was discovered after Alexander Fleming saw something odd on a petri dish. (Location 1340)
purpose. When a system is so complex that it is hard to anticipate how, exactly, it might respond—and what changes in a genome might yield the desired effect—one often needs to use a certain amount of randomness to find out what the system can do. (Location 1350)
Applying biological thinking to technology involves recognizing that tinkering is a way of both building a system and learning about it. As Stewart Brand noted about legacy systems, “Teasing a new function (Location 1357)
There are now computational tools that can help find unexpected outcomes in a system, part of the domain known as “novelty detection.” Machines might be able to act in partnership with these field biologists and naturalists, helping us to better understand—even if only partly—our own technologies. (Location 1370)
So, when confronted with a complex piece of technology, we must begin by acting like field biologists, experimenting around its edges to see how it behaves, with the end goal of some degree of generalization. (Location 1424)
You first collect huge amounts of information about your virtual world—what you can do, what you can’t, what kills you, how you successfully survive—and then begin to make little mental models, small-scale generalizations within a much larger whole. (Location 1426)
For example, if you look at a massive network of interactions—such as who follows whom on Twitter—you can ignore the details of the interactions and note that the number of connections between individuals follows a specific category of probability curve known as a heavy-tailed distribution. (Location 1441)
how something diffuses based on the structure of some sort of porous space, such as how petroleum moves through a (Location 1445)
“The patterns of a river network and of a retinal nerve are both the same and utterly different. It is not enough to call them both fractal, or even to calculate a fractal dimension. (Location 1469)
There was a desire to chronicle and ponder the unexpected and the weirdest aspects of the natural world. As Daston notes, “The first scientific facts were stubborn not because they were robust, resisting all attempts to sweep them under the rug, but rather because they were outlandish, resisting all attempts to subsume them under theory.” (Location 1520)
T-shaped individuals have deep expertise in one area—the stem of the T shape—but breadth of knowledge as well: the bar of the T. (Location 1549)
Generalists can be found in such areas as consulting and book editing, and you can even find them in the world of venture capital, where there are many people who are knowledgeable in multiple different areas and can use this expertise widely. (Location 1553)
In fact, it seems that the places where generalists can thrive best are the places where we understand the least, where the systems are so complicated and interconnected that the best we can do is hope for a chronicling of the miscellaneous. (Location 1569)
but also learning ways of exploring the unknown, the new, and the unexpected. (Location 1571)
Of course, generalists alone are relatively useless. They are best when working alongside specialists, helping in the process of translation and communication, or playing other roles complementary to the still-important contributions of specialists. (Location 1577)
Simulations are a way to provide us with the beginnings of intuition into how a complex technology works. (Location 1765)
There are two potential extreme responses to mystery in our complex technological systems. The first is to underemphasize it, to such a degree that there are no mysteries. (Location 1827)
But they’re back. The confluence of three trends—better algorithms, huge amounts of data, and new and improved computer hardware on which to run these networks—has engendered new deep neural networks that not only can compete with other AI technologies, but surpass the more traditional approaches. (Location 1907)