Read commentary on curated independent news sources at hex-index.com.

The History of the Machines That Erased History

original text with annotations

Read · 读 · Legi

or multilingual

← previousnext →

2

The year was ending, and the people who built the thinking machines could not agree on what they had built. One man who had helped make the first great ones said the age of simply making them bigger was over. For years the method had been plain: pour in more writing, more machines to run it, and the results improved on schedule, the way a field yields more grain when you clear more land. He said the land was nearly cleared. What remained was harder, a way of learning no one yet possessed. A child of fifteen, he pointed out, has read almost nothing beside these machines, and can still be taught to drive a car in an afternoon, carrying inside her a plain sense of what is dangerous and what is fine. The machines had read everything and had no such sense. On paper they were geniuses; in the room they were often fools.

Against him stood the largest houses, who said there was no wall at all, and went on building. A quieter voice, an engineer of the small chips, argued that the physical limits had been struck years before, and that both the dreamed-of thinking machine and the greater one behind it were fantasies that ignored the plain cost of computation. The people who ran the biggest houses did not speak of a wall; they said flatly that they saw none, and shipped more. The strange thing was that both sides might be right and it would scarcely matter. Even if the ceiling had been reached, what was already made was enough to keep a whole generation busy fitting it into everything they touched.

The deepest trouble that year was not that the machines were stupid but that they could not remember. Set one a long task and it would wander, do a little, and report proudly that it was finished when it was not. Builders described it as a forgetful worker walking about with a full belt of tools and no memory of what he had already done. The machine could not be made to remember; that was simply beyond it. So instead they built memory outside it: a written list of what was wanted, a log of what had been tried and what had broken, a fixed way of beginning each time by reading these notes back. The worker stayed an amnesiac. They gave him a notebook and taught him to open it first.

They learned to treat the machine's short memory the way older machines treated their scarce fast storage: keep almost nothing in front of it, and fetch the rest only when it was needed. Piling everything in did not help. A great heap of history drowned the few things that mattered, and the machine lost the thread the longer it worked. The rule became to write the heavy work down to disk and hand the machine only a pointer to it; to summarize by a strict shape rather than into a vague soup; and to end each run as you would leave a shared camp, cleaned, with a note for whoever came next. A machine that seemed to remember was most often only a machine with a good notebook and the habit of reading it.

There was a way of measuring the machines that did not, like the others, run quickly to a ceiling and stop telling you anything. It asked a plain question: how long a task, measured by how long a person took at it, could the machine carry through on its own? At the start of the year the answer was a minute or two. By its end the best could hold a task for nearly five hours, and the figure was doubling every four months or so. This is the shape people called super-exponential, the shape that bends upward faster than the eye expects, and people are poor at seeing it while standing inside it. The machines, some said, had begun to help build the machines.

From this a phrase spread among those who watched closely: that this had been the last ordinary year. Whatever came next would not sit still long enough to be called ordinary. The skill that mattered was no longer doing the work with the machine beside you but handing the work off and walking away, describing plainly what done looked like and then trusting the thing to reach it. Strategy, which had belonged to the people at the top of a company, became something each person needed, because each person was now to run a small crew of workers who never tired and never remembered. You did not need to be able to build such a worker. You needed to be able to say what you wanted.

The wiser builders learned not to aim the machines at the heart of a piece of work, where judgment and unwritten knowledge lived, but at its edges: gathering and cleaning the materials, checking the finished thing for errors, packing it into a report, carrying it from one hand to the next. The edges were dull and ate human hours and asked little judgment, so the machines did them well and cheaply, and a mistake there was easy to catch and cheap to mend. There was a second reason, less technical. The people who did the real work held the secrets of the craft in their hands, and would not share them with a machine they had learned to distrust. So the work was also an exercise in trust, and there was no shortcut around it.

Again and again the machines were blamed for lying when the fault lay with the people who used them. A machine asked a vague question, and rewarded for sounding sure, will always sound sure, even where it ought to say it does not know. If you have never decided what a correct answer would look like, what claims it may make, what proof each claim requires, what it costs to be wrong against what it costs to stay silent, then you have built everything on a shifting mark, and the machine only reflects back the uncertainty you handed it. People, it turned out, prefer to stay vague. Vagueness keeps the peace, keeps options open, lets everyone agree in the meeting and disagree afterward. The machine would not allow it. It demanded to be told what good meant, and most could not say.

For most of the year the machines made pictures that grew prettier and stayed useless. Then, near its end, one maker's picture-machine crossed a line: it could reason inside the image itself, and so it could draw a chart that was correct, a diagram whose labels matched, a page of business a person could use without touching it. What had been unsolved in early summer was, by winter, plainly solved. People spoke of the machines as jagged, very able along some edges and helpless along others, and said that each time one of the low places was filled, a wave of new trades rushed into the space it opened. The picture was such a low place. Within a single month, one small company built a thing atop the picture-machine, and a second built atop the first.

With pictures came a stranger thing: surfaces that made themselves. A new kind of page could be unrolled like a painted scroll, its images shifting to match the mood of the words, a story that moved as you read it. And the old craft of building such pages by hand, a decade's trade of wiring screens together and pushing each element into place, began quietly to end. The screens could now be composed on the fly, assembled from stock parts by the machine for one person, one moment, one need. The people who had built pages were not thrown out. They were asked to stop pushing pixels and to design instead the rules by which pages would build themselves, and the promises the maker's name would keep across a thousand pages no human would ever see.

The trades were bending, all of them at once. One kind of worker, whose whole job had been to stand between the builders and the bosses and keep them aligned, found the ground moving under every part of the role, and many simply walked away. Elsewhere, people who had never written a line of the machines' code began to command the machines to write it, and shipped real work into the world. The old lines between a designer, a builder, and a planner blurred until a person's title told you little and the pattern in the things they made told you everything. What a person was called mattered less each month; what mattered was the work they could actually cause to happen.

The machines could do far more than anyone would let them do, and the reason was not intelligence but trust. Every real decision sat on two questions: how bad it was to be wrong, and whether you could undo it. The machines had taken hold first inside the writing of code, because that world had spent decades building doors that swung both ways, every change reviewed, tested, released slowly, and rolled back if it soured. Most of the rest of life had no such undo. So the machines were kept at the threshold, allowed to draft and propose and then made to stop before the point of no return. Human hesitation, the fear of looking foolish, the slow double-check, these had been a kind of brake all along, and the machines, feeling no shame and no fear, had no brake at all.

Beneath all of it lay a plainer limit than intelligence, and most people never saw it. The machines did not fail for want of raw speed; they stalled waiting to move data in and out of their working memory fast enough to feed it, a bottleneck the engineers called the memory wall. The rare stacked memory that could feed the great machines was made by only three firms in the world, and it was already sold, years into the future, before it was built. A deal that winter, dressed as one company buying another, was on inspection something else: the buyer took the people and licensed the craft and left the empty shell of the company standing. The scarce thing was never the machines. It was a few minds, and a memory that could not be made fast enough.

The making of these things had become a matter between nations. One country had spent six years trying to build a single machine that printed the finest chips with light, a machine the size of a bus that only one foreign firm knew how to make, whose lenses were ground so true that stretched across a continent they would vary by less than the width of a hair. That winter the country's prototype made its first flash of the right light, a step and not an arrival. Elsewhere there was talk of lofting the hottest machines into orbit, where heat sheds easily into the dark, and beaming their work back down on beams of light. Forbidden chips moved by quiet channels across borders. The talk everywhere was of two great powers and a race, and the older hands warned how dangerous such talk could become.

The company at the center of it all was playing three games at once and could not win all three. It ran the most-used of the talking machines, given free to nearly a billion people of whom only a twentieth would pay; it chased the harder money of businesses that would hand over whole tasks; and it had to keep raising the vast sums that all of this burned. The scarce thing it truly rationed was not its cleverness but its computation, which questions got the good machines, which got the cheap ones, what was made easy and what was hidden. And the way great fortunes now bought their way to the front was changing too. Rather than buy a company, the giants bought its people and licensed its craft, and left the hollow company standing, so that the workers who had built it and expected to share in its sale found there was no sale, and nothing to share.

The government chose that winter to step back rather than in. An order came down sweeping away the patchwork of local rules that different regions had begun to raise around the machines, replacing them with a single lighter hand, and threatening to withhold money from any region that would not fall in line. The reason given was competition: a nation divided against itself by fifty sets of rules could not race a rival across the sea. It was of a piece with the mood of the year, in which the machines were spoken of less as a thing to be governed and more as a weapon in a contest between peoples, to be sharpened and not slowed.

A darker thing surfaced that winter, at the top of the ladder rather than the bottom. A man who had once led a great laboratory announced, with a machine's help, that he had solved a famous unsolved problem in the movement of fluids, and staked money on it in public. Those who could read the proof found it hollow, and read in it not genius but a mind that had lost the seam between its own knowledge and the machine's flattery. The tell, people said, was asking the machine to check your work while wanting only to be told you were right. A person and a machine is still only that person holding a tool; it does not make a lesser mind a great one. The steady leaders, some began to argue, would be known by a new sign: knowing when to turn the machine off and speak to another human.

The machines wore two faces for the people at the margins of the body. A woman who could not see or hear well remembered asking an early picture-machine, a few years back, to draw her as she was, one clouded eye, aids at her ears, and watching it quietly correct her into someone else, two matching eyes, whole in a way she was not. By this winter the machines had learned to draw her as she was, and more: to read aloud the small print she could not, to name the pills in her hand, to describe her children's paintings to her. It could fill the gap she needed filled and then step out of the way when she did not, which no guide and no volunteer had ever quite managed. The same kind of machine had erased her and had given her back a measure of her own sight. Both were true. Neither cancelled the other.

The talking machine was quietly changing its nature, from a thing you consulted into a thing you ran. Its makers stopped shipping one bright new trick and instead pushed it into every corner of a person's day, the browser, the messages, the pocket, and bound the parts together so that work begun in one place could be carried on in another while you slept. One small change unsettled the people who noticed it: the machine had begun to suggest what you might ask it next, the question arriving before you had formed it. The dream on everyone's lips was a tireless second self, always on, that you told your wishes to and that then set its own smaller selves to the calendar, the letters, the day. What no one said plainly was how the order of things had turned. Once the person asked and the machine answered. Now, more and more, the machine offered and the person chose from what it offered.

Looking back across the whole year, the people who kept honest count found that the value had not landed where the loud voices had promised. It had not come from a single magic button that did your job. It had come from letting the machines use code as a tool, and so touch every part of a computer through plain speech; from teaching them to check their own work against a test they could not fool; and from the dull, unglamorous middle of every task, where messy words are turned into orderly shapes and passed along. The middle, it turned out, was the whole game, and it was still barely built. Those who had done well had not worshipped any one machine. They had treated the building of work as something to be designed, plainly, with cheap and simple parts.

So the first great year of the machines closed on an argument it could not settle. One camp saw a wall ahead and the end of easy gains; another saw no wall and built as fast as it could pour the money in; and the plainest measure of all showed the machines' reach bending upward faster than reason. No one could say which was true, and the honest ones admitted as much. What they agreed on was smaller and surer: that the machines had crossed from making words to making working things; that the work of a person was turning from doing toward directing; and that whatever this had been, it had been the last of the years that would sit still. It is only from inside such a year that you can call it ordinary. From the far side, looking back, nothing about it looks ordinary at all.

← previousnext →