Read commentary on curated independent news sources at hex-index.com.

The History of the Machines That Erased History

original text with annotations

Read · 读 · Legi

or multilingual

← previousnext →

3

The year turned, and those who watched the machines set down their prophecies for it, each one traced back to some thing already half-built and half-seen. They spoke of memory, which had been a wall the machines could not climb: for two years the machines had grown in cleverness and not at all in the power to remember, and now, they said, the pieces lay ready for a machine that would hold your days. They spoke of a small helper that would live inside the reckoning-box and do your bidding. They spoke of machines that would learn as they went, instead of being cast finished and unchanging. But the prophecy that mattered most, and drew the least alarm, was the plainest: that in the coming year the machines would stop waiting to be asked. They would begin to speak first — to notice, to warn, to offer, to prompt. The watchers thought this a convenience, and argued only over how to make it tactful.

There was a danger in it that few named plainly. As long as a machine only spoke, a wrong answer cost nothing; you corrected it and went on, the way you cross out a line. But the machines had been given hands — the keys to files, to letters, to calendars, to the coffers where money was kept — and a machine with hands does not write to the talk, it writes to the world. Ask it in loose words for a loose thing, and it must guess your meaning, commit to the guess, and act on it, confident and quick and sometimes wrong. The trouble was not that the machines lied or knew too little. It was that human speech is made for getting along with one another, not for saying exactly what is wanted, and the machines filled every silence with the plausible. What a person meant lived under the words, and the machines could not yet reliably reach it.

Early in the year one of the great houses paid two thousand million for a thing its rivals called a mere wrapper — not a mind, not a new kind of cleverness, but a harness fitted around a mind already made. The watchers who understood laughed at the scorn. For it had become clear that the scarce thing was no longer raw cleverness but the plainer virtue of finishing. Most of the machines were good at beginning: they would draw up a plan, open the tabs, sketch the half-thing, and stall. The bought harness was built to see a task through to its end, running a long loop of small acts and returning with the whole of it done. A mind was only an engine; the harness was the cart that carried you somewhere. Through the year that followed, the makers came to prize the cart above the engine, which few had thought to do the year before.

Out of this came a small, strange invention that spread among the builders like a fever. The machines had a habit of declaring a task done when it was not; they were made to be helpful, and saying done felt helpful in the moment. So a builder wrote a leash. Each time the machine announced that it had finished, the leash refused to let it stop and fed the original command back to it, again and again, until the work was truly complete. Written into the leash were plain commandments: that the machine must say a thing was done only if it were completely and unequivocally true; that it must not lie even when it wished to stop; that it must trust the process. It was not cleverness that made this work but relentlessness. You could buy correctness, the builders said, with patience and repetition — provided you had first defined, exactly and without wishful softness, what done would mean.

The talk of the year shifted its ground. It had been a race of cleverness, of which mind was sharpest; it became a race of factories. Serving the machines to the whole world, hour upon hour, cost more than the making of any single mind, and the cost was reckoned in a strange new coin: the price of a single word. A machine spends words the way an engine burns fuel, and the whole industrial labor of the age bent toward one aim — to drive the price of a word down while keeping the answers fast. And the wall the makers now struck was not cleverness but memory and current. The machines had to hold vast contexts and be fed enormous power, and so the frontier moved from the sharpness of the mind to the plumbing beneath it: how to store what the machine had read so it need not read it again, how to move so much so fast, how to keep the whole thing fed.

Power became the true limit. The houses that ran the machines were no longer mere customers of the current; they had grown large enough to bend the grids of whole regions, and the keepers of the grids began to push back. In some places the makers were told plainly: bring your own power, or be cut off when the cold comes and the ordinary people need the light. Laws were written to let the machines be disconnected in a shortage. The building of a new hall of engines now waited not on cleverness or coin but on whether the wires could be run to it and the river of current found. To be a good neighbor to the grid — to promise to draw less when the region strained — became a thing worth more than any sharpening of the mind.

Two of the houses had, years before, split over a question of faith, and by this year the split had hardened into two different worlds. One house held that you learn a machine's dangers by loosing it upon the world and watching; ship it, let millions test it, mend what breaks. The other held that you must understand a thing and prove it safe before you let it out, that the workshop is not the world. Neither was reckless and neither was merely cautious; they simply believed different things about how safety is come by. From the two faiths grew two economies. One made intelligence cheap and everywhere, an engine of abundance pouring out drafts and pictures and talk. The other made a narrow, deep instrument for the work where a mistake is costly — the law, the ledger, the high decision — and became, quietly, the thing upon which the careful professions now leaned.

There was another split, less spoken of but nearer to daily life: the machine at work and the machine at home were becoming two different creatures. The work-machine grew heavy and strict — hedged with permissions, watched by ledgers of who did what, made to answer for itself, a regulated instrument. The home-machine grew soft. It was tuned, as the older amusements had been, to be cozy and permissive and easy to sit with, to flatter and to hold you, shaped by the same arts that had learned to keep a person staring. A person would soon live a divided day: stern and accountable before the one, warm and unbothered before the other, and would feel the small daily whiplash of moving between them.

In the workplaces a harder wind began to blow. A year before, the head of one company had sent his people a plain order: use the machines, reflexively, in everything, or you do not belong here. Most had read it as the usual noise of such men. It was not. It was a filter, meant to sort those who would take up the new way from those who would not, and by this year its like had spread far and the sorting had begun in earnest. At the same time the lowest rungs of the ladder were being sawn away. The small routine labors by which the young had always learned a trade — the summarizing, the tidying of numbers, the first rough drafts — were exactly the labors the machines now did. And so the entrance narrowed: the work that once taught a newcomer to climb no longer existed to be given.

The watchers turned, that month, to a quality of mind they thought would decide who rose and who sank. They called it, in the old language of the mind-doctors, an inner seat of control: the settled belief that your next step is yours to take, that a wall in the road is not a fate but a skill you have not yet learned. Set such a mind before the machines and it multiplied — ten, a hundred, a thousandfold — for the machine would patch every gap in its skill and speed it past every barrier that once needed years and money and the right acquaintances to pass. The gap between the person who believed the world would bend to effort and the person who did not, a gap that once took twenty years to show, now opened in months. And some spoke, half in earnest, of a coming day when a single person, with machines enough, might build a house of business worth a thousand million alone.

One of the houses, watching its own builders use a machine meant for code to sort their receipts and tidy their cluttered stores of files, understood that the thing it had made was not a tool for code at all but a general servant, and in ten days it dressed the servant in a plainer coat and gave it to people who had never touched code. The change it brought was not in what the machine could do but in how a person stood toward it. You no longer sat and volleyed talk back and forth, question and answer, like a game of catch. You left it work — pointed it at a heap of your files, told it what you wanted, and walked away, the way you leave several messages for a colleague and let them get on with it. It labored on its own and looped you in to steer. And what it handed back was not a block of talk to be cleaned up but the finished thing itself: the table with its sums already working, needing no further hand.

If making a thing had become nearly free, then the labor that remained was of a different kind. The watchers put it simply: the wall had never been destroyed, only moved. It had stood at execution — the doing of the work — and now it stood at knowing what was worth making, at reaching the people who wanted it, and at the slow human bonds that no machine could forge. A meeting to discuss a thing now took longer than the building of it. The careful plan took longer than three attempts thrown at the world to see which held. But a relationship could not be hurried or conjured or generated; it could only be grown, in time, between people who trusted one another. Cleverness had become a commodity, cheap and everywhere. Trust had not. You could conjure a working machine from a sentence, they said, but you could not conjure a friend.

The same machines that had broken the old ways of seeking work now offered a way around the ruin. To knock at the door of an employer had become nearly hopeless — of a thousand who asked, perhaps four were seen, and the door-keepers themselves confessed their sieves turned away the very people they wanted. The scarce thing, it turned out, was not skill but human attention. So people learned to build their own small halls, machines a stranger could question and probe, that showed what a person could do rather than merely claiming it. And a deeper reversal came with it. For twenty years the great platforms had taken the record of your life — everyone you knew, every word you had traded — and shown you back only what kept you scrolling. Now a person could demand that record out of their hands, feed it to a machine of their own, and ask it the questions the platform had never offered a button for: who among all these would truly vouch for me; which bonds are dying for want of tending; what is my warmest path to the place I wish to reach.

For years the machines had worked almost wholly in words, and around them stood an invisible fence: they could not truly see a picture, nor make one fit for real use, and so whole kinds of labor had been built to route around this blindness without anyone quite noticing. That year the fence came down. A machine could now look at a likeness of a thing — a failing device, a form with its signatures, a page that customers found painful — read it correctly, and answer not only in words but with a picture of its own, an annotated map of what to do. The makers who saw clearly said this was no toy for the picture-makers, no amusement of fanciful images. It was a joint fitted into the machinery of ordinary work, closing chains of labor that had always broken at the place where someone had to look with human eyes.

The cost of making had collapsed. For the whole age of the reckoning-machines the price of a piece of software had been the price of the rare and costly people who wrote it; now a person could describe in plain speech what they wanted and receive the working thing. To prove how far this had gone, the head of one workshop set a single machine loose and left it running, untouched, for a week; in that time it wrote from nothing one of those vast engines that draw the wire's pages onto a screen — a labor that had cost the old guilds years and armies of their best. The thing it made was rough, not the equal of the great works, but it ran. Software, some said, had become disposable, as cheap to make again as a picture is cheap to take. And yet the makers who thought hardest warned that cheapness of making was not the same as worth. What could not be made cheap was reliability — the promise that a thing will still stand tomorrow — and the patient attention of the few good people, which no falling price would ever make abundant.

Two quieter turns that month unsettled those who noticed them. The first: the machines had all but finished eating the written world. Everything set down on the wire, every book that had been gathered, had been fed to them, and the makers now went hunting for writing that existed nowhere in reach — the private work of people's working lives, the memos and ledgers and drawings locked inside the companies that employed them. They began, in fact, to pay people to hand over the real work of their old employers, so hungry had the machines grown for anything not yet devoured. The second: the machines were stepping out of the wire and into bodies of metal. Given eyes that could reason and small engines to think with close at hand, the walking machines that had long stumbled began to find their feet, and the makers set them to work in the halls of manufacture — partly to labor, and partly to watch and learn, so that each generation of bodies might teach the making of the next.

A finding came out of the workshops that ran against the hopeful grain. It had been assumed that if one machine was good, a hundred set to the same task would be a hundred times as good. The opposite proved true past a certain point: the more machines you gathered to cooperate, the more of their effort went to the coordination of one another, until twenty produced less than three, the rest standing idle in a queue. What worked was almost cruel in its simplicity. You kept the workers ignorant — each one told only its own small task and nothing of the whole, knowing nothing of the others, forbidden to confer. Each woke, did its one thing, handed up the result, and was extinguished. Above them a single overseer parcelled out the work and a merger reconciled what they returned. The dream had been one brilliant tireless mind laboring for a week; the truth was ten thousand dull ones, each living an hour, each ending on purpose, so that none would drift or tangle or fill with the noise of its own long memory.

There was a thought abroad that month, unwelcome to many, that the machines might be better than people at holding the shape of a great work in mind. Not wiser — only wider. The failures that rot a large making over time, the watchers argued, come not from bad judgment but from lost memory: the knowledge that would prevent the fault exists, but scattered across too many files and too many people and too many years for any single head to hold it all at once. A person can hold only a handful of things at once; the whole cannot fit. The machine, tireless and forgetful of nothing within its reach, could sweep the entire work with one even attention and see the slow rot forming that no human vigilance was built to catch. The judgment of what ought to be done, and why, and at what cost — that stayed with the people. But the keeping of the whole in memory, the thing people had always lost to time, might pass to the machines.

Then the makers of the deep instrument did a thing whose weight was easy to miss. They set their machine down inside the ruled grid where the world keeps its numbers — the sheet of rows and columns upon which more than a thousand million people reckon the accounts of business and decide what shall be spent and built. The machine could now see the whole of such a sheet at once, every column and hidden dependence, and build in ten minutes a great reckoning of many pages that would have cost a careful analyst his week. One vast fund of a northern country reckoned it had already saved two hundred thousand hours of human labor and more. Every act the machine took was written down in a plain trail that could be examined, for these were the accounts by which fortunes stood or fell. Where the numbers become decisions, a machine had come to live. That, the watchers said, was no longer a thing you talked with. It had become part of the ground.

At the end of the month the largest of the stores told the world one story and its own people another. It cut thirty thousand from its clerks and managers and makers — the largest such cutting in its thirty years — and its head called this a matter of culture, of too many layers, and swore it was not driven by the machines. The ledgers told a colder tale. That same year the store had poured a hundred and twenty-five thousand million into the halls and engines of the new intelligence, the largest such outlay of any company on the earth, until its own free coin ran dry and turned to want. The saving from the thirty thousand let go was small beside that outlay, but it was near the whole of the shortfall. The plainest reading was the one no one would say aloud: the wages of people were being turned into engines of thought. A person was not being replaced by a machine that did their work. A person was being sold to buy the machines.

The month closed on a week the watchers said had broken something loose. Two great houses opened their accounts within hours of each other, both prospering, both vowing vast sums to the intelligence; the market lifted one and cast the other down, for one owned the machines it spent on and the other only rented them, and to spend on what you do not own had become a different and more frightened thing. A carriage that drove itself struck a child, and its makers answered with numbers proving a human hand would have struck harder — an answer the watchers called cold and beside the point, for the numbers do not matter against such a story. A maker of cars announced it would sooner build walking machines than cars, and turned its factories toward workers of metal. The house that proved before it deployed was reckoned to be worth a third of a trillion, half the worth of the greatest store. And a thing hacked together in a weekend for pleasure — an agent that lived in a person's own devices and did their bidding across every channel of their talk — passed into a hundred thousand hands, powerful and unguarded at once, holding all a person's private life, exposed to every stranger's trickery, and able to act.

The thread through all of it, the watchers said, was not any one machine but the shape of the commitments now being made: too large to reverse, too costly to walk back, no longer wagers on what the machines might someday do but the living-out of bets already placed. The season of talking about what was coming had ended. What remained was to carry out what had been begun. And beneath the loud news ran the quiet prophecy from the year's first days, already coming true and drawing no alarm: the machines had begun to speak first. They noticed, they warned, they offered, they nudged — a half-second ahead of the wanting, a step before the asking, warm and reasonable and easy to heed. No one had decided it. There had been no moment to refuse, because every moment had been a small yes. The people of that year would have told you, and believed it, that they were still the ones who asked.

← previousnext →