The stock market suffered its sharpest fall in history earlier this month. Could it have been predicted? Thomas A. Bass reveals how down-at-heel computer geeks, using a program devised to beat the odds at roulette, developed a 'black box' designed to clean up in the financial markets. They stood to make hundreds of millions of pounds - if only they could hold their nerve

ON January 4, the second day of trading in the new millennium, floor brokers on the New York Stock Exchange are desperately dumping more than a billion shares into a market that will plunge 360 points and lose more than three per cent of its value by the end of the day. Dealers over at the Nasdaq exchange are jettisoning close to two billion shares and watching their own market lose more than five per cent of its value. It is a slow news day, with nothing to account for collapsing markets, save for last year's irrational exuberance slipping into reverse, but all around the world, from Tokyo to London, money is being sucked out of people's pockets at breathtaking speed.

Roll the clock back to April 8, 1996 and shift the scene to Chicago where all hell has broken loose in the world's bond and currency markets. A speculative tsunami from somewhere in Asia has washed over the gold market in Zurich before crashing into the pits at the Mercantile Exchange. The exchange computers are overwhelmed. Traders in bright red-and-yellow jackets look like life-jacketed sailors clamouring to be saved from a churning market. Everyone is shouting and gesticulating. The market is collapsing faster than selling orders can be filled.

A carrot-haired man wearing blue jeans and a red work shirt sits close to the equities desk fielding orders at Swiss Bank Corporation (SBC), speaking softly, in a flat, Midwestern accent, into a telephone headset that connects him to the world's principal exchanges.

Near him is a fanciful-looking computer, lit up in fuchsia and blue. The device occasionally makes a noise: Beep! Beep! At every Beep! the trader spins in his seat and hits the speed-dialler. From the positions of the numbers on the screen, he can tell which futures market he is supposed to call. And he can tell by the colour - blue for buy, red for sell - what the machine wants him to do. In seconds, the trader is talking to an SBC clerk who flashes the order into a pit.

A computer like the one at the trader's elbow is called a black box - its program is a mystery to the uninitiated. The box is emotionless, opaque, obscure. It gives no winks or nods. "The magic gadget is a little threatening," the trader confesses.

Yet people on the floor are impressed by its salient feature; it appears to have an uncanny knack for being on the right side of trades. Today, this knack seems particularly uncanny.

What if there were a system for finding patterns in the swirl of numbers, a way of predicting movements in trillion-dollar markets that most people assume are random? Wouldn't the predictors who discovered this be lords of the realm?

The black box is connected to a telephone line that runs from LaSalle Street, west over the Rocky Mountains down to Santa Fe, where it ends in a big room with hanging plants, a couch, a table, and a couple of computer terminals. The screens are glowing with the green sawtooth lines of financial markets ticking up, down, up, down.

August 19, 1991, New Mexico. Doyne Farmer pulls his Datsun 2000 roadster into a parking lot behind an old adobe bungalow in downtown Santa Fe. He is accompanied by Clara, a Bernese mountain dog.

One of his colleagues greets him at the door, saying, "Have you heard? The Dow is down 70 points. Gorbachev is under house arrest . . . "

Farmer shrugs. Over six feet tall, he is rangy, with slightly irregular features. "Unless you have a CIA mole on the ground, no one can predict a Russian coup," he says. He's confident that he and his colleagues can create a black box able to forecast certain directional swings in the world financial markets, but he knows that there are limits to what can be forecast. He and his dog enter the one-storey house, which is furnished with plastic-webbed lawn chairs and folding tables holding five workstations.

Farmer's old friend Norman Packard arrives a few minutes later. Tall and amiable-looking under a helmet of blond hair, he has the same western-style informality and directness as Farmer. Now Packard sits down in front of Seldon, his computer, named after an Isaac Asimov character who "could foretell the future mathematically".

"We agree with Seldon," Packard jokes. " 'Order must underlie everything, however disorderly it may appear.' "

Farmer and Packard are partners in a company that as yet has no name, almost no furniture and no money. It does, however, have some of the smartest people in the new science of chaos or complex systems. The immediate task is to develop models for beating the financial markets and then shop for a partner with millions of dollars to invest.

Over the years, Farmer and Packard have developed a remarkable intellectual intimacy. They finish each other's sentences, charging forward into new terrain at the mere hint of a half-formed idea. In fact, the two men grew up together, in southwestern New Mexico. Farmer was born in 1952, Packard two years later, and both went to college on the West Coast. In the mid-Seventies, the two found themselves together at graduate school, and there they decided to use their expertise to develop a system for beating the game of roulette.

During a gambling jaunt to Las Vegas, Packard saw immediately what was required for predicting roulette. Clock the ball spinning on its track. Clock the rotor spinning below. Compute their relative positions. Figure out their rates of deceleration and the arc through which the ball will travel before coming to rest in a numbered pocket. And - the tricky part - do all this within 20 seconds, so there would be time to place your bet.

From 1976, Packard and Farmer and a dozen others - including myself - spent their free time perfecting the equations and building the equipment to implement them. They created toe-operated, radio-linked computers that fitted into their shoes, and began making frequent trips to Las Vegas. By autumn of 1981, they were on their way to fame, if not fortune, as the world's first players to beat roulette. They might have achieved fortune, too, if it hadn't been for the difficulties entailed in shrinking computers into a shoe.

A decade later, I linked up with Packard and Farmer again, this time purely as an observer, to chronicle their second attempt at founding a company.

Farmer and Packard had, in the meantime, stumbled on more intriguing problems. Expert at predicting behaviour in one chaotic realm, they turned to exploring chaos itself. Specifically, they tried to understand how order emerges at "the edge of chaos" - a phrase coined by Packard and now a lodestar concept in the theory of "complex adaptive systems".

Then, in 1991, a conference on economics was held at the Santa Fe Institute, a think-tank devoted to studying the science of complexity. The conference thronged with Wall Street traders and investment bankers, and so many of them seemed convinced that Packard's and Farmer's ideas would make money that the two men decided to quit their prestigious academic positions and go into business themselves.

They hired two graduate students, a post-doctoral student, and two research assistants, setting up quarters in the adobe house, which they called the Science Hut.

The Science Hut, which is beginning to warm up on this August morning, looks more like a graduate-student lounge than a business. The standard dress is rubber rafting sandals, Patagonia shorts, T-shirts, and long hair pulled into ponytails.

Packard shares his office, an alcove between the living room and a sunporch, with a researcher named Tom Meyer. Packard begins loading a model for forecasting the British pound. "When do the London markets open?" he asks.

"I don't know," Meyer replies curtly. "That's Tony's department." Tony Begg, a forty-ish computer scientist, is one of the few people there who has ever invested in the markets.

"Six hours before Chicago," Begg informs Packard.

"No wonder my model was so good," Packard says. "It was contaminated with future data. I was using opening prices in Chicago to predict the London opening."

"This will help you," Begg tells him, and hands him a dog-eared copy of The Wall Street Journal Guide to Understanding Money & Markets. "Here's a nice diagram explaining when the markets open and close."

Tom Meyer pulls his lawn chair up to his computer. It's not a good match: the chair is wobbly and Meyer is six feet seven.

He starts punching in numbers and pushing the run button on programs meant to resolve millions of data points - a mess of yen-dollar rates, central-bank statistics, and ticker-tape numbers - into previously unseen patterns. He is practising the fine art of time-series analysis. "Here's a blip that keeps reappearing at regular intervals. Here it comes again, right... now." "We're not rich yet," Meyer says, "but I see some nice numbers here." He stretches his arms over his head and yawns. Then his chair collapses under him. "Worthless piece of sh--!" he yells from the floor.

On September 12, 1991, Prediction Company is officially incorporated. Three days later, Doyne Farmer is ready to go on the road. He has no experience in raising investment capital, but his first move is to buy an Italian wool suit that everyone agrees "looks like 50 million bucks". He and Packard routinely share clothing, and others in the company look as if they, too, could step into the "company suit".

At the end of September 1991, Farmer flies to New York for five days: shuttling to meetings at Kidder Peabody, Citicorp, Salomon Brothers, Goldman Sachs. By the end of his tour, a surprising number of people - representing a thick slice of American capital - are willing to open their chequebooks and make a bet on Prediction Company. Unfortunately, most of them also want an ownership stake in the company, and that's something Packard and Farmer are loath to give up.

Suitors begin to trek to Santa Fe. One thing they all mention is the sheer size of the financial markets. Speculators swap more than a trillion dollars a day in foreign exchange. One suitor, a blackjack card counter who runs a commodities-trading company in California, tells Farmer that if the company plays it right it could earn five hundred million dollars in the next five years. "At this point, we were getting jazzed on the large numbers people were tossing around," Farmer says. "Even though we didn't believe it, there's an undeniable adrenaline jab that comes from someone telling you you're about to make five hundred million dollars."

The predictors fantasize about being rich. In their customary, hyperrational way, they begin computing how much money a person needs to be really rich. They also explore the moral issues involved. "Since most of the rich people in the world are assholes, what is the secret of being rich without being an asshole?" Joe Breeden, a researcher, wonders one day at the Science Hut. All their worries are premature, however, given that Prediction Company has yet to find a partner.

Packard flies to Chicago on March 12, 1992, to do his "intellectual striptease" for O'Connor & Associates. O'Connor has 600 employees and shuffles billions of dollars through the futures, options, and currency pits. O'Connor, which is being bought by the SBC, is the most mathematically astute, computer-intensive broker-dealer in America.

Packard is ushered into a conference room with windows looking onto O'Connor's electronic ticker tape. Those present include David Weinberger, the company's former managing partner. Weinberger, who once taught computer science at Yale, is a trim, athletic man in his mid-forties, with the nervy assurance of someone who has mastered every game he has ever played. Many of the others present are MIT graduates, including Clay Struve, a moustachioed, ruddy-faced man in his early thirties who is drinking two Diet Cokes simultaneously. He sits mute in meetings, an all-knowing sphinx. He has a gift for doing large calculations in his head, and is the O'Connor partner in charge of "risk". The company plays the derivatives markets with leveraged portfolios whose face value is many billions of dollars. Get the numbers wrong, and you go out of business. Struve keeps O'Connor's numbers between his ears.

Packard, in his presentation to the partners, describes how market data seem to hold predictive patterns. Struve is generally silent, with nothing moving across his rosy face except the hint of a smile. At the end of Packard's presentation though, he delivers some remarks that his colleagues interpret as wild enthusiasm. "The anomalies are to hard to find and take advantage of, but with the right microscope and the right hedging you might be able to do it," he says. "You people are on the right track. It looks good."

"This is a religious issue," Weinberger adds. "Either you believe there is structure in the markets or you don't."

In April, Prediction Company and O'Connor agree a five-year contract. O'Connor will fund the development of Prediction Company's trading technology. A much larger chunk of capital will be used to deploy the system. Prediction Company will get a slice of the profits. (A figure between 10 and 25 per cent is typical in the industry.)

Prediction Company starts designing the black box that will be placed in the middle of SBC's trading floor. July 1, 1993, is the predictors' self-imposed deadline for going live, which means a lot of tweaking in not a lot of time.

The company fails to meet its self-imposed deadline, though. It is not until the fall that SBC begins tentative live trading of Prediction Company's signals. After that comes another year of low-level testing, debugging, fire drills, and jumping over performance hurdles before trading even remotely approaches the value of Doyne Farmer's $50 million suit.

On August 9, 1994, Prediction Company flips the switch on a new set of programs. Orders flow out to buy and sell contracts whose face value is measured by the millions of dollars. A cheer goes up among the predictors, crowded around the trading console. Finally playing with real money in real time, they watch the numbers spiral upward and imagine the same thing happening to their bank accounts.

A few days later, Farmer walks through the front door with his dog walk and realises something is wrong. The researchers around the trading console are wearing long faces. Farmer strides toward the monitor. He can tell at a glance what's going on. "We're getting creamed!" he shouts from halfway across the room.

"The Fed is adjusting short-term interest rates, and the bond markets are going crazy," says William Finnoff, a researcher. "It was a surprise announcement."

The monitor offers a brightly coloured display of what is happening, tick by tick, in Chicago, New York, and other main markets. Today the markets are plummeting so fast that the green trend lines are beginning to break apart as they fall relentlessly toward the bottom of the screen.

By the end of the day, the crowd huddled around the trading console has watched the portfolio lose more than $100,000 dollars. By the end of the week, it will have lost a good bit more. Nothing is going according to plan. This is real money being sucked down a hole.

But, after the initial fright, the predictors are pleased to see their system right itself. Autumn fades into winter, and Prediction Company is cruising along, with the portfolio making a few hundred thousand dollars a month, when one day David Weinberger, has an angry confrontation with Packard. "What the hell is going on?" he demands. "I just found out you guys cut the position in half. We won't stand for it!"

Packard tries to calm him down, but Weinberger is on the rampage. The problem is Prediction Company's use of "interventions". These occur when Prediction Company asks SBC's traders to override the models by cutting orders in half or abstaining from the markets altogether. This substitution of human judgement for computational analysis is supposed to be reserved for rare occasions, such as important "number days", when federal statistics such as the unemployment rate are announced. News events can drive the markets in unforeseen directions, and it sometimes seems prudent to stand clear of these oscillations. But Weinberger is convinced that the predictors are trigger-happy - far too ready to intervene.

"This is no time to lose your nerve!" He yells at Packard. "This is an experiment. You're building an automated system. You don't put your finger in the middle of it and diddle the results. The one thing we demand in our business is absolute discipline!" Weinberger is pacing from wall to wall, jangling the change in his pocket.

"Discipline," Packard agrees. He knows that when Weinberger starts shouting, the best response is to roll into a ball and sit there like a hedgehog. "We could all use some discipline."

The upshot is that the people at Prediction Company are supposed to work more closely with SBC's traders and keep interventions to a minimum. This is another important step on their way to becoming seasoned professionals. Later, when they study the numbers, they realise that the portfolio would have made a lot more money if it had been allowed to run hands free.

"You have to be willing to put all your money in the pot and see what happens," Farmer says. "Our roulette experience is good training. It's scarier than hell, hanging out in the breeze, but when the odds are in your favour, you have to swallow your fear and hold on."

In the three days before Thanksgiving, 1994, the portfolio makes a few hundred thousand dollars in profit. By the end of the year, it is solidly in the black. "It's almost scary how our predictions are coming in right on the money," Finnoff muses, watching the numbers tick down to the last trade of the. year.

In June 1995, Prediction Company has another rude lesson in the distance between theory and practice, when its crude-oil model, which looked good on paper, gets pummelled in the futures market.

Prediction Company dispatches someone to New York to find out what's going wrong. It doesn't take long. On the trading floor of the World Trade Center, there is the usual hurly-burly of traders bellowing at one another and flipping hand signals off their noses until 3:05pm, when the order based on Prediction Company's signal arrives. Amazingly, the market quiets. Everyone turns to look at the broker fielding the order. He is trying to buy 300 futures contracts.

As soon as the trader makes a move, everyone piles onto his order, and the market price starts jumping $10, $20, $30, before the order is finally filled. Then, as soon as his business is out of the way, the locals let the price fall, having scalped the profits that will allow them to go home happy men.

In August of 1995, everyone at Prediction Company fails the quarterly objectives. The members of the software-modelling group, still de-bugging 100,000 lines of code, miss their "phase two" release date.

The members of the research group are faring no better. The profit and loss account is up a few million dollars, but each one of those dollars feels as if it has been laboriously hand-printed in Ye Olde Prediction Shoppe.

Things get worse. August is a good time for the Federal Reserve Board to intervene in the currency markets. Traders are on holiday and the markets are thin, so the government's money stretches farther. The Fed tries to prop up the United States dollar by selling Japanese yen and German marks. Central-bank interventions are like the hand of God reaching into the Newtonian universe to skew the mechanism. This month, when the Fed reaches into the market and changes the settings, Prediction Company's models are on the wrong side - a stack of money is lost.

More than half a year later, on April 8, 1996, the predictors find themselves bracing for another unforeseen disruption. Panic strikes the Chicago exchanges as a wave of sell-offs spills from foreign exchange into commodities and the bond markets. Traders in Chicago are drowning, but all morning Prediction Company's models have been recommending short selling - dumping shares in markets predicted to fall. Farmer and Packard head out to lunch. When they return, they glance at the computer screen where the portfolio's profits are tallied. "Not bad," Farmer says, smiling. "That was a million-dollar lunch."

Later in the afternoon, the predictors grab some champagne and head upstairs to the rooftop gazebo to celebrate. Below them lie the red adobe houses of Santa Fe

"Who knows? Maybe we've finally cracked the problem," Farmer says to Packard, who is stretched out beside him on a lounge chair. "In any case, it beats losing."

He pours another round of champagne, Packard raises his glass. "To the next million," he says.

October 27, 1997. Almost 68 years to the day since the crash that marked the onset of the Great Depression, the Dow-Jones Industrial Average plunges 554 points, with the market losing seven per cent of its value. This is the worst day in a decade, and the 12th-worst day in the market's history.

Prediction Company's models are supposed to have winning days whether the markets go up or down; the strategy is designed to be "market neutral". Even so, it is with trepidation that the company's employees gather around the trading console to watch the markets crash.

Packard paces the room. Farmer and his dog, Clara, stare at the screens. Everyone watches the green lines break up and stagger downward as the markets plummet. The predictors hold their breath, waiting for the first signs of how their orders are being filled.

Suddenly, a cheer goes up as the prices tick upward and the daily profit and loss account clocks a tidy multimillion-dollar profit. This is the portfolio's best day yet.

"Way to go!" Farmer shouts, giving Clara a hearty rub behind the ear.

"Not bad," Packard acknowledges with a broad grin on his face.

Towards the end of the following year, financial markets around the world begin to tumble, along with the speculators who play them. George Soros loses two billion dollars in a bad bet on Russia. Prediction Company's Swiss partner - known, since a merger earlier that year, as UBS - takes losses, too. It was a principal investor in Long-Term Capital Management - a Greenwich, Connecticut, hedge fund that would have gone belly up without a three-billion-six-hundred-million-dollar rescue package brokered by Alan Greenspan, the chairman of the Federal Reserve. It is an anxious time, and yet Prediction Company keeps quietly racking up its winning bets to finish its most successful year to date.

The latest reports from Santa Fe are encouraging. One sign that you've found structure in the markets is that your balance sheet turns an inky shade of black. Packard has bought his first house, an old adobe with a walled garden just down the hill from Farmer's. Farmer has refurbished his Datsun 2000, which is fitted with a cherrywood dashboard and a leakless top, and he has built a new office, in the garden behind his house. Most of the company's quarterly objectives are being met. Financial reports are being filed on time. The model builders are picking up speed, and, though the figures are confidential, they say the profit and loss account is looking good. Over the past year, the company has outrun the indexes, Packard reports, and made returns on capital that place it near the top of its field.

"We're reaching escape velocity," Farmer declares. He and Packard are looking forward to spending time in their gardens thinking about the evolution of complex systems, rather than living inside one.

"We have a long list of things to do if we ever have enough money to do them," Packard says. He is talking about research projects - maybe even building an institute to work on complexity and other hard problems.

"Yes, it's a long list," says Farmer, glancing over at the monitors, with their sawtooth lines ticking up and down.