HOFOLDING, Germany — The other drivers wouldn’t have noticed anything unusual as the two sleek limousines with German license plates joined the traffic on France’s Autoroute 1.
But what they were witnessing — on that sunny, fall day in 1994 — was something many of them would have dismissed as just plain crazy.
It had taken a few phone calls from the German car lobby to get the French authorities to give the go-ahead. But here they were: two gray Mercedes 500 SELs, accelerating up to 130 kilometers per hour, changing lanes and reacting to other cars — autonomously, with an onboard computer system controlling the steering wheel, the gas pedal and the brakes.
Decades before Google, Tesla and Uber got into the self-driving car business, a team of German engineers led by a scientist named Ernst Dickmanns had developed a car that could navigate French commuter traffic on its own.
The story of Dickmann’s invention, and how it came to be all but forgotten, is a neat illustration how technology sometimes progresses: not in small steady steps, but in booms and busts, in unlikely advances and inevitable retreats —“one step forward and three steps back,” as one AI researcher put it.
It’s also a warning of sorts, about the expectations we place on artificial intelligence and the limits of some of the data-driven approaches being used today.
“I’ve stopped giving general advice to other researchers,” said Dickmanns, now 82 years old. “Only this much: One should never completely lose sight of approaches that were once very successful.”
From the skies to the street
Before becoming the man “who actually invented self-driving cars”, as Berkeley computer scientist Jitendra Malik put it, Dickmanns spent the first decade of his professional life analyzing the trajectories space ships take when they reenter the Earth’s atmosphere.
Trained as an aerospace engineer, he quickly rose through the ranks of West Germany’s ambitious aerospace community so that in 1975, still under 40, he secured a position at a new research university of Germany’s armed forces.
By this point, he had already started mulling what would soon become his life mission: teaching vehicles how to see. The place to start, Dickmanns became increasingly convinced, was not spaceships but cars. Within a few years, he had bought a Mercedes van, installed it with computers, cameras and sensors, and began running tests on the university premises in 1986.
“The colleagues at the university said, well, he’s an oddball, but he’s got a track record [of achievements in aerospace technology,] so let’s just let him do it,” Dickmanns said during an interview at his family house, located steps from an onion-domed church in Hofolding, a small town outside of Munich.
In 1986, Dickmanns’ van became the first vehicle to drive autonomously — on the skidpan at his university. The next year, he sent it down an empty section of a yet-to-be-opened Bavarian autobahn at speeds approaching 90 kilometers per hour. Soon afterward, Dickmanns was approached by the German carmaker Daimler. Together, they secured funding from a massive pan-European project, and in the early 1990s, the company came up with an idea that first seemed “absurd” to Dickmanns.
“Can’t you equip one of our large passenger cars for the final demonstration of the project in Paris in October [of 1994], and then drive on the three-lane motorway in public traffic?” he remembered officials asking.
He had to take a deep breath, “but then I told them that with my team, and the methods we’re using, I think we’re capable of doing that.”
Daimler boosted the project’s funding. Car lobbyists ironed out doubts inside the French government. And in October 1994, Dickmanns’ team picked up a group of high-ranking guests from Charles de Gaulle airport, drove them to the nearby motorway and switched the two cars into self-driving mode.
“Sometimes, we would take our hands off the wheel” — Reinhold Behringer, one of the engineers who sat in the driver’s seat during the demonstration
An engineer remained in the front seat of each car — with his hands on the steering wheel in case something went wrong — but the cars were doing the driving.
“Sometimes, we would take our hands off the wheel,” said Reinhold Behringer, one of the engineers who sat in the driver’s seat during the demonstration, with excitement still in his voice 24 years later.
Newspapers ran frontpage stories about the demonstration, he remembered. And a year later, Dickmanns’ team took a re-engineered car on an even longer trip, traveling for more than 1,700 kilometers on the autobahn from Bavaria to Denmark, reaching speeds of more than 175 kilometers per hour.
Not long afterward, the project was over. The technology Dickmanns was using hit its limits. Daimler lost interest in funding the basic research needed to move it forward. Before long, Dickmanns’ pioneering effort was all but forgotten.
The history of artificial intelligence is a history of buzzy springs followed by what researchers call “AI winters,” when the attention and funding fades away.
Dickmanns’ work on autonomous driving began during the first winter and ended after a second one hit the field.
Research on AI — efforts to make machines do tasks that would otherwise require human thinking — started in the late 1950s. From its early days, the field was characterized by hype, leading some ambitious researchers like economist Herbert Simon to predict in the 1960s that machines would “be capable within 20 years of doing any work a man can do.”
Spurred by such promises, funding exploded — but technology failed to deliver and the bubble burst in the mid-1970s. Money dwindled and AI research was consigned to the backroom labs.
This first AI winter was one of the reasons Dickmanns kept his work on machine vision largely to himself in the early years. He was aware, he said, that “people would have said that guy has a screw loose somewhere.”
By the time he sent his self-driving van down an empty German autobahn in the mid-1980s, another AI spring had arrived. His proof of concept generated enough interest to hire a team that would eventually grow to 20 people ahead of the 1994 Paris demonstration.
Then came another winter, in the early 1990s, and Dickmanns’ momentum was lost.
“It was an interesting concept,” said Behringer, the engineer who sat behind the wheel in Paris. “But for many it was still way too futuristic.”
Teaching a car to see
Technologists say there two types of inventions: Those like the light bulb, which have been in use and continuously improved ever since they were first invented. And those like supersonic planes — remember the Concorde? — which embody revolutionary technological process but are too advanced to survive, at least at the time of their invention.
Dickmanns’ self-driving cars belong to the second category.
When he started developing them in the early 1980s, computers needed up to 10 minutes to analyze an image. To drive autonomously, a car needs to react to its surroundings, and to do that, Dickmanns calculated that computers would need to analyze at least 10 images per second.
Facing what looked like an insurmountable hurdle, he drew inspiration from human anatomy. Cars, he decided, should be programmed to see streets like humans perceive their surroundings.
The human eye is only able to see a small spot in the center of its visual field in high resolution. Similarly, Dickmanns thought, a car should focus only on what’s relevant for driving, such as road markings. This slashed the amount of information the onboard computers had to process.
He also found other computational shortcuts — a significant amount of computing time was freed up when Dickmanns realized he didn’t need to spend valuable processing power saving each image. He also programmed the car to learn from its mistakes, gradually improving its understanding of its surrounding.
Altogether, it was enough to keep the car on the road — barely.
Driving on a highway, it turns out, is one of the easier tasks a self-driving car can perform. The conditions are well-defined: Traffic flows predictably, in one direction. Lanes are clearly marked.
And even then, the demonstration didn’t go perfectly. “It was a test,” said Behringer. “When, for instance, there was a car in front of us that covered up the road markings, and on the other side, the markings were washed away, then the lane identification feature had a problem.”
After the second AI winter set in, and the buzz surrounding the Paris demonstration faded away, Daimler told Dickmanns it “wanted to have a product for the market as soon as possible,” he recalled. The carmaker had lost interest in his expensive fundamental research, which was unlikely to produce any real-life applications within the next couple of years.
“In hindsight, it was probably a mistake that those projects weren’t immediately continued,” Jürgen Schmidhuber, co-director of the Dalle Molle Institute for Artificial Intelligence Research in Lugano, Switzerland, said. “Otherwise there would be no question about who would be leading in the field today.”
German companies continue to hold most — almost half of all — patents in self-driving technology, but newer players, among them U.S. tech giants like Alphabet’s Waymo, have been catching up. Experts describe the current race for leadership in autonomous driving technology as neck-and-neck.
“There is a deep lack of awareness of what has been done in the past, especially among machine-learning scientists” — Longtime AI researcher
“It’s possible that [Germany] threw away its clear vanguard role because research wasn’t consistently continued at the time,” Schmidhuber said. He added that carmakers might have shied away from self-driving technology because it seemed to be in opposition to their marketing, which promoted the idea of a driver in charge of steering a car.
In the late 1990s, Dickmanns turned overseas and signed a four-year-contract with the United States’ Army Research Lab.
The cooperation led to another generation of self-driving cars, which were able to navigate more complicated surfaces; its results — published around the time Dickmanns retired — caught the eye of Darpa, the Pentagon’s emerging technologies division. It inspired the agency to launch a series of “challenges,” starting in 2004, tasking inventors with sending self-driving cars racing through spectacular territory.
Those challenges, promoted by massive marketing campaigns, were the first time a broad public heard about autonomous driving. They made German-born computer scientist Sebastian Thrun — who won the challenge in 2005 as a Stanford University professor and later founded Google’s self-driving team — a celebrity in the AI community.
Meanwhile, Ernst Dickmanns’ pioneering work fell into oblivion.
When in 2011, 17 years after Dickmanns’ Paris demonstration, the New York Times ran a front-page story about Thrun’s efforts to build a self-driving car, it had to run a correction afterward, making clear that “though Mr. Thrun developed a driverless car, he was not the first to do so.”
“There is a deep lack of awareness of what has been done in the past, especially among machine-learning scientists,” said one longtime AI researcher, who asked to remain anonymous.
He added that he regularly interviews high-ranking candidates who dismiss five-year-old papers as “outdated” or simply don’t know about research done in previous decades.
Winter is coming?
In 2018 — as AI undergoes yet another round of hype — could a new winter be looming? Some think that’s a distinct possibility.
Much recent research into AI has been into so-called “deep learning,” in which algorithms “learn” by recognizing patterns. Its underlying principle — finding correlations in complex data — works great for most applications but proves to be a dead end in some cases. And since deep learning is driven by data, its algorithms are always just as good as the data they’re being fed.
Filip Piękniewski — a San Diego-based computer scientist and the author of an essay titled “The AI Winter is well on its way” — said that much of the funding pouring into AI, particularly in the context of self-driving cars and robotics, is based on unrealistic expectations raised about what deep learning is capable of doing.
“This is the place where expectations collide with reality,” Piękniewski said. “And a lot of people will be annoyed that they invested so much money, and the expectations don’t materialize.”
Virginia Dignum, a professor at the University of Delft, agreed that if AI researchers keep focusing primarily on deep learning, “at some point, people will be disappointed.” The field, she said, has to look beyond it and invest in other approaches that depend on less data, or models based on causation rather than the correlation deep learning relies on.
But nonetheless — echoing widespread opinion among researchers and analysts — Dignum stressed she does not believe another “AI winter” is coming anytime soon. Unlike previous booms, today’s developers are turning cutting-edge AI into commercial real-life applications, thanks to recent technological advances starting in the early 2010s, particularly in computing power and data storing.
This makes the situation different from previous generations, which were often said to be doing “blue skies research” — scientists like Ernst Dickmanns, who said that he figured at the time of his experiments that it would take decades more until autonomous cars would ever become an everyday reality.
Indeed, Dickmanns, seated in his winter garden, said he still believes truly autonomous vehicles are still a decade or two away.
The self-driving vehicles currently being tested use a different, less computationally expensive process that needs less processing power in the on-board computer. They make up for the difference by using maps, GPS positioning and databases of previously observed objects.
“I’m glad I could be one of the pioneers. But if I could start anew today, with the technology that’s available, this would be a whole different story” — Ernst Dickmanns
Rather than truly “seeing,” Dickmanns said they rely on what he calls “confirmation vision.” That means they might work well on roads and areas that have been extensively mapped but fail when it comes to less controlled environments.
The approach he pioneered — which he calls “pathfinder vision” and which is still being pursued at a couple of research institutions — would allow cars to operate anywhere. “At some point, people will realize that after a storm, after an earthquake, or significantly more often in a military context when you get into new surroundings, the [current approach] won’t work,” he said.
Someday, he predicts, the industry will realize the limitations of its approach, and his work will see a resurgence.
“I’m glad I could be one of the pioneers,” he added, “But if I could start anew today, with the technology that’s available, this would be a whole different story.”