"They had on prison uniforms and they looked truly miserable. This was a big shock for me," recalled the 94-year-old, who is currently on trial for his role as an SS guard at the Stutthof concentration camp in what was then German-occupied Poland.
"The Wehrmacht officers were eloquent," said Rehbogen in a rare testimony read out in court by his lawyer last month. "They seemed downright heroic to us. But when I saw the prisoners, it was clear that this picture the Wehrmacht was trying to convey, was wrong."
Rehbogen has denied knowledge of a deliberate killing campaign.
The country is now racing against time to bring the last surviving perpetrators of Nazi war crimes -- now well into old age -- to justice.
But for many survivors it is too little, too late.
'Tiny percentage' of Nazis brought to justice
The number of suspects that have been brought to trial is a tiny percentage of the more than 200,000 perpetrators of Nazi-era crimes, said Mary Fulbrook, a professor of Germany History at University College London.
"It's way too late," she told CNN of the latest trials. "The vast majority of perpetrators got away with it."
"Immediately after the war, there were the Nuremberg trials. But these Allied trials were seen as victor's justice. This was, in a way, vilified and not taken seriously," she explained. "The first five to 10 years after the war, there were a lot of trials. Then they dwindled down massively," said Fulbrook.
"Then, in the interest of the Cold War and fighting communism, there was a move to rehabilitate former Nazis and a general climate of amnesty. Some perpetrators who were given severe sentences in the 1940s were released with much lighter sentences in the 1950s," Fullbrook said.
'It's easy to say you could have done things differently'
In the 1960s and 1970s a new generation of Germans pressed their parents and grandparents to answer: What did you do in the war?
In 2011, Demjanjuk was found guilty by a Munich court of being an accessory to the murder of more than 28,000 people based on evidence that he had served as an SS guard at the Sobibor Nazi death camp, a landmark decision that allowed prosecutors to go after lower-ranking suspected Nazi war criminals.
"From today's perspective, it's easy to say you could have done things differently in the 1950s," said Jens Rommel, lead prosecutor at Germany's Federal Authority for the Investigation of Nazi Crimes. "It may not have been possible to carry out the prosecution of tens of thousands of suspects as accessories to war crimes."
"There are also other reasons," said Rommel. "Many of the obvious leaders did not survive the war -- or died before they could be prosecuted. Some escaped prosecution by emigrating."
He added that "because of the combination of police officers, prosecutors and judges in Germany's post-war society -- people who may have had roles in the Third Reich -- the will to persecute was weakened."
Instead, Germany developed a "Culture of Remembrance" to address its wartime history. Memorials abound across the country and school children routinely visit memorial sites like Auschwitz to ensure the horrors of the Holocaust are never repeated.
"Germany is one of the most moral countries in addressing its history," said Fulbrook. "To some extent, it is an outpouring, an inherited sense of shame, but without being able to rectify that failure of bringing the guilty to justice."
"It's just an awful shame that while former Nazis were still in a position of influence, Germany didn't have the political will to bring Nazis to trial when they could have," she said.
The 'Bookkeeper of Auschwitz'
The recent trials have, however, facilitated a kind of belated dialogue between perpetrators and Holocaust survivors.
He made several statements in court, at times harrowing in their vivid detail but also repellent in his candid recollection of Nazi ideology.
On his first day in court, the 96-year-old recounted how he witnessed the murder of a small child. "A Jewish mother had hidden her little girl in a small suitcase on arrival. She was found out in the sorting. An SS commander took the baby and smashed the baby against the truck until her screaming stopped. My heart stopped," said Groening.
"I went to the man and said, 'You cannot do this.' But I was not allowed to question this. The next morning, I requested a transfer," he added.
Groening's testimony also showed that, at the time, he had been a fully indoctrinated member of the Nazi SS and though he had objected to the method of the killing, he had not opposed the murder itself.
"The baby broke the world for me. The horror of this action broke me," he said, adding, "It would have been different had he simply taken the gun and shot the baby dead."
Ironically, Groening's Nazi past was only discovered when he began speaking out against Holocaust deniers, by recounting his personal experiences in Auschwitz. Groening maintained that although he was never directly responsible for the killings he acknowledged his "moral complicity."
The court sentenced Groening to four years in prison. One of the Holocaust survivors who testified against him, Eva Kor, made a public statement forgiving Groening and demanding that his prison sentence be changed to community service.
"Groening said in his statement that he was wrong, it never should have happened, and it should never happen again. That is exactly what I want him to tell the young people in Germany who want to bring back a Nazi regime," Kor's statement said. "I told Oskar Groening that I have forgiven him, but that does not absolve or condone what he has done."
Groening died in the midst of appealing his sentence, but Fulbrook said his case highlighted the importance of recording the testimonies of alleged perpetrators.
"You certainly don't want to taint the memory of victims with their tormentors," she said. "But you do need more education about the Nazi system, what made it possible. Not just the nasty SS thugs but the wider group that made it possible."
A glimpse into the mindset of a SS soldier
Like Groening's testimony, Rehbogen's personal statement is rare. It reads like a diary, a glimpse into the mindset of an SS soldier.
His defense hinges on two claims -- that as an ethnic-German living in Hungary he was involuntarily drafted into the SS, and that he had no knowledge of the camp's systematic killings.
Rehbogen claims he was unaware of the existence of a gas chamber, though he remembered the stench that came from the crematorium. "Nothing could disguise that," his statement read.
"Even if it sounds like a flimsy justification, I did not perceive Stutthof as a camp designed to kill prisoners," he added. "I was aware that the conditions were terrible and that many died of disease and hunger. But that it was a systemic killing only dawned on me much later."
In court, a historian testifying as an expert witness disputed both of Rehbogan's claims, pointing out that more than 10,000 people were killed in Stutthof, despite the camp's small size.
Johannes Tuchel, the director of the German Resistance Memorial Center, backed up the historian's testimony.
"The Waffen SS did not have the ability to conscript people," Tuchel told CNN. "Germany and Hungary had a military alliance and Germany did not have the power to conscript in Hungary at the time. All ethnic Germans in Hungary in 1942 came voluntarily to the Waffen SS. I have not seen any documents that would prove otherwise."
Too little, too late for victims
One Holocaust survivor who remembers Rehbogan from Stutthof is Judy Meisel, who was 14 when she was ordered to line up naked outside the camp's gas chamber with her mother. The teenager survived after a guard suddenly pulled her out of the line.
"She was essentially ripped from her mother's arms at the steps of the gas chamber," Meisel's grandson, 34-year-old Benjamin Cohen, told CNN.
"Her mother told her, 'Run, Judy, run!' And she did. She ran all the way back and found her sister in the barracks and the two remained together and survived," Cohen said.
When German investigators contacted Judy Meisel, now 89, she immediately recognized Rehbogen as one of the guards -- though not the one that pulled her out that day.
"He must face responsibility for what he did when he was in Stutthof," Meisel wrote in a statement to the court. "Responsibility that he helped with the unimaginable crimes against humanity -- that he helped murder my beloved mother whom I have missed all my life."
Cohen told CNN that if the trial had happened 10 years earlier, Meisel would have been able to attend herself. Instead, because of her frail health, he sat in her place, watching as Rehbogen was brought to court in a wheelchair.
"It's never easy to see an old man wheeled into a courtroom, but I mostly thought about how disappointing it is that these trials have taken so long to happen," Cohen told CNN.
"My hope is that he would at least tell us what happened, even if he refuses to admit to anything he did. Instead, he wants us to believe he could stand guard in the camp for two years and not know anything."
In court, Rehbogen concluded his statement by saying, "I would like to say once again that I am not a Nazi. I never was one. And in the little time I have left, I will never be one."
Now, he waits for Germany's courts to decide.