When and how did Martin Luther arrive at the justification by faith teaching? Was it something that he came up with as a result of his stance against the indulgences and later against the Pope, or did he arrive at this teaching even prior to those events?
First to understand the background to the story that Luther recalls, Luther used to hate Romans 1:17. He struggled with this verse in particular, and the phrase 'God's righteousness' in particular, because he always read it in the sense which it was preached by the Catholic theologians at the time. At that time this verse was understood as the "formal or active righteousness" with which "God is righteous and punishes the unrighteous sinner." In other words, Luther while believing in God and having some faith in Christ struggled with Romans 1:17 before his 'confidence burst' and his faith began to posses him more violently. This is why Romans 1:17 is brought up in his recollections but actually plays little part in his explaining his actual beliefs later on. This he did, regarding justification by faith, most fully in his lectures on Galatians, although Romans as a whole still held an important place as well.
His experience or turning point in breaking through on his understanding of Romans 1:17 is referred to as his 'Tower Experience' because it occurred in the tower of the Black Cloister in Wittenberg (later Luther’s home) at an undetermined date between 1508 and 1518.
Luther, before overcoming his doubts about Romans 1:17, used to think 'God's righteousness' in the gospel 'was revealed', not in giving perfect righteousness freely to sinners forever apart from the fact they were sinners, but in punishing sinners and rewarding the righteous. Luther, originally as a monk, viewed the gospel as an extension of the law, not a way to find freedom from its curse. Only later did he discover that a person is saved by faith, without works of the law entering into the equation. He always saw that faith produced many works, but not allowing those works to take part in the subject of justification, where we 'passively receive' righteousness as a gift, apart from our own merit, was something he learned later on. I think the Romans 1:17 'tower experience' that he had was probably during his lectures on Romans which began in the year 1516.
Before this experience He says he had faith but it was not clear yet:
In 1545, he describes his own experience at greater length. He seems to take a longer view if it, like a growing faith and struggle that begins by referring to his days as a monk, his weak faith in his earlier days as a Biblical Professor and finally his overwhelming sense of what Romans 1:17 really meant. He recalled these experiences when describing the events that occurred in the year 1519 when he got into trouble with the Pope. To understand at what point in his thinking 1519 occurs it is helpful to know that Luther first lectured on Romans at around the year 1516, he also lectured a preliminary version of Galatians and Hebrews shortly after this time. However, he did not lecture on Galatians, formally, in the format in which they were published, until around 1531. It took around 16 years for his faith to really explode in the form of Galatians, long after he had kicked the hornets nest in Rome.
In fact, this gripping realization of justification by faith made him revisit his old lectures and begin to rewrite them as early as 1519. It is just before this time that he made his breakthrough in the 'tower experience'. Removing his misunderstanding of that verse, his faith seems to have broke into a full confidence and the verse that used to trouble him became an anchor that symbolized his overwhelming conviction that began slowly years before and grew more and more years later. Here we find he began to revise his work on the Psalms in 1519 and his breakthrough over his doubts about Romans Chapter 1:17 that had 'stood in his way' until this point.
He continues to explain the effects of his experience about Romans 1:17 and how he later found additional support through Augustine. He already had his doctorate in theology in October 19, 1512 and may seem strange that he did not encounter Augustine's work on the subject until years later, but it is a very specific one which Luther mentions, called 'The Spirit and the Letter.'
The truth is although Romans was the place where he made his initial breakthrough it was not the place where his faith finally rested on. For this we must turn to Galatians. In fact Luther hardly has any comments at all under Chapter 1:7 in his works on Romans, but in every verse of Galatians, Luther uses as one more opportunity to pound and pound away at the doctrine that changed his life forever. No bible commentary on any book in the Bible since can pretend in any way to have had the same impact on the world.
As a result of the revolutionary changes in Luther and his dramatic growing faith is there is a problem with Luther's Works in that he had to re-write many things and where the need was not urgent he seems to have left them as they were. Romans is a little thin on doctrine compared to Galatians, as he wrote it years before, it is also was not needing much revision, so Luther probably found most of it still acceptable to him even under his enlarged views years later.
His commentary on Galatians was when he was crystal clear in 1531 and he does not seem to have had enough time to fully rewrite everything before to measure up to his final stage of assurance and knowledge. Therefore regarding the timing of his understanding of the doctrine that in many ways resulted in the Protestant church, I would say that his faith was crystallized around 1517 with Romans and from there it grew until it exploded with Galatians in 1531. He seems to have hung his faith not on Romans at all by this point. Galatians was his eventual favorite work and the essential Luther.
For anyone interested in reading Luther, He wrote his works on Genesis after Galatians so they are not in any need up of updating at all and are a good place to start after Galatians. Some of his earlier works however must be viewed and even possibly corrected by comparing them to Galatians.
After all his years Luther clearly favored his writing on Galatians above all else. I am sure he would have instantly agreed to the burning of all his books if he might keep his work on Galatians. It is here where you find Luther's views in the doctrine of salvation by faith, apart from works. One can't understand Luther at all without reading it. Anyone who has read it will understand why. I challenge anyone interested in Luther to read his work on Galatians in order to begin to understand him.
According to Luther's own testimony, it was after an in depth study of Romans. Romans 1:17 in particular (often called Paul's thesis statement for the book of Romans) says:
That last line (from Habakkuk) convinced Luther that no work of man, and hence no religious decree of the Church, trumped Faith.