If we go back to the earliest Christian teachers we will find a substantial focus on metaphorical and spiritual meanings of the Bible, more than maybe a modern Christian might expect. Roger Forster and Paul Marston write in "Reason and Faith" (Monarch, 1989):
In [the Church Fathers] there was, compared with today, a much greater emphasis on allegorical meaning of scripture. Thus, for example, Psalm 90:4 and 2 Peter 3:8 led many of them to take an allegorical interpretation of the 'days' in Genesis 1 to mean millenia. This view is expressed, for example, by Barnabas, Irenaeus, Hippolytus, Methodius, Lactantius, Theophilus and John of Damascus.
That was not the universal view:
Basil specifically refers to 24 hour periods, yet later writes: "Whether you call it a 'day' or whether you call it 'eternity', you express the same idea.
Chrysostom, who appears to take the 'days' literally in his homilies, repeatedly emphasises that ideas are being given 'concreteness of expression' in Genesis 1-3, to help our 'limited human understanding'. Thus on the 'rib' used to form Eve, he writes: "Don't take the words in human fashion; rather interpret the concreteness of the expressions from the viewpoint of human limitations. You see, if he had not used these words, how would we have been able to gain knowledge of these mysteries, which defy description".
Origen writes (around 231 AD):
What man of intelligence, I ask, will consider it a reasonable statement that the first and the second and the third day, in which there are said to be both morning and evening, existed without sun and moon and stars, while the first day was even without a heaven? And who could be found so silly as to believe that God, after the manner of a farmer, 'planted trees in a paradise East of Eden'? ... And ... when God is said to 'walk in the paradise in the evening, ... I do not think anyone will doubt that these are figurative expressions which indicate certain mysteries through semblance of history.
Augustine of Hippo writes:
Of what fashion these days were, it is either very hard or almost impossible to think, much more to speak. As for ordinary days, we see that they have no morning or evening but as the sun sets and rises. But the first three days had no sun, for that was made on the fourth day.
Augustine thought that the Genesis language reflected the Angelic perspective, which could know something either directly in God or in its later actual being.
The issue was not thought of as a crucial one to Christian belief.
The scientific evidence that pointed to an extremely old earth did not arise until the 18th century. This redating was not at the time generally taken as a 'disproof' of either Genesis or Christianity, and many of the scholars supporting it were Christian.
Significant Christian opposition to the geological dating did not arise until the early 20th century, with the Fundamentalism revival. Leading early exponents were George McReady price and Henry Morris. The Wikipedia article gives an excellent overview of the history.