Specifically, I want to know if there is evidence in the early Church of Christians venerating Mary. Evidence from the time following Mary's earthly life is preferred, though not required.
As has been mentioned in answers to other questions, the prayer Sub tuum praesidium appeared around 250. It was originally written in Greek.
Ὑπὸ τὴν σὴν εὐσπλαγχνίαν,
Τὰς ἡμῶν ἱκεσίας, μὴ παρίδῃς ἐν περιστάσει,
ἀλλ᾽ ἐκ κινδύνων λύτρωσαι ἡμᾶς,
μόνη Ἁγνή, μόνη εὐλογημένη.
Beneath your compassion,
We take refuge, O Mother of God:
do not despise our petitions in time of trouble:
but rescue us from dangers,
only pure, only blessed one.
Dr Mark Miravalle of the Franciscan University of Steubenville has written an entire book on the subject, Introduction to Mary: The Heart of Marian Doctrine and Devotion [Queenship 2006], a summary of which — with references — is available from piercedhearts.org. He mentions St Ambrose (d. 397) calling Mary "Mother of Salvation".
He goes on
The Christian witness of the first centuries of the Church also provides us with examples of direct prayer to Mary as a means of intercession to the graces and the protection of her Son.
For St Irenaeus, Mary is an "Advocate," or interceding helper, for Eve and for her salvation. St. Gregory Thaumaturgis (d.350) depicts Mary interceding for those on earth from her position in Heaven.
St Ephraem (d.373), the great Eastern doctor and deacon, directly addresses the Blessed Virgin in several Marian sermons. Direct prayer to Mary is also found in a sermon of the great Eastern Father, St Gregory Nazianzen (330-389). By the last part of the fourth century and the beginning of the fifth, we have numerous explicit examples of direct prayer to the Mother of God, for example in the writings of St Ambrose, as well as by St Epiphanius.
The amount of evidence that is found in early Church patristic writings to support Marian veneration/devotion is so overwhelming that it must be categorized into 5 basic teachings:
1 - Mary is the New Eve and Most Blessed Among Women
2 - Mary is our Powerful Intercessor
3 - Mary is Ever-virgin
4 - Mary’s Immaculate Conception
5 - Mary is the Mother of God
Quoted here are only a handful of writings from Early Church Fathers. There are many more excerpts predating A. D. 1000, but for now we will only look at the first four centuries.
Mary is the New Eve and Most Blessed Among Women
“There is one Physician who is possessed both of flesh and spirit; both made and not made; God existing in flesh; true life in death; both of Mary and of God; first possible and then impossible, even Jesus Christ our Lord." Ignatius, To the Ephesians, 7 (c. A.D. 110).
"[T]hey blessed her, saying: O God of our fathers, bless this child, and give her an everlasting name to be named in all generations. And all the people said: So be it, so be it, amen. And he brought her to the chief priests; and they blessed her, saying: O God most high, look upon this child, and bless her with the utmost blessing, which shall be for ever." Protoevangelium of John, 6:2 (A.D. 150).
"He became man by the Virgin, in order that the disobedience which proceeded from the serpent might receive its destruction in the same manner in which it derived its origin. For Eve, who was a virgin and undefiled, having conceived the word of the serpent, brought forth disobedience and death. But the Virgin Mary received faith and joy, when the angel Gabriel announced the good tidings to her that the Spirit of the Lord would come upon her, and the power of the Highest would overshadow her: wherefore also the Holy Thing begotten of her is the Son of God; and she replied, 'Be it unto me according to thy word.' And by her has He been born, to whom we have proved so many Scriptures refer, and by whom God destroys both the serpent and those angels and men who are like him; but works deliverance from death to those who repent of their wickedness and believe upon Him." Justin Martyr, Dialogue with Trypho, 100 (A.D. 155).
"In accordance with this design, Mary the Virgin is found obedient, saying, 'Behold the handmaid of the Lord; be it unto me according to thy word.' But Eve was disobedient; for she did not obey when as yet she was a virgin. And even as she, having indeed a husband, Adam, but being nevertheless as yet a virgin (for in Paradise 'they were both naked, and were not ashamed,' inasmuch as they, having been created a short time previously, had no understanding of the procreation of children: for it was necessary that they should first come to adult age, and then multiply from that time onward), having become disobedient, was made the cause of death, both to herself and to the entire human race; so also did Mary, having a man betrothed [to her], and being nevertheless a virgin, by yielding obedience, become the cause of salvation, both to herself and the whole human race. And on this account does the law term a woman betrothed to a man, the wife of him who had betrothed her, although she was as yet a virgin; thus indicating the back-reference from Mary to Eve, because what is joined together could not otherwise be put asunder than by inversion of the process by which these bonds of union had arisen; s so that the former ties be cancelled by the latter, that the latter may set the former again at liberty… Wherefore also Luke, commencing the genealogy with the Lord, carried it back to Adam, indicating that it was He who regenerated them into the Gospel of life, and not they Him. And thus also it was that the knot of Eve's disobedience was loosed by the obedience of Mary. For what the virgin Eve had bound fast through unbelief, this did the virgin Mary set free through faith." Irenaeus, Against Heresies, 3:22 (A.D. 180).
"For whereas the Word of God was without flesh, He took upon Himself the holy flesh by the holy Virgin, and prepared a robe which He wove for Himself, like a bridegroom, in the sufferings of the cross, in order that by uniting His own power with our moral body, and by mixing the incorruptible with the corruptible, and the strong with the weak, He might save perishing man." Hippolytus, Treatise on Christ and antiChrist, 4 (A.D. 200).
"But the Lord Christ, the fruit of the Virgin, did not pronounce the breasts of women blessed, nor selected them to give nourishment; but when the kind and loving Father had rained down the Word, Himself became spiritual nourishment to the good. O mystic marvel! The universal Father is one, and one the universal Word; and the Holy Spirit is one and the same everywhere, and one is the only virgin mother. I love to call her the Church. This mother, when alone, had not milk, because alone she was not a woman. But she is once virgin and mother--pure as a virgin, loving as a mother. And calling her children to her, she nurses them with holy milk, viz., with the Word for childhood. Therefore she had not milk; for the milk was this child fair and comely, the body of Christ, which nourishes by the Word the young brood, which the Lord Himself brought forth in throes of the flesh, which the Lord Himself swathed in His precious blood." Clement of Alexandria, The Instructor, I:6 (A.D.202).
"Accordingly, a virgin did conceive and bear 'Emmanuel, God with us.' This is the new nativity; a man is born in God. And in this man God was born, taking the flesh of an ancient race, without the help, however, of the ancient seed, in order that He might reform it with a new seed, that is, in a spiritual manner, and cleanse it by the re-moral of all its ancient stains. But the whole of this new birth was prefigured, as was the case in all other instances, in ancient type, the Lord being born as man by a dispensation in which a virgin was the medium. The earth was still in a virgin state, reduced as yet by no human labour, with no seed as yet cast into its furrows, when, as we are told, God made man out of it into a living soul…For it was while Eve was yet a virgin, that the ensnaring word had crept into her ear which was to build the edifice of death. Into a virgin's soul, in like manner, must be introduced that Word of God which was to raise the fabric of life; so that what had been reduced to ruin by this sex, might by the selfsame sex be recovered to salvation. As Eve had believed the serpent, so Mary believed the angel. The delinquency which the one occasioned by believing, the other by believing effaced. But (it will be said) Eve did not at the devil's word conceive in her womb. Well, she at all events conceived; for the devil's word afterwards became as seed to her that she should conceive as an outcast, and bring forth in sorrow. Indeed she gave birth to a fratricidal devil; whilst Mary, on the contrary, bare one who was one day to secure salvation to Israel, His own brother after the flesh, and the murderer of Himself. God therefore sent down into the virgin's womb His Word, as the good Brother, who should blot out the memory of the evil brother. Hence it was necessary that Christ should come forth for the salvation of man, in that condition of flesh into which man had entered ever since his condemnation." Tertullian, Flesh of Christ, 17 (A.D. 212).
"And I think it in harmony with reason that Jesus was the first-fruit among men of the purity which consists in chastity, and Mary among women; for it were not pious to ascribe to any other than to her the first-fruit of virginity." Origen, Commentary on Matthew, 10:17 (A.D. 244).
"Many, my beloved, are the true testimonies concerning Christ. The Father bears witness from heaven of His Son: the Holy Ghost bears witness, descending bodily in likeness of a dove: the Archangel Gabriel bears witness, bringing good tidings to Mary: the Virgin Mother of God [Theotokos] bears witness: the blessed place of the manger bears witness" Cyril of Jerusalem, Catechetical Lectures, 10:19 (A.D. 350).
"In what remains we have the appointment of the Father's will. The Virgin, the birth, the Body, then the Cross, the death, the visit to the lower world; these things are our salvation. For the sake of mankind the Son of God was born of tile Virgin and of the Holy Ghost. In this process He ministered to Himself; by His own power--the power of God--which overshadowed her He sowed the beginning of His Body, and entered on the first stage of His life in the flesh. He did it that by His Incarnation He might take to Himself from the Virgin the fleshly nature, and that through this commingling there might come into being a hallowed Body of all humanity; that so through that Body which He was pleased to assume all mankind might be hid in Him, and He in return, through His unseen existence, be reproduced in all. Thus the invisible Image of God scorned not the shame which marks the beginnings of human life. He passed through every stage; through conception, birth, wailing, cradle and each successive humiliation. What worthy return can we make for so great a condescension? The One Only-begotten God, ineffably born of God, entered the Virgin's womb and grew and took the frame of poor humanity. He Who upholds the universe, within Whom and through Whom are all things, was brought forth by common childbirth; He at Whose voice Archangels and Angels tremble, and heaven and earth and all the elements of this world are melted, was heard in childish wailing. The Invisible and Incomprehensible, Whom sight and feeling and touch cannot gauge, was wrapped in a cradle.” Hilary of Poitiers, On the Trinity, 2:24-25 (A.D. 355).
"And when he had taken her, 'he knew her not, till she had brought forth her first-born Son.' He hath here used the word 'till,' not that thou shouldest suspect that afterwards he did know her, but to inform thee that before the birth the Virgin was wholly untouched by man." John Chrysostom, Homily on Matthew, 5:5 (A.D. 370).
Mary is our Powerful Intercessor
"For as Eve was seduced by the word of an angel to flee from God, having rebelled against His Word, so Mary by the word of an angel received the glad tidings that she would bear God by obeying his Word. The former was seduced to disobey God, but the latter was persuaded to obey God, so that the Virgin Mary might become the advocate of the virgin Eve. As the human race was subjected to death through [the act of] a virgin, so it was saved by a virgin." Irenaeus, Against Heresies, V:19,1 (A.D. 180).
"Under your mercy we take refuge, O Mother of God. Do not reject our supplications in necessity, but deliver us from danger,[O you] alone pure and alone blessed." Sub Tuum Praesidium, From Rylands Papyrus, Egypt (3rd century).
"Recalling these and other circumstances and imploring the Virgin Mary to bring assistance, since she, too, was a virgin and had been in danger, she entrusted herself to the remedy of fasting and sleeping on the ground." Gregory of Nazianzen, Oration 24:11 (A.D. 379).
"Mary, the holy Virgin, is truly great before God and men. For how shall we not proclaim her great, who held within her the uncontainable One, whom neither heaven nor earth can contain?" Epiphanius, Panarion, 30:31 (ante A.D. 403).
"Give milk, Mother to him who is our food, give milk to the bread coming down from heaven ...give milk to him who made you such that he could be made fruitfulness in conception and in birth, did not take from you the ornament of virginity." Augustine, Sermon 369:1 (A.D. 430).
"Hail to thee Mary, Mother of God, to whom in towns and villages and in island were founded churches of true believers." Cyril of Alexandria, Homily 11 (ante A.D. 444).
"Hail, our desirable gladness; Hail, O rejoicing of the Churches; Hail, O name that breathes out sweetness; Hail, face that radiates divinity and grace; Hail, most venerable memory…" Theodotus of Ancrya, Homily 4:3 (ante A.D. 446)
Mary is Ever-virgin
“And indeed it was a virgin, about to marry once for all after her delivery, who gave birth to Christ, in order that each title of sanctity might be fulfilled in Christ's parentage, by means of a mother who was both virgin, and wife of one husband. Again, when He is presented as an infant in the temple, who is it who receives Him into his hands? Who is the first to recognize Him in spirit? A man just and circumspect,' and of course no digamist, (which is plain) even (from this consideration), lest (otherwise) Christ should presently be more worthily preached by a woman, an aged widow, and the wife of one man;' who, living devoted to the temple, was (already) giving in her own person a sufficient token what sort of persons ought to be the adherents to the spiritual temple,--that is, the Church. Such eye-witnesses the Lord in infancy found; no different ones had He in adult age."Tertullian, On Monogamy, 8 (A.D. 213).
"For if Mary, as those declare who with sound mind extol her, had no other son but Jesus, and yet Jesus says to His mother, Woman, behold thy son,' and not Behold you have this son also,' then He virtually said to her, Lo, this is Jesus, whom thou didst bear.' Is it not the case that every one who is perfect lives himself no longer, but Christ lives in him; and if Christ lives in him, then it is said of him to Mary, Behold thy son Christ.' What a mind, then, must we have to enable us to interpret in a worthy manner this work, though it be committed to the earthly treasure-house of common speech, of writing which any passer-by can read, and which can be heard when read aloud by any one who lends to it his bodily ears?" Origen, Commentary on John, I:6 (A.D. 232).
"Therefore let those who deny that the Son is from the Father by nature and proper to His Essence, deny also that He took true human flesh of Mary Ever-Virgin; for in neither case had it been of profit to us men, whether the Word were not true and naturally Son of God, or the flesh not true which He assumed." Athanasius, Orations against the Arians, II:70 (A.D. 362).
"And when he had taken her, he knew her not, till she had brought forth her first-born Son.' He hath here used the word till,' not that thou shouldest suspect that afterwards he did know her, but to inform thee that before the birth the Virgin was wholly untouched by man. But why then, it may be said, hath he used the word, till'? Because it is usual in Scripture often to do this, and to use this expression without reference to limited times. For so with respect to the ark likewise, it is said, The raven returned not till the earth was dried up.' And yet it did not return even after that time. And when discoursing also of God, the Scripture saith, From age until age Thou art,' not as fixing limits in this case. And again when it is preaching the Gospel beforehand, and saying, In his days shall righteousness flourish, and abundance of peace, till the moon be taken away,' it doth not set a limit to this fair part of creation. So then here likewise, it uses the word "till," to make certain what was before the birth, but as to what follows, it leaves thee to make the inference.” John Chrysostom, Gospel of Matthew, V:5 (A.D. 370).
Mary’s Immaculate Conception
"He was the ark formed of incorruptible wood. For by this is signified that His tabernacle was exempt from putridity and corruption." Hippolytus, Orations Inillud, Dominus pascit me (ante A.D. 235).
"This Virgin Mother of the Only-begotten of God, is called Mary, worthy of God, immaculate of the immaculate, one of the one." Origen, Homily 1(A.D. 244).
"Let woman praise Her, the pure Mary." Ephraim, Hymns on the Nativity, 15:23 (A.D. 370).
"Thou alone and thy Mother are in all things fair, there is no flaw in thee and no stain in thy Mother." Ephraem, Nisibene Hymns, 27:8 (A.D. 370).
"O noble Virgin, truly you are greater than any other greatness. For who is your equal in greatness, O dwelling place of God the Word? To whom among all creatures shall I compare you, O Virgin? You are greater than them all O Covenant, clothed with purity instead of gold! You are the Ark in which is found the golden vessel containing the true manna, that is, the flesh in which divinity resides." Athanasius, Homily of the Papyrus of Turin, 71:216 (ante AD 373).
"Mary, a Virgin not only undefiled but a Virgin whom grace has made inviolate, free of every stain of sin." Ambrose, Sermon 22:30 (A.D. 388). "We must except the Holy Virgin Mary, concerning whom I wish to raise no question when it touches the subject of sins, out of honour to the Lord; for from Him we know what abundance of grace for overcoming sin in every particular was conferred upon her who had the merit to conceive and bear Him who undoubtedly had no sin." Augustine, Nature and Grace,4 2 (A.D.415).
"As he formed her without my stain of her own, so He proceeded from her contracting no stain." Proclus of Constantinople, Homily 1 (ante A.D. 446). "A virgin, innocent, spotless, free of all defect, untouched, unsullied, holy in soul and body, like a lily sprouting among thorns." Theodotus of Ancrya, Homily VI:11(ante A.D. 446).
"The angel took not the Virgin from Joseph, but gave her to Christ, to whom she was pledged from Joseph, but gave her to Christ, to whom she was pledged in the womb, when she was made." Peter Chrysologus, Sermon 140 (A.D. 449).
Mary is the Mother of God
"After this, we receive the doctrine of the resurrection from the dead, of which Jesus Christ our Lord became the first-fruits; Who bore a Body, in truth, not in semblance, derived from Mary the mother of God in the fullness of time sojourning among the race, for the remission of sins: who was crucified and died, yet for all this suffered no diminution of His Godhead." Alexander of Alexandria, Epistle to Alexander, 12 (A.D. 324).
"Many, my beloved, are the true testimonies concerning Christ. The Father bears witness from heaven of His Son: the Holy Ghost bears witness, descending bodily in likeness of a dove: the Archangel Gabriel bears witness, bringing good tidings to Mary: the Virgin Mother of God bears witness: the blessed place of the manger bears witness." Cyril of Jerusalem, Catechetical Lectures, X:19 (c. A.D. 350).
"And the Angel on his appearance, himself confesses that he has been sent by his Lord; as Gabriel confessed in the case of Zacharias, and also in the case of Mary, bearer of God." Athanasius, Orations III, 14(A.D. 362).
"Just as, in the age of Mary the mother of God, he who had reigned from Adam to her time found, when he came to her and dashed his forces against the fruit of her virginity as against a rock, that he was shattered to pieces upon her, so in every soul which passes through this life in the flesh under the protection of virginity, the strength of death is in a manner broken and annulled, for he does not find the places upon which he may fix his sting." Gregory of Nyssa, On Virginity, 14 (A.D. 370).
"He reshaped man to perfection in Himself, from Mary the Mother of God through the Holy Spirit." Epiphanius, The man well-anchored, 75 (A.D. 374).
"Let, then, the life of Mary be as it were virginity itself, set forth in a likeness, from which, as from a mirror, the appearance of chastity and the form of virtue is reflected. From this you may take your pattern of life, showing, as an example, the clear rules of virtue: what you have to correct, to effect, and to hold fast. The first thing which kindles ardour in learning is the greatness of the teacher. What is greater than the Mother of God?"Ambrose, Virginity, II:6 (c. A.D. 378).
"If anyone does not believe that Holy Mary is the Mother of God, he is severed from the Godhead." Gregory of Nazianzus, To Cledonius, 101 (A.D. 382).
"To the question: 'Is Mary the bearer of Man, or the bearer of God?' we must answer: 'Of Both.'" Theodore of Mopsuestia, The Incarnation, 15 (ante A.D. 428).
"And so you say, O heretic, whoever you may be, who deny that God was born of the Virgin, that Mary the Mother of our Lord Jesus Christ ought not to be called Theotocos, i.e., Mother of God, but Christotocos, i.e., only the Mother of Christ, not of God. For no one, you say, brings forth what is anterior in time. And of this utterly foolish argument whereby you think that the birth of God can be understood by carnal minds, and fancy that the mystery of His Majesty can be accounted for by human reasoning, we will, if God permits, say something later on. In the meanwhile we will now prove by Divine testimonies that Christ is God, and that Mary is the Mother of God." John Cassian, The Incarnation of Christ, II:2 (A.D. 430).
St. John Chrysostom’s well know Divine Liturgy is saturated with the utmost devotion to The Blessed Virgin Mary.
In the entrance prayers and 1rst 2 antiphons the following prayer is offered to God:
Deacon: Remembering our most holy, pure, blessed, and glorious Lady, the Theotokos and ever virgin Mary, with all the saints, let us commit ourselves and one another and our whole life to Christ our God.
On a final note, St. Augustine (who is for the most part universally accepted as one of the Church’s greatest theologians) composed the following well known prayer of veneration to Mary:
O blessed Virgin Mary, who can worthily repay thee thy just dues of praise and thanksgiving, thou who by the wondrous assent of thy will didst rescue a fallen world? What songs of praise can our weak human nature recite in thy honor, since it is by thy intervention alone that it has found the way to restoration. Accept, then, such poor thanks as we have here to offer, though they be unequal to thy merits; and receiving our vows, obtain by thy prayers the remission of our offenses. Carry thou our prayers within the sanctuary of the heavenly audience, and bring forth from it the antidote of our reconciliation. May the sins we bring before Almighty God through thee, become pardonable through thee; may what we ask for with sure confidence, through thee be granted. Take our offering, grant us our requests, obtain pardon for what we fear, for thou art the sole hope of sinners. Through thee we hope for the remission of our sins, and in thee, O blessed Lady, is our hope of reward. Holy Mary, succor the miserable, help the fainthearted, comfort the sorrowful, pray for thy people, plead for the clergy, intercede for all women consecrated to God; may all who keep thy holy commemoration feel now thy help and protection. Be thou ever ready to assist us when we pray, and bring back to us the answers to our prayers. Make it thy continual care to pray for the people of God, thou who, blessed by God, didst merit to bear the Redeemer of the world, who liveth and reigneth, world without end. Amen.
To venerate is to revere or show respect.
Mary is venerated in scripture by angels, her relatives, the disciples and Jesus on the cross.
The angel said:
Luke 1:28-31 Hail, [thou that art] highly favoured, the Lord [is] with thee: blessed [art] thou among women…And the angel said unto her, Fear not, Mary: for thou hast found favour with God. And, behold, thou shalt conceive in thy womb, and bring forth a son, and shalt call his name JESUS.
Luke 1:42-43 Blessed [art] thou among women, and blessed [is] the fruit of thy womb. And whence [is] this to me, that the mother of my Lord should come to me?
In Acts 1:14 Mary the mother of Jesus was identified with those that waited in the upper room for the Holy Spirit.
Romans 16:6 Greet Mary, who bestowed much labour on us
John 19:26 When Jesus therefore saw his mother, and the disciple standing by, whom he loved, he saith unto his mother, Woman, behold thy son