Christians know Mary was a virgin in conceiving and bearing Christ to term. The belief that Mary remained a virgin consists of two additional parts. The first is that she remained a virgin even in the act of giving birth and second is that she remained a virgin thereafter. These two beliefs source to the same book; that is, the Infancy Gospel of James (aka Book of James or Protoevangelium of James) (PoJ). The PoJ was written c170 CE. Origen mentioned it about 220 CE. Clement of Alexandria also mentions it about 200 CE. Augustine, also a “church father”, however, appears to be the first who will clearly clean up, expound, and tie the doctrine together from where the PoJ ends.
The first part of the ever-virgin idea (even in birth) was explained in the PoJ that there was no afterbirth, no umbilical cord, no placenta; it was not a normal birth. The baby just appeared. She thus remained a virgin.
And immediately the cloud disappeared out of the cave, and a great light shone in the cave, so that the eyes could not bear it. And in a little that light gradually decreased, until the infant appeared, and went and took the breast from His mother Mary. And the midwife cried out, and said: This is a great day to me, because I have seen this strange sight. And the midwife went forth out of the cave, and Salome met her. And she said to her: Salome, Salome, I have a strange sight to relate to you: a virgin has brought forth— a thing which her nature admits not of. Then said Salome: As the Lord my God lives, unless I thrust in my finger, and search the parts, I will not believe that a virgin has brought forth. (source)
Clement of Alexandria, a “church father” confirms the PoJ as the source of the belief that Mary remained a virgin during birth, even as he denies its veracity.
But, as appears, many even down to our own time regard Mary, on account of the birth of her child, as having been in the puerperal state, although she was not. For some say that, after she brought forth, she was found, when examined, to be a virgin. (source)
In other words, when examined, PoJ says that Mary was still a virgin, remained in the puerperal (childbirth) state. This means it was not a normal human birth with everything that entails. Clement denies she remained in that childbirth state; that is, Christ was born normally and virginity would thus be over. Keep this in mind as Augustine will pick up on this PoJ teaching.
The second part of the ever-virgin idea (no sex) means that there were no subsequent brothers or sisters of Christ. There was no sex between Mary and Joseph. The brothers are really step-brothers; this was also explained in the PoJ that Joseph was previously married and had children.
And the priest said to Joseph, You have been chosen by lot to take into your keeping the virgin of the Lord. But Joseph refused, saying: I have children, and I am an old man, and she is a young girl. I am afraid lest I become a laughing-stock to the sons of Israel. (source)
Origen, also a “church father” confirms this book as the source of the ever-virgin after birth by referencing in the book that Jesus' brothers were sons of Joseph and a previous wife saying,
And depreciating the whole of what appeared to be His nearest kindred, they said, “Is not His mother called Mary? And His brethren, James and Joseph and Simon and Judas? And His sisters, are they not all with us?” They thought, then, that He was the son of Joseph and Mary. But some say, basing it on a tradition in the Gospel according to Peter, as it is entitled, or “The Book of James,” that the brethren of Jesus were sons of Joseph by a former wife, whom he married before Mary. (source)
So, we’ve seen that the idea of the ever-virgin during birth and no sex afterwards sources to PoJ, but is mentioned early on by two “church fathers”. One disagrees and one agrees. Here now is Augustine some 200 years later picking up and tying together the two-pronged trail that Mary remained a virgin at birth and after birth.
And if her [Mary] virginity had been marred even in bringing Him [Christ] forth, He would not have been born of a virgin; and it would be false (which God forbid) that He was born of the Virgin Mary (source)
This next quote is from Aquinas who quotes Augustine about the closed gate (no sex thereafter).
It is written (Ezech. 44:2): "This gate shall be shut, it shall not be opened, and no man shall pass through it; because the Lord the God of Israel hath entered in by it." Expounding these words, Augustine says in a sermon (De Annunt. Dom. iii): "What means this closed gate in the House of the Lord, except that Mary is to be ever inviolate? What does it mean that 'no man shall pass through it,' save that Joseph shall not know her? And what is this---'The Lord alone enters in and goeth out by it'---except that the Holy Ghost shall impregnate her, and that the Lord of angels shall be born of her? And what means this---'it shall be shut for evermore'---but that Mary is a virgin before His Birth, a virgin in His Birth, and a virgin after His Birth?" (source)
This interpretation and application that the eastern gate would be shut simply follows the PoJ way that the baby just appears at her side; that is, at her eastern gate, rather than being born normally in the normal manner through the “southern gate”.
So, while the author, whoever it may be, of PoJ is not considered a “church father”, Origen, Augustine, Aquinas, and many other “church fathers” over the centuries continue to trek on its way that the brothers of Jesus were sons of Joseph and a previous wife, that Mary remained in the puerperal state at His birth, and that Jesus was born out from her side. Thus, in this way, she remained a virgin even in the act of giving birth and thereafter.
There is a common misunderstanding that Irenaeus supported the notion of ever-virginity. He didn't.
And if the former [Eve] did disobey God, yet the latter was persuaded to be obedient to God, in order that the Virgin Mary might become the patroness4617 (advocata) of the virgin Eve. And thus, as the human race fell into bondage to death by means of a virgin, so is it rescued by a virgin; virginal disobedience having been balanced in the opposite scale by virginal obedience.
Everyone agrees that the virgin Eve did not remain in that state. Yet, people skip that application!
Moreover Irenaeus explains further the nature of the birth. He contrasts those who believe, like the PoJ teaching, that Jesus took nothing from Mary, merely passing through her without afterbirth and umbilical cord with those who like Irenaeus, Clement of Alexandria, Tertullian, Africanus, and others, believed in a normal birth with a normal human birth.
He asks, identifying "of Mary", not "of the ever-virgin",
Why, then, did not God again take dust, but wrought so that the formation should be made of Mary? It was that there might not be another formation called into being, nor any other which should [require to] be saved, but that the very same formation should be summed up [in Christ as had existed in Adam], the analogy having been preserved.
- Those, therefore, who allege that He took nothing from the Virgin do greatly err, [since,] in order that they may cast away the inheritance of the flesh, they also reject the analogy [between Him and Adam]. ... For if He did not receive the substance of flesh from a human being, He neither was made man nor the Son of man; and if He was not made what we were, He did no great thing in what He suffered and endured. ... But Eve was disobedient; for she did not obey when as yet she was a virgin. ... And thus also it was that the knot of Eve’s disobedience was loosed by the obedience of Mary. For what the virgin Eve had bound fast through unbelief, this did the virgin Mary set free through faith.
People wrongly take the same words applied to both Eve and Mary and then based on today's belief fail to apply what is obvious. Yet does not mean perpetual.
As to other allegations about Polycarp, Justin Martyr, and Ignatius also believing in an ever-virgin motif, please note that no quotations from them are provided that say such beliefs. In other words, there's no proof any of them taught or thought that Jesus was born in any way but by a normal human birth.