In some early church liturgies, and continuing even up to the Middle Ages, there was a custom of offering milk and honey to a newly baptized person as part of their first Eucharist. This took place after the bread and before the wine.1 An example is the liturgy of the Apostolic Tradition of Hippolytus (c. 215), 21:27-30:
Then the deacons shall immediately bring the oblation. The bishop shall bless the bread,
which is the symbol of the Body of Christ; and the bowl of mixed wine, which is the
symbol of the Blood which has been shed for all who believe in him; and the milk and
honey mixed together, in fulfillment of the promise made to the fathers, in which he said,
"a land flowing with milk and honey," which Christ indeed gave, his Flesh, through which
those who believe are nourished like little children, by the sweetness of his Word,
softening the bitter heart; and water also for an oblation, as a sign of the baptism, so
that the inner person, which is psychic, may also receive the same as the body. The
bishop shall give an explanation of all these things to those who are receiving.
Jerome's Dialogue against the Luciferians (c. 379) also talks about this baptismal custom, though he does not embed it within the Eucharist:
For many other observances of the Churches, which are due to tradition, have acquired the authority of the written law, as for instance the practice of dipping the head three times in the laver, and then, after leaving the water, of tasting mingled milk and honey in representation of infancy.
The symbolism of milk and honey, and its biblical resonance, is pretty clear. There is no reference here to having milk instead of wine, and it is meant to be only for the first communion. The existence of this tradition, though, does demonstrate a background where milk was not completely alien to the liturgy.
Clement of Alexandria wrote a long discussion in his Paedogogus (c. 198) I.6, where he takes the view that milk can represent blood, and even the blood of Christ. Part of his reasoning is that in his view of human physiology, milk is basically blood that has been somehow sweetened and purified.2 He also appeals to 1 Corinthians 3:2, in which Paul likens his teaching to milk and meat. Clement continues:
The blood of the Word has been also exhibited as milk. [...] Thus in many ways the Word is figuratively described, as meat, and flesh, and food, and bread, and blood, and milk. The Lord is all these, to give enjoyment to us who have believed on Him. Let no one then think it strange, when we say that the Lord's blood is figuratively represented as milk. For is it not figuratively represented as wine? [...] The same blood and milk of the Lord is therefore the symbol of the Lord's passion and teaching.
He also talks about drinking a mixture of milk and wine (yuck!):
Furthermore, milk is mixed with sweet wine; and the mixture is beneficial, as when suffering is mixed in the cup in order to immortality. For the milk is curdled by the wine, and separated, and whatever adulteration is in it is drained off. And in the same way, the spiritual communion of faith with suffering man, drawing off as serous matter the lusts of the flesh, commits man to eternity, along with those who are divine, immortalizing him.
It seems quite possible that some people, following this teaching of Clement, considered it acceptable to add a milk element to the Eucharist, mix wine and milk together, or to substitute milk for wine completely.
1 Eucharistic doctors: a theological history, Owen F. Cummings (Paulist Press, 2005), p34.
2 Honestly, that entire chapter is pretty startling in a lot of ways. Apparently semen is also blood, that's been whipped into a foam. Thankfully the digression ends there.