The mention of Mary does not seem to have come in the books against Celcus, however, Celcus did address the symbol of the Crucifix, not as a means of idolatry. Origen swiftly and in great detail every argument and misconception of Celcus, who seems to be a Gnostic from the mentions of archons. Origen mocked the pagan gods and idols, including the virgins who stand by them. In Contra Celsus BOOK VII.CHAP. XLVIII
And attached to the other so-called gods are a select number of virgins, who are guarded by men, or it may be not guarded (for that is not the point in question at present), and who are supposed to live in purity for the honour of the god they serve. But among Christians, those who maintain a perpetual virginity do so for no human honours, for no fee or reward, from no motive of vainglory; but "as they choose to retain God in their knowledge," they are preserved by God in a spirit well-pleasing to Him, and in the discharge of every duty, being filled with all righteousness and goodness.
Regarding the Cross: Celcus argued that the obsession of Christians with the cross (tree) could be merely incidentally for various reasons, including that perhaps their Master was a carpenter by profession. Origen refuted him by describing the symbolism of the tree of life from the Garden of Eden to the tree of Jesus, and that Jesus was never described as a carpenter in any of the Gospels (Mark 6:3 variant, Matt 13:55). The accusation was not about Christians committing idolatry, but it was merely the reference of the talk about the tree.
Search for the "tree" for its references here book VI:
CHAP. XXXVI. He next scoffs at the "tree," assailing it on two grounds, and saying, "For this reason is the tree introduced, either because our teacher was nailed to a cross, or because he was a carpenter by trade;" not observing that the tree of life is mentioned in the Mosaic writings, and being blind also to this, that in none of the Gospels current in the Churches is Jesus Himself ever described as being a carpenter.
XXXVII. Celsus, moreover, thinks that we have invented this "tree of life" to give an allegorical meaning to the cross; and in consequence of his error upon this point, he adds: "If he had happened to be cast down a precipice, or shoved into a pit, or suffocated by hanging, there would have been invented a precipice of life far beyond the heavens, or a pit of resurrection, or a cord of immortality." And again: "If the 'tree of life' were an invention, because he--Jesus-- (is reported) to have been a carpenter, it would follow that if he had been a leather-cutter, something would have been said about holy leather; or had he been a stone-cutter, about a blessed stone; or if a worker in iron, about an iron of love."
If Celcus defended his or others' idolatry by saying that they were merely means to connect with the divine, then it should not be a surprise as every single idolater justifies his idolatry by the same rhetoric, that they are not worshipping their idols. Be it the Muslims who venerate their holy Kaba stone for forgiveness of their sins, or the Hindus who have various idol gods, all give the same reasoning. If Celcus was aware of any idolatry by Christians, he would've called out their hypocrisy. Every offensive and abusive charge however, was based on his misconceptions where he may have been describing the evil Gnostic-Christians practices, not the real Christians.
After the Image of God Vs God's Image CHAP. LXIII.
Celsus, not observing the difference between "after the image of God" and "God's image," next asserts that the "first-born of every creature" is the image of God,--the very word and truth, and also the very wisdom, being the image of His goodness, while man has been created after the image of God; moreover, that every man whose head is Christ is the image and glory of God;--and further, not observing to which of the characteristics of humanity the expression "after the image of God" belongs, and that it consists in a nature which never had nor longer has "the old man with his deeds," being called "after the image of Him who created it," from its not possessing these qualities,--he maintains: "Neither did He make man His image; for God is not such an one, nor like any other species of (visible) being." Is it possible to suppose that the element which is "after the image of God" should exist in the inferior part--I mean the body--of a compound being like man, because Celsus has explained that to be made after the image of God? For if that which is "after the image of God" be in the body only, the better part, the soul, has been deprived of that which is "after His image," and this (distinction) exists in the corruptible body,--an assertion which is made by none of us. But if that which is "after the image of God" be in both together, then God must necessarily be a compound being, and consist, as it were, of soul and body, in order that the element which is "after God's image," the better part, may be in the soul; while the inferior part, and that which "is according to the body," may be in the body,--an assertion, again, which is made by none of us. It remains, therefore, that that which is "after the image of God" must be understood to be in our "inner man," which is also renewed, and whose nature it is to be "after the image of Him who created it," when a man becomes "perfect," as "our Father in heaven is perfect," and hears the command, "Be ye holy, for I the LORD your God am holy," and learning the precept, "Be ye followers of God," receives into his virtuous soul the traits of God's image. The body, moreover, of him who possesses such a soul is a temple of God; and in the soul God dwells, because it has been made after His image.