they drank from the spiritual rock that followed them
This follows immediately after:
all drank the same spiritual drink
which itself follows after:
all ate the same spiritual food
which is preceded by:
all were baptized into Moses in the cloud and in the sea
Now, if the latter is to be taken as a figure of (Christian) baptism, then the two in the middle are most likely a type of the Eucharist (John 6 : 31-35, 41-42, 48-58); in which case, whether the water stemming from the rock is meant to symbolize baptism (in Christ's name : see Matthew 28:19; John 4:1-2; Acts 2:38, 8:12-16, 19:4-5; Romans 6:3-4; 1 Corinthians 1:12-15; Galatians 3:27) or the water and blood that poured forth from his side at the crucifixion on Golgotha (John 19:34; 1 John 5:6-8), it stands to reason that, even from this perspective alone, the rock in question is a symbol of Christ.
at the time of the Exodus, one would hear of Yahweh
Furthermore, throughout both Testaments, various metaphors expressing solidity and stability are applied to God Himself, one of them being the word rock (1, 2); then, since sons resemble their fathers, the same should logically hold for Christ as well, since he is the son of God (Matthew 16:15-18). In fact, let us analyze the latter passage a bit: Why is Peter called a rock in the first place ? Because of the profession of faith coming from his lips. That profession of faith constitutes the (bed)rock of the entire Christian faith, which is why the person uttering it was granted that specific title or surname to begin with. Thus, to summarize, God the Father is the ultimate Rock; Christ, his only-begotten son, is then also a Rock, since He was born of the Father; faith in Him, by extension, is also a rock; as is the person uttering this statement, Simon.
Chrysostom wrote a commentary on almost every New Testament book, including Corinthians, so it would only be common sense to consult him on this matter as well:
(Note that, in the Christian East, sacraments are called mysteries).
And wherefore says he these things? To point out that as they were nothing profited by the enjoyment of so great a gift, so neither these by obtaining Baptism and partaking of spiritual Mysteries [Sacraments], except they go on and show forth a life worthy of this grace. Wherefore also he introduces the types both of Baptism and of the Mysteries [Sacraments].
But what is, "They were baptized into Moses?" Like as we, on our belief in Christ and His resurrection, are baptized, as being destined in our own persons to partake in the same mysteries [sacraments]; for, "we are baptized," says he, "for the dead," i.e., for our own bodies; even so they putting confidence in Moses, i.e., having seen him cross first, ventured also themselves into the waters. But because he wishes to bring the Type near the Truth; he speaks it not thus, but uses the terms of the Truth even concerning the Type.
Further: this was a symbol of the Font, and that which follows, of the Holy Table. For as you eat the Lord's Body, so they the manna: and as you drink the Blood, so they water from a rock. For though they were things of sense which were produced, yet were they spiritually exhibited, not according to the order of nature, but according to the gracious intention of the gift, and together with the body nourished also the soul, leading it unto faith. On this account, you see, touching the food he made no remark, for it was entirely different, not in mode only but in nature also; (for it was manna;) but respecting the drink, since the manner only of the supply was extraordinary and required proof, therefore having said that "they drank the same spiritual drink," he added, "for they drank of a spiritual Rock that followed them," and he subjoined, "and the Rock was Christ." For it was not the nature of the rock which sent forth the water, (such is his meaning,) else would it as well have gushed out before this time: but another sort of Rock, a spiritual One, performed the whole, even Christ who was every where with them and wrought all the wonders. For on this account he said, "that followed them."
Do you perceive the wisdom of Paul, how in both cases he points cut Him as the Giver, and thereby brings the Type near to the Truth? "For He who set those things before them," says he, "the same also has prepared this our Table: and the same Person both brought them through the sea and you through Baptism; and before them set manna, but before you His Body and Blood."