Why couldn't the disciples begin following the great commission right after Jesus ascended into heaven? Why was the sending of the Holy Spirit necessary?
The disciples could have carried out the Great Commission immediately, but they had already received word from the Lord that they would receive the Holy Spirit prior to this. Jesus had commanded that they wait in Jerusalem until then.
Act 1:4,6-8 KJV - And, being assembled together with [them], commanded them that they should not depart from Jerusalem, but wait for the promise of the Father,... When they therefore were come together, they asked of him, saying, Lord, wilt thou at this time restore again the kingdom to Israel? And he said unto them, It is not for you to know the times or the seasons, which the Father hath put in his own power. But ye shall receive power, after that the Holy Ghost is come upon you: and ye shall be witnesses unto me both in Jerusalem, and in all Judaea, and in Samaria, and unto the uttermost part of the earth.
This passage and the Great Commission of Matthew 28 are invariably linked: They both involve going to all the world with an evangelistic mission. While Matthew doesn't record Jesus mentioning the Holy Spirit like this before his ascension, these additional words of Jesus from around the same time in Acts (after appearing to his disciples and before He ascends) makes the connection to the Commission clear. The apostles already had experience going out on their own and bringing the Message with power of the Holy Ghost (Mark 6:12-13), but they knew to wait, or else Jesus wouldn't have given that timeline: The Holy Ghost comes upon you, You receive power, You will be My witnesses. This is why it was necessary that they wait before following the Great Commission. The second part of your question is why the Holy Spirit is necessary.
The Gospel's effectiveness doesn't come with persuasion, but with power.
1Co 1:17-18 KJV - For Christ sent me not to baptize, but to preach the gospel: not with wisdom of words, lest the cross of Christ should be made of none effect. For the preaching of the cross is to them that perish foolishness; but unto us which are saved it is the power of God.
1Co 2:4-6 KJV - And my speech and my preaching [was] not with enticing words of man's wisdom, but in demonstration of the Spirit and of power: That your faith should not stand in the wisdom of men, but in the power of God.
Without the power of the Holy Spirit blowing, there can be no conversion:
Jhn 3:6-8 KJV - That which is born of the flesh is flesh; and that which is born of the Spirit is spirit. Marvel not that I said unto thee, Ye must be born again. The wind bloweth where it listeth, and thou hearest the sound thereof, but canst not tell whence it cometh, and whither it goeth: so is every one that is born of the Spirit.
I suppose you could keep asking why and way, but at the end of the day, this is just how God works. Were it not for the Holy Spirit, there could be no effective evangelism. There is active agency on His part that is more important than the actual words used.
The mission of the Son and the Spirit, is a joint one, consistent with the view that all the works of God are the works of his hands.
CCC 292 The Old Testament suggests and the New Covenant reveals the creative action of the Son and the Spirit,1 inseparably one with that of the Father. This creative co-operation is clearly affirmed in the Church's rule of faith: "There exists but one God. . . he is the Father, God, the Creator, the author, the giver of order. He made all things by himself, that is, by his Word and by his Wisdom", "by the Son and the Spirit" who, so to speak, are "his hands".2 Creation is the common work of the Holy Trinity.
1. cf. Ps 33:6; 104:30; Gen 1:2-3.
2. St. Irenaeus, Adv. haeres. 2,30,9; 4,20,I: PG 7/1,822,1032.
cf. The Work of the Spirit [cf. Jn 16:4-15 (RSVCE)].
Specifically v. 7
7 Nevertheless I tell you the truth: it is to your advantage that I go away, for if I do not go away, the Counselor will not come to you; but if I go, I will send him to you.
(Which has scriptural references)
cf. I. THE JOINT MISSION OF THE SON AND THE SPIRIT (sans footnotes) in the Catechism of the Catholic Church
689 The One whom the Father has sent into our hearts, the Spirit of his Son, is truly God. Consubstantial with the Father and the Son, the Spirit is inseparable from them, in both the inner life of the Trinity and his gift of love for the world. In adoring the Holy Trinity, life-giving, consubstantial, and indivisible, the Church's faith also professes the distinction of persons. When the Father sends his Word, he always sends his Breath. In their joint mission, the Son and the Holy Spirit are distinct but inseparable. To be sure, it is Christ who is seen, the visible image of the invisible God, but it is the Spirit who reveals him.
690 Jesus is Christ, "anointed," because the Spirit is his anointing, and everything that occurs from the Incarnation on derives from this fullness. When Christ is finally glorified, he can in turn send the Spirit from his place with the Father to those who believe in him: he communicates to them his glory, that is, the Holy Spirit who glorifies him. From that time on, this joint mission will be manifested in the children adopted by the Father in the Body of his Son: the mission of the Spirit of adoption is to unite them to Christ and make them live in him:
The notion of anointing suggests . . . that there is no distance between the Son and the Spirit. Indeed, just as between the surface of the body and the anointing with oil neither reason nor sensation recognizes any intermediary, so the contact of the Son with the Spirit is immediate, so that anyone who would make contact with the Son by faith must first encounter the oil by contact. In fact there is no part that is not covered by the Holy Spirit. That is why the confession of the Son's Lordship is made in the Holy Spirit by those who receive him, the Spirit coming from all sides to those who approach the Son in faith.
CCC 485 The mission of the Holy Spirit is always conjoined and ordered to that of the Son.1 The Holy Spirit, "the Lord, the giver of Life", is sent to sanctify the womb of the Virgin Mary and divinely fecundate it, causing her to conceive the eternal Son of the Father in a humanity drawn from her own.
1. cf. Jn 16:14-15.
The importance of sending the Holy Spirit appears to have been given by Jesus himself in:
John 14:26 But the Comforter, which is the Holy Ghost, whom the Father will send in my name, he shall teach you all things, and bring all things to your remembrance, whatsoever I have said unto you.
Being human they like we would not have completely remembered or understood all that Jesus said to them, and therefore the Holy Spirit had a mission to remind them, and teach them the meaning of those sayings.