What is the Biblical Basis for God the Holy Spirit having a spouse?
This is a metaphor for the Virgin Mary as having been espoused by the Holy Spirit as she became pregnant through the overshadowing of the Paraclete.
The best place to start is at the Annunciation.
26 And in the sixth month, the angel Gabriel was sent from God into a city of Galilee, called Nazareth,
27 To a virgin espoused to a man whose name was Joseph, of the house of David; and the virgin's name was Mary.
28 And the angel being come in, said unto her: Hail, full of grace, the Lord is with thee: blessed art thou among women.
29 Who having heard, was troubled at his saying, and thought with herself what manner of salutation this should be.
30 And the angel said to her: Fear not, Mary, for thou hast found grace with God.
31 Behold thou shalt conceive in thy womb, and shalt bring forth a son; and thou shalt call his name Jesus.
32 He shall be great, and shall be called the Son of the most High; and the Lord God shall give unto him the throne of David his father; and he shall reign in the house of Jacob for ever.
33 And of his kingdom there shall be no end.
34 And Mary said to the angel: How shall this be done, because I know not man?
35 And the angel answering, said to her: The Holy Ghost shall come upon thee, and the power of the most High shall overshadow thee. And therefore also the Holy which shall be born of thee shall be called the Son of God.
36 And behold thy cousin Elizabeth, she also hath conceived a son in her old age; and this is the sixth month with her that is called barren:
37 Because no word shall be impossible with God.
38 And Mary said: Behold the handmaid of the Lord; be it done to me according to thy word. And the angel departed from her. - Luke 1:26-38
Catholic Answers has this little bit to add:
Was the Blessed Virgin Mary Joseph’s spouse or the Holy Spirit’s spouse?
Answer:
St. Joseph was the husband of Mary in the common usage of the idea of marriage.
The pious custom of referring to the Holy Spirit as the spouse of Mary is a symbolic expression of Mary’s perpetual virginity and the virgin birth of Jesus. It is not meant in a literal manner but rather in terms of Mary’s singular devotion to God and unique relationship to the Trinity. It is similar to how religious sisters sometimes refer to Jesus as their spouse.
Scripture contains several examples of a mystical spousal relationship:
For your Maker is your husband, the LORD of hosts is his name . . . (Isaiah 54:5).
. . . as the bridegroom rejoices over the bride, so shall your God rejoice over you (Isaiah 62:5).
. . . my covenant which they broke, though I was their husband, says the LORD (Jeremiah 31:32).
I feel a divine jealousy for you, for I betrothed you to Christ to present you as a pure bride to her one husband (2 Cor. 11:2).
The following may be of interest to some: