There is no source in the bible (or anywhere else that I know of) which states how many other wives Joseph had. There are many a scripture which talk about Jesus's 'brothers. (Mt 12:46, Mk 3:31, Lk 8:19, Mk 6:3) which is troublesome, but the notes in my bible read thus:
In semitic usage the terms 'brother' and 'sister' are applied not only to children of the same parents, but to nephews, nieces, cousins, half-brothers and half sisters; cf Gn 14:16;29:15.
While one cannot suppose that the meaning of a greek word should be sought in the first place from semitic usage, the Septuagint often translates the hebrew 'āh by the Greek word adelphos, "brother," as in the cited passages, a fact that may argue for the similar breadth of meaning in some new testament passages. For instance, there is no doubt that in [mark 6:17] , "brother' is used of Philip, who was actually the half-brother of Herod Antipas. On the other hand, mark may have understood the terms Literally; see also Mt 3:31-32, 12:46, 13:55-56, Lk 8:19; Jn 7:3, 5 The question of meaning here would not have arisen but for the faith of the church in Mary's perpetual virginity.
That passage is taken from a note on the text of Mk 6:21 (NAB)
Other then biblical commentary, I know that since life didn't last very long 2000 years ago, it was very possible for Joseph to have had a wife or two before Mary --its entirely plausible that they simply died before the beginning of our Gospels, and the children of these women end up living in the same House or community as Jesus.
This way, we can reconcile the text of 'brothers' with the Perpetual Virginity of Mary (which not every christian acknowledges...)
Also, I am pretty sure I read somewhere that Augustine held an opinion similar to the above. Just saying...