To begin, let's be sure that we understand what double predestination means in Calvinism. You have it right, that God chooses both who is elect and who is non-elect, but it's important to note that in Calvinist double predestination, God actively saves the elect, but passively passes over the reprobate:
God positively or actively intervenes in the lives of the elect to insure their salvation. The rest of mankind God leaves to themselves. He does not create unbelief in their hearts.
This quote comes from Chosen by God, chapter 7, by R. C. Sproul, a leading Calvinist theologian. This book is a well-known introduction to Calvinism, particularly the difficult issues of God's sovereignty, original sin, predestination, and assurance, and I'll continue to refer to it below.
So what is the basis for this view? First R. C. Sproul argues that double predestination is logically necessary:
If there is such a thing as predestination at all, and if that predestination does not include all people, then we must not shrink from the necessary inference that there are two sides to predestination. It is not enough to talk about Jacob; we must also consider Esau.
Here Sproul is referring to the story of Jacob and Esau, beginning in Genesis 25. This opens up the second line of reasoning – that double predestination is taught in Scripture, particularly in Romans 9:10–18:
when Rebekah had conceived children by one man, our forefather Isaac, 11 though they were not yet born and had done nothing either good or bad—in order that God's purpose of election might continue, not because of works but because of him who calls— 12 she was told, “The older will serve the younger.” 13 As it is written, “Jacob I loved, but Esau I hated.”
14 What shall we say then? Is there injustice on God's part? By no means! 15 For he says to Moses, “I will have mercy on whom I have mercy, and I will have compassion on whom I have compassion.” 16 So then it depends not on human will or exertion, but on God, who has mercy. 17 For the Scripture says to Pharaoh, “For this very purpose I have raised you up, that I might show my power in you, and that my name might be proclaimed in all the earth.” 18 So then he has mercy on whomever he wills, and he hardens whomever he wills. (ESV)
Sproul summarizes, pointing to verse 18:
Some people get mercy, others get justice. The decision for this is in the hand of God.
A few more verses in Romans 9, verses 21–24, "clearly" teach double predestination, in Sproul's eyes:
Has the potter no right over the clay, to make out of the same lump one vessel for honorable use and another for dishonorable use? 22 What if God, desiring to show his wrath and to make known his power, has endured with much patience vessels of wrath prepared for destruction, 23 in order to make known the riches of his glory for vessels of mercy, which he has prepared beforehand for glory— 24 even us whom he has called, not from the Jews only but also from the Gentiles? (ESV)
To summarize, Sproul argues that double predestination is 1) logically necessary, if we accept that only some are elect, and 2) biblical, particularly in light of Romans 9.