Here is a good overview on how the canon of the Tanakh and the canon of the New Testament came to be. It is by Dr. K. Wheeler of Carson-Newman University. Dr. Wheeler's web site is found here.
The "Old Testament"
In brief, the three-fold division of the Tanakh (i.e., the Torah, the Prophets, and the Writings) were likely "canonized" at different times in Judaism's history. According to Dr. Wheeler,
(1) The Torah (Law) appeared as a collection possibly as early as the Babylonian captivity. It was probably accepted as authoritative sometime after the Babylonian exile, perhaps during the time of Ezra (5th century BCE). For the Sadducees and the Samaritans, the canonization process stopped here. They argued that none of the later additions to the Hebrew Bible were legitimate.
(2) The Prophets, for the Hebrew editors, existed as a collection by at least the second century BCE. The apocryphal book of Ecclesiasticus, the Wisdom of Jesus, the Son of Sirach, had a prologue attached to the work. Sirach's grandson probably did this about 132 BCE. He refers to the "the law and the prophets" and the "others" or the "other books." (See "Prologue," Sirach, RSV Apocrypha, p. 128). This is the earliest evidence we have that the first two parts of the Old Testament Canon had been linked together and anthologized and that the first half of the Canon was on its way to becoming accepted as authoritative.
(3) The Writings . . . [at] the time of Jesus and the early church . . . [were seen by] the Jews . . . as authoritative. Luke 24:44 and Matthew 23:35 contain statements strongly indicating the authors already had some vague idea of an authoritative collection--one covering the range from Genesis to Chronicles, the entire range of Old Testament Scripture. Hence, the Old Testament or the Hebrew Bible had gained recognition as official by 90 CE, and it would receive official sanction in the Council of Jamnia . . . sometime around 95 CE, perhaps twenty years after the Romans destroyed Jerusalem during the Jewish-Roman War. [This] council of exiled Jewish rabbis met in Western Palestine. The leader of this council was Rabbi Akiba (Jaroslav Pelikan. Whose Bible Is It?: A History of the Scriptures Through the Ages. New York: Penguin Books, 2005, p.46). The council officially "closed" the canon of the Hebrew Bible, intending for no new works to be added.
The New Testament
Dr. Wheeler cites Metzger and Coogan's four stages in the completion of the New Testament canon (The Oxford Companion to the Bible. Oxford, Oxford U P, 1993. pp. 101-04), which are as follows:
(1) During the first phase (c. 90 CE-96 CE), the young Christian church discovered that Christ had not yet returned as predicted, but the generation of the apostles and those who had directly encountered Christ was rapidly dying out. Originally, knowledge of Christ had been transmitted orally. At this point, the church fathers wrote down the early Gospels and preserved the Pauline letters as ways to give pastoral guidance to new churches.
(2) From 96 CE until about 150 CE, the written gospels gradually replace the oral traditions. Initially, early Christians relied on oral tradition alongside (and even in preference to) the written Gospels, but as the reliability of the former declined, the four Gospels replaced them. . . . In Polycarp's letter to the Philippians (c. 135 CE), for the first time we see the argument made that the Gospel texts are more reliable than word of mouth, and more to be trusted. In 2 Clement (c. 140 CE), twice as many quotations come from the written Gospels than from any other source . . ..
After 150 CE, the written Gospels have clearly become predominant over word-of-mouth transmission. By the time of the Apocryphal Gospel of Truth, the author clearly knows all four of the Orthodox Gospels, but only uncertain, faded remnants persist of alternative oral traditions. About this time, the Pauline correspondence begins to appear in a collected form rather than as scattered, independent letters. The fact that these texts all have Paul's name attached to them by tradition help them survive as a group when anonymous texts tend to fall by the wayside.
(3) From 150 CE to 190 CE, the canon wars begin in earnest. The earliest attempt at a canonical collection (or at least the first writer to actually visualize the idea of a New Testament collection to match the canonicity of the old Testament) actually comes from a heretical author. The Gnostic Christian, Marcion, creates an "official" canon. This new Bible deliberately excludes the Old Testament, and the New Testament is limited to Luke and ten Pauline letters (from which Marcion removed certain Jewish traits). In reaction, the orthodox church emphasized the normative qualities of all four Gospels and all thirteen letters ascribed to Paul, but their version did not include . . . Revelation . . .. Bishop Irenaeus reflects this stance in his writings by excluding Revelation, but he broadens his idea of the Canon to include all of the other current New Testament books and he also includes the Wisdom of Hermas as an additional text. Irenaeus also reacts positively to 1 Clement and the Wisdom of Solomon, but it is not clear if he regarded them as "holy Scripture." Clearly, the idea of the canon has appeared, but the church cannot yet agree as to what parts should go into this canon. What is holy Scripture? What is not? A broad base of consensus exists for a core collection (Luke, certain Pauline epistles), but the peripheral works remain contested.
(4) The period from about 190-400 CE is the time Andrie Du Tott calls "The Closing of the Canon." During these centuries, many heretical groups had prophets who declared they had received new revelations from God. The Montanists, in particular, called for a clear limitation of the canon since their sect stressed apocalyptic and prophetic movements. Origin and Eusebius in the Eastern church and an anonymous author of the Canon of Muratori together argued that the scriptures should be classed into the three groups: 1) the texts nearly all Christians accept; 2) the uncertain texts about which Christians are mostly undecided; and 3) works that should definitely be excluded because only small, isolated groups of often heretical Christians hold them to be scripture.
It is during this time that readers appear to start thinking of the "Bible" as a single work. Note that the title Biblia in Greek is originally neuter plural, meaning "little books." That very title harkens back to the older time period when Christians thought of their text as the Scriptures (plural--multiple independent entities) rather than the Scripture (singular--one unified entity).