It is well known that the most important position in the Catholic church is the pope, but how did this position arise? I know that Peter was told by Jesus that he was "The rock upon which He will build His church", but the Scriptures seem to lack the specific commandment for the duties of the pope, or even the existence of the papacy or the Vatican. I am unable to find scripture referring to a pope-like figure or the papacy.
There is a possible biblical basis in Matthew's Gospel, although the wider biblical record is mixed:
Matthew 16:13-18: When Jesus went into the region of Caesarea Philippi 9 he asked his disciples, "Who do people say that the Son of Man is?" They replied, "Some say John the Baptist, others Elijah, still others Jeremiah or one of the prophets." He said to them, "But who do you say that I am?" Simon Peter said in reply, "You are the Messiah, the Son of the living God." Jesus said to him in reply, "Blessed are you, Simon son of Jonah. For flesh and blood has not revealed this to you, but my heavenly Father. And so I say to you, you are Peter, and upon this rock I will build my church, and the gates of the netherworld shall not prevail against it.
John Dominic Crossan says, in The Birth of Christianity, page 109, that there is "a fairly massive (but by no means total) consensus of contemporary critical scholarship holds that much of Matthew is copied from Mark's Gospel", but when we look at Mark 8:27-30, there is no mention of Peter as the rock, leaving open the suspicion that this is a literary elaboration on the part of Matthew's author. The First and Second Epistles of Peter make no mention of this commendation, although the authors make every effort to demonstrate Peter's authority and any reference to Jesus nominating Peter as the rock on which he would build his church would add greatly to the authority of the epistles' message. 1 Corinthians 10:4 says that Jesus is the rock.
Wikipedia explains "The Catholic Church teaches that, within the Christian community, the bishops as a body have succeeded to the body of the apostles (apostolic succession) and the Bishop of Rome has succeeded to Saint Peter." Wikipedia goes on to say, "Some historians have argued that the notion that Peter was the first bishop of Rome and founded the episcopal see there can be traced back no earlier than the 3rd century."
Apostolic succession is a fundamental principle, as the authority of the pope rests not only on the claim that Peter went to Rome and led the church there, but on the further claim that there is an unbroken chain of successors in Rome, from Peter to the present day. Tradition says that Linus took office as bishop of Rome while Peter was still alive, serving from 55 to 67.
John W. O’Malley, S.J. says, in A History of the Popes, page 8, that there is no single piece of evidence that tells us unambiguously that Peter ever went to Rome. He cites Peter's reference to 'Babylon' in 1 Peter 5:13 as potential evidence for Peter's presence in Rome, because 'Babylon' became a Christian euphemism for Rome. Although Fr. O’Malley acknowledges that Peter may not have been the real author of this epistle, he believes that Peter was its inspiration, yet overlooks that any reference to ‘Babylon’ is an anachronism before 70 CE.
Francis A. Sullivan SJ says, in From Apostles to Bishops, page 15:
There exists a broad consensus among scholars, including most Catholic ones, that such churches as those of Alexandria, Philippi, Corinth and Rome most probably continued to be led for some time by a college of presbyters, and that only during the course of the second century did the threefold structure become generally the rule, with a bishop, assisted by presbyters, presiding over each local church.
The research of O'Malley and Sullivan suggests that, although the role of Pope is firmly rooted in tradition of apostolic succession from Peter, the 'rock' of the church, this is not really the case if only because there was no monepiscopal bishop of Rome until around the middle of the second century.
The section "Of the Institution of the Apostolic Primacy in blessed Peter" of the First Vatican Council's Dogmatic Constitution on the Church of Christ (Pastor Æternus) quotes these Scripture verses:
We therefore teach and declare that, according to the testimony of the Gospel, the primacy of jurisdiction over the universal Church of God was immediately and directly promised and given to blessed Peter the Apostle by Christ the Lord. For it was to Simon alone, to whom he had already said: 'Thou shalt be called Cephas,' [St. John 1:42] that the Lord after the confession made by him, saying: 'Thou art the Christ, the Son of the living God,' [St. Matthew 16:16] addressed these solemn words: 'Blessed art thou, Simon Bar-Jona, because flesh and blood have not revealed it to thee, but my Father who is in heaven. And I say to thee that thou art Peter; and upon this rock I will build my Church, and the gates of hell shall not prevail against it. And I will give to thee the keys of the kingdom of heaven. And whatsoever thou shalt bind on earth, it shall be bound also in heaven; and whatsoever thou shalt loose on earth, it shall be loosed also in heaven.'[St. Matthew 16:17-19] And it was upon Simon alone that Jesus after his resurrection bestowed the jurisdiction of chief pastor and ruler over all his fold in the words: 'Feed my lambs; feed my sheep.' [St. John 21:15-17] At open variance with this clear doctrine of Holy Scripture as it has been ever understood by the Catholic Church are the perverse opinions of those who, while they distort the form of government established by Christ the Lord in his Church, deny that Peter in his single person, preferably to all the other Apostles, whether taken separately or together, was endowed by Christ with a true and proper primacy of jurisdiction; or of those who assert that the same primacy was not bestowed immediately and directly upon blessed Peter himself, but upon the Church, and through the Church on Peter as her minister.
St. Robert Cardinal Bellarmine, the greatest Doctor of ecclesiology, wrote a De Romano Pontifice or On the Roman Pontiff (part of his De Controversiis or On the Controversies series) in which he shows—in more detail, against the errors and heresies of Calvin, Luther, et al.—how the above scriptural verses support the papacy.