First, it should be noted that even if Jews did not view the passage as Messianic before Jesus, that does not prove that it is a invalid interpretation. In Old Testament times, Biblical prophecies were often not recognized until they were fulfilled. That said, let's look at the evidence.
Targum Jonathan ben Uziel
The Targums are interpretative translations of the original Hebrew into Aramaic. They were authorized for reading within the Synagogue - someone would read the Hebrew original and then someone else would recite the Targum from memory. Since they are not literal translations, they can provide valuable insight into the thinking of orthodox Jews during the period in which they were written
There is a general consensus that the Targum Jonathan, also known as the Targum Isaiah, began to be developed roughly contemporaneously with the time of Jesus. It probably received the bulk of its editing during the second century, and was completed (standardized) by the forth century. Regardless of the precise date, it is the oldest extant information on non-Christian Jewish thought of Isaiah 52:13-53:12, other than the passage itself, of course.
Jonathan translates 52:12 as follows:
Behold my servant Messiah shall prosper; he shall be high, and increase, and be exceeding strong: (Pauli's translation)
comparing this to straight translation of the original Hebrew:
Behold, my servant shall act wisely; he shall be high and lifted up, and shall be exalted. (ESV)
it is obvious the Jonathan has added "Messiah" to the passage. (See also 53:10.) Jewish apologists note that Jonathan reinterprets the rest of the passage to de-emphasize the suffering aspects, or transfer them from the servant to the community he is rescuing. For example, the
But he was pierced for our transgressions; he was crushed for our iniquities; upon him was the chastisement that brought us peace, and with his wounds we are healed.
of 53:5 becomes:
Therefore He shall pray for our sins, and our iniquities for His sake shall be forgiven; for we are considered crushed, smitten of the Lord, and afflicted.
Indeed, if Targum Jonathan was the original, the passage would not bode well for the Christian interpretation. However, it is not the original, and the fact remains that Jonathan saw the passage as Messianic and that this interpretation was mainstream enough to be read in the Synagogue.
There are two basic possibilities here. If the Targum was written before Jesus, or otherwise independent to Christian thought, then Jonathan was reinterpreting the passage due to difficulties in resolving it with other Messianic passages that expected a triumphant Messiah. This reinterpretation, naturally, would only be necessary if the passage was already seen as Messianic by a significant portion of the Jewish community.
The second possibility is that Jonathan wrote (indirectly) in response to Christian use of the passage. If true, this actually strengthens the Christian's case. The Targum is then implying "Jesus didn't fulfill this passage," but definitely not "this is not a Messianic passage." The fact that the Messianic reading remained despite the Christian use of it would strongly imply that many or even most Jews saw it as Messianic before Jesus came. If that wasn't the case, it would have been much simpler and more effective to leave out the Messianic overtones than to reinterpret the passage to be a different kind of Messiah than what Christians said Jesus was.
New Testament usage
Acts 8:26-40 contains the story of how Philip converted an Ethiopian eunuch. In the passage, the eunuch is reading a portion of Isaiah 53 which he doesn't understand and asks Philip to help him interpret it. By itself, this passage only shows that early Christians interpreted the passage as Messianic. This could be the invention of Philip or the author of Acts, and does not necessarily reflect a pre-existent Jewish interpretation. A number of other NT passages also make an explicit comparison.
What I find more interesting is a large group of passages that scholars have suggested were influenced by the language of Isaiah 53 without quoting or even alluding to it directly. According to Simon Gathercole "There is virtually a scholarly consensus now that Paul's letters were influenced by Isaiah 53."1 Passages cited include Romans 4:25 (perhaps rising to the level of clear allusion), Romans 8:32, and 1 Corinthians 15. J. Christopher Edwards sees a connection between Isaiah 53 and Mark 10:45; Matthew 20:28.2 Craig Evans sees Isaiah's influence in Peter's speeches of Acts 3 and Acts 10, as well in several passages of 1 Peter.3 Daniel J. Brendsel examines John's use of Isaiah 53 and concludes that the direct quote in John 12:38 is just "the tip of the iceberg" of its influence on John's writing.4
The fact that Isaiah 53 is used by NT writers not only in arguments, but also influences numerous passages, written by a diverse set of writers, suggests that the idea was very deeply embedded within Christian thought from a very early date. If the idea was the invention of Christians, one might expect it to be developed over time, but what we actually see is the motif in every strata of the data - it is in the words of Jesus, it is in the Letters of Paul, it is in Mark, in is in the other Synoptics, it is in John, and it is in the catholic letters - all without noticeable development. This suggest the idea either heavily influenced Jesus and thus was a common theme of his teaching, or it was an already developed idea in Jewish thought before the first century - or both.
Rabbinic writings
Talmud
The next source (chronologically) of potential information is the Talmud. Recorded in the 3rd-5th centuries, it often records thoughts that date back further than that. There are five references to Isaiah 53 in the Talmud. Three are unrelated to the suffering servant image.[Note A] The first of the remaining two is Sotah 14a, which reads:
R. Simlai expounded: Why did Moses our teacher yearn to enter the land of Israel? ... because he poured out his soul unto death, and was numbered with the transgressors; yet he bare the sins of many, and made intercession for the transgressors
Here, through a reference to Isaiah 53:12, Rabbi Simlai ties the suffering servant to Moses. In another portion of the Talmud, Sanhedrin 98b, we read:
What is his [the Messiah's] name? ... The Rabbis said: His name is 'the leper scholar,' as it is written, Surely he hath borne our griefs, and carried our sorrows: yet we did esteem him a leper, smitten of God, and afflicted.
Here, an unidentified group of rabbis tie the suffering servant to the Messiah through Isaiah 53:4. (Various other rabbis tie other passages to the Messiah is the same passage.)
Midrash Rabbah
The next work of note is the Midrash Rabbah, a collection of Midrash mostly written between the 5th and 8th centuries.
In Ruth Rabbah, written sometime after the 6th century, we find six "interpretations" of Ruth 2:14 by Rabbi Jonathan.5 The six interpretations list the decedents of Ruth & Boaz: David, Solomon, Hezekiah, Menashah, and then:
The fifth interpretation makes it refer to the Messiah... AND DIP YOUR MORSEL IN THE VINEGAR refers to his sufferings, as it is said, "But he was wounded because of our transgressions"
Here, Rabbi Jonathan desired a way to tie Ruth to the coming Messiah and chose 53:5 to make the connection. If the suffering servant passage had no Messianic connection, Jonathan would have picked a different verse to make the connection.
In Deuteronomy Rabbah, dating to the eighth century, there is a passage in which Moses, starting with Deuteronomy 3:8 as the basis for his speech, says to God:
You said to me, ‘You shall not pass over.’ Therefore ‘it is all the same so I said...’ After all this I was ‘cut off out of the land of the living.’ You decreed on me that I should not enter the land.
Here, Moses is made to compare himself to the suffering servant via Isaiah 53:8.
There is a third passage, found in Numbers Rabbah. This work's final form dates to the 11th or 12th century, but it probably incorporates older sources. Commenting on Numbers 13:2, the Rabbi writes:6
I have eaten my honeycomb with my honey: because the Israelites poured out their soul to die in the captivity, as it is said, ‘Because he poured out his soul to die.’
Here, the commentator ties Israel during the exile (presumably the Babylonian one) to the suffering servant via Isaiah 53:5.
So, from the Midrash Rabbah, we have three passage - one that compares the suffering servant to the Messiah, one that compares it to Moses, and one (comparatively late) passage that compares it to Israel during the exile.
Jewish objections
In response to the above data, Jewish commentators offer two critiques. First, they say Midrash is a type of story telling and shouldn't be read literally. Second, they note all these examples post-date Jesus.
It is certainly true that caution should be used when interpreting Midrash. For example, it certainly would be a mistake to say Ruth Rabbah itself is a prophecy. On the other hand, this objection doesn't really take away the force of the Christian's argument either. Sticking with the same example, it is not Jonathan's purpose to explain Isaiah in his "story" about Ruth's decedents. But, the interpretation loses all meaning if we say he picked a random verse to justify his Messianic line. If Isaiah 53 never had any Messianic meaning for any Jew, there would be no reason for him to even consider the verse he uses - it's not like it is an inherently close match to Ruth 2:14. He picks the verse for a reason.
The second objection has even less force. Of course all these passages post-date Jesus. There wasn't any written Jewish commentary before the Talmud began to be written down. It seems very unlikely that Judaism would begin to see Isaiah 53 as Messianic in response to Christian claims that it is, which is the only way the late date would be relevant. If anything, the pressure should be to find different interpretations of the passage to refute Christian claims.
Day of Atonement Prayer
The last first-millennium Jewish source that I'm aware of is a prayer attributed to Eleazar ben Qalir and written sometime between the 7th and 10th centuries. Recited on the Day of Atonement, it reads:7
Our righteous Messiah has departed from us; we are horror-stricken, and there is none to justify us.
Our iniquities and the yoke of our transgressions he carries, and is wounded for our transgressions.
He hears on his shoulder our sins to find pardon for our iniquities.
May we be healed by his stripes!
While not in any way authoritative for Jews, this prayer is especially interesting for a couple reasons. First, it would suggest the Messianic interpretation of Isaiah 53 was still popular within Judaism near the end of the first millennium. (The prayer was found in the Machzor, a collection of Jewish prayers read on holidays.) Second, it pretty much matches the Christian interpretation exactly - if one didn't know better, they might think a Christian wrote this.
Later writings
Shortly after the year 1000, we begin to find clear interpretations of Isaiah 53 as referring to the nation of Israel in Rabbinic writings (as well as a much greater number of Rabbinic writings in general). The most important of these is Rashi (1040-1115). It is sometimes even suggested he invented this interpretation. (This suggestion is clearly false, as will be shown in the "However" section below.) Yet his importance to modern Jewish thought on all matters, not just this passage, is hard to overstate. Here is what he wrote on Isaiah 53:3:8
So is the custom of this prophet: he mentions all Israel as one man, e.g., (44:2), “Fear not, My servant Jacob” ; (44:1) “And now, hearken, Jacob, My servant.” Here too (52:13), “Behold My servant shall prosper,” he said concerning the house of Jacob.
Thus Rashi equates the servant with the nation of Israel. This interpretation would gain in popularity over time, but dissenters remained, in part because of the difficulty of aligning 53:8, which implies the servant dies, with this view. Rashi himself offers:
The tribulations that befell him, for from the beginning, he was cut off and exiled from the land of the living that is the land of Israel for because of the transgression of my people, this plague came to the righteous among them.
I think this is supposed to mean that the people of Israel were cut off from the land of Israel. I'll leave it to the reader to decide if this interpretation is reasonable or not.
Another important thinker, Moses Maimonides (1135-1204), sees at least part of Isaiah 53 as Messianic. In Epistle to Yemen, he writes in response to a Messianic movement in Yemen. As such, he supplies a lot of information on what he believes the true Messiah will look like. Of interest to this study, he writes:9
Similarly, Isaiah referring to the arrival of the Messiah implies that neither his father nor mother, nor his kith nor kin will be known, "For he will shoot up right forth as a sapling, and as a root out of the dry ground." ... What the great powers are, which all the prophets from Moses to Malachi ascribe to the Messiah, may be inferred from various statements in the twenty-four books of Scripture. The most significant of them all is the fact that the mere report of his advent will strike terror into the hearts of all the kings of the earth, and their kingdoms will fall, neither will they be able to war or revolt against him. They will neither defame nor calumniate him, for the miracles he will perform will frighten them into complete silence. Isaiah refers to the submission of the kings to the Messiah in the verse, "Kings shall shut their mouth because of him."
Here, Maimonides references two verses from our passage (53:2 & 52:15) as descriptions of the Messiah. It is certainly true his overall Messianic expectation would not describe Jesus (which is the typical Jewish response to this material). Still, that does not change the fact that he saw at least some of Isaiah 53 as Messianic.
However
So far we do not have any direct evidence of Jewish interpretation of Isaiah 53 supporting a nation of Israel view before the 11th century. At best, we have one example recorded in the 12th century that might record teaching from a earlier period. However, this view does find some support in an unexpected source. In Against Celsus, written around 250, Tertullian writes about debating Isaiah with a group of Jews:10
I quoted these prophecies; to which my Jewish opponent replied, that these predictions bore reference to the whole people, regarded as one individual, and as being in a state of dispersion and suffering, in order that many proselytes might be gained, on account of the dispersion of the Jews among numerous heathen nations.
While we have no idea who Tertullian's "Jewish opponent" was, this is solid evidence that at least some Jews did see Isaiah 53 as referring to the nation of Israel. Reading further, it becomes clear that this opponent, however, didn't have a good way of explaining 53:8 under this view:
But we seemed to press them hardest with the expression, “Because of the iniquities of My people was He led away unto death.” For if the people, according to them, are the subject of the prophecy, how is the man said to be led away to death because of the iniquities of the people of God, unless he be a different person from that people of God?
Conclusion
This is an issue on which there is a lot of hyperbole - Christians tend to claim Jews never interpreted Isaiah 53 as referring to the nation of Israel during the first millennium, while Jews tend to claim authoritative Rabbis have always viewed it that way. Neither conclusion is backed by the data.
What we actually see is that, historically, three interpretations have been suggested within Orthodox Judaism - the passage refers to Moses, the passage refers to the Messiah, and the passage refers to the nation of Israel as a whole. We do not have enough data to decide which view was most common prior to the 11th century - the Messianic view has the most support, but not enough to be conclusive. Nor can we say with any confidence which view is the oldest - We have no data at all on interpretations for the first 500+ years after the passage was written.Note B
We can, however, be confident that first century converts to Christianity did not invent the idea that Isaiah 53 has Messianic tones. The idea that the passage refers to Jesus is fully developed in every strata of the New Testament, implying that the association dates back to the teachings of Jesus himself. Furthermore, the appearance of Messianic interpretations in the near-contemporary Targum and in later Jewish writing is hard to explain if the idea did not predate Christianity. It is unlikely that Jewish commentators would adopt a Messianic interpretation of a passage which previous had none in response to Christian claims that the passage was Messianic. The earliest Christians adopting or modifying existing Messianic claims about the passage is a much more likely explanation for the data.
Other views of Isaiah 53 may also predate Christianity, but we can not be confident about those claims. The earliest source backing the modern Jewish interpretation states clearly that the idea was offered in response (on that occasion) to Christian claims, and the first clear development of the argument in writing authoritative to Jews would not appear until 1000 years after the founding of Christianity.
If we are pushed to make a conclusion about primacy, we must say some sort of Messianic interpretation is more likely. However, that conclusion is not firm and the cautionary note at the start of this post applies in reverse as well - just because a Messianic interpretation might be older does not mean it is better. This post has only attempted to answer the question of whether Isaiah 53 was interpreted as Messianic before Christianity. No attempt has been made to establish what the correct interpretation is.
Notes
A It has been suggested that Berakoth 5a:
If the Holy One, blessed be He, is pleased with a man, he crushes him with painful sufferings. For it is said: And the Lord was pleased with [him, hence] he crushed him by disease. Now, you might think that this is so even if he did not accept them with love.
supports the "nation of Israel" interpretation of Isaiah 53. That seems to be quite a stretch to me, as the Rabbi seems to be providing an explanation for the suffering of righteous individuals.
B It has been suggested that certain Dead Sea Scrolls (which definitely do predate Christianity), especially 4Q491, may reflect Messianic interpretations of Isaiah 53. Thus far, I have not been able to properly evaluate such claims and so have left them aside in favor of clearly interpretations of Isaiah 53.
References
1 Defending Substitution by Simon Gathercol
2 The Ransom Logion in Mark and Matthew by J. Christopher Edwards
3 The Gospel According to Isaiah 53 edited by Darrell L. Bock & Mitch Glaser
4 "Isaiah Saw His Glory" By Daniel J. Brendsel
5 Midrash Rabbah on Chapter 2 of the Book of Ruth
6 The Fifty-third chapter of Isaiah according to the Jewish interpreters by A. Neubauer and S. R. Driver
7 Tyndale Bible Dictionary by Walter A. Elwell and Philip Wesley Comfort
8 Rashi's Commentary at Chabad.org
9 Epistle to Yemen by Moses Maimonides, chapter 17, via Wikisource.
10 Against Celsus by Tertullian, chapter 55, via CCEL.
11 "The Effective History of Isaiah 53" by Martin Hengel and Daniel Bailey in The Suffering Servant: Isaiah 53 in Jewish and Christian Sources