The spiritual gift of tongues serves one purpose: communication where ordinary language skills do not serve. Paul saw prophecy as useful to non-believers even though its primary purpose is for the edification of believers. But speaking in a language the unbeliever does not understand fails to accomplish any purpose at all.
A little context is in order:
If I speak in the tongues of men and of angels, but have not love, I am a noisy gong or a clanging cymbal.—1st Corinthians 13:1 (ESV)
The idea of speaking in angelic languages, which is sometimes practiced in Pentecostal denominations, comes from this passing reference. Paul seems to be using hyperbole when looking at the rest of the chapter. But even if he does approve of speaking in a non-human tongue, it's clear that it must be done in love. At this point, however, it's not immediately clear how speaking in tongues (apart from saying unloving things) could lack love.
After an excursion on the topic of love (perhaps the greatest chapter ever written by anyone), Paul returns to the topic of speaking in tongues:
Now, brothers, if I come to you speaking in tongues, how will I benefit you unless I bring you some revelation or knowledge or prophecy or teaching? If even lifeless instruments, such as the flute or the harp, do not give distinct notes, how will anyone know what is played? And if the bugle gives an indistinct sound, who will get ready for battle? So with yourselves, if with your tongue you utter speech that is not intelligible, how will anyone know what is said? For you will be speaking into the air. There are doubtless many different languages in the world, and none is without meaning, but if I do not know the meaning of the language, I will be a foreigner to the speaker and the speaker a foreigner to me. So with yourselves, since you are eager for manifestations of the Spirit, strive to excel in building up the church.—1st Corinthians 14:6-12 (ESV)
Now we can understand how Paul could say that the mere act of speaking in a language that is not understood by its hearers could lack love. If you've ever been in a situation where other people speak a language that you don't, you'll understand how frustrating it can be to not have anyone translate for you. Conversely, when you are around someone who doesn't understand what you are communicating, you probably have felt a combination of superiority and (hopefully) compassion toward that unfortunate soul.
Notice that Paul repeats the imagery of a worthless musical instrument from the previous chapter. The purpose of language is to communicate meaning, but if you can't understand you are a foreigner and outsider in relationship to the speaker. When it comes to building up the church (or in fact any community) putting people in the relationship of outsider doesn't seem like a good plan. That's why Paul encourages (in the rest of chapter 14) that church meetings be understandable to everyone present. The primary consideration is given to building up the church in love.
The historical context of the practice of speaking in tongues is Acts 2:1-13. On the day of Pentecost, with Jerusalem packed with foreigners, God gave the church the extraordinary gift to speak in many languages they did not themselves understand. Later that day, three thousand people were baptized and, presumably carried their new faith back home with them. So God gave the gift of tongues at an ideal moment to carry out His mission to spread the gospel to the ends of the earth. It may be that some of Paul's Corinthian readers were witnesses, or even products, of that event.
Luke records that some observers thought the church was drunk on Pentecost, which might be in Paul's mind:
If, therefore, the whole church comes together and all speak in tongues, and outsiders or unbelievers enter, will they not say that you are out of your minds?—1st Corinthians 14:23 (ESV)
Clearly, this sort of speaking in tongues (i.e., in a language not understood by unbelievers) fails to perform the mission of building the church. The strong implication is that this sort of tongues should not be practiced at all. Paul seems to be building on the theme of chapter 10: '“All things are lawful,” but not all things are helpful. “All things are lawful,” but not all things build up. Let no one seek his own good, but the good of his neighbor.' (10:23-24 ESV) He seems unwilling to condemn the practice of glossolalia, but gives some good reasons to avoid it in meeting where outsiders are present. Xenoglossy, however, would be quite useful to Paul and other missionaries.
Speaking in tongues is not commonly practiced at my church, but I have some personal experience with it:
When I've visited highly charismatic churches, I've found speaking in tongues in particular to be off-putting. Even in the rare cases where a translation can be given, there's a too-obvious division between those who participate and those who don't.
I speak conversational Spanish, but I believe that I've been given the gift of tongues several times on short-term mission trips to Mexico and Bolivia. Since I already knew Spanish, these experiences would not convince a skeptic and I'm certain nobody but myself knew it happened. However, I was able to speak fluently beyond my skill in a way that closely resembles my occasional experiences with prophecy.
I've known people who use private prayer languages. One person asked to use it during prayer in a (very) small group. I found the experience of listening to be odd, but comforting. It seems possible that this sort of speaking in tongues could be edifying even if lacking in semantics. The communication seemed to be of an emotional nature.