First and foremost I want to say that I think it's pretty much just an old cliche and I've heard it mostly used when people are talking while the preacher is preaching. I think the idea behind the cliche is generally saying that we aren't here to socialize we are here to worship.
However, let's see if we can take that cliche a little deeper and let's start with one scripture that just about sums it up, Matthew 21:13:
And said unto them, It is written, My house shall be called the house of prayer; but ye have made it a den of thieves. (KJV)
However, in context this verse alone isn't enough because Jesus was speaking of some very specific activities outlined in the previous verse, Matthew 21:12:
And Jesus went into the temple of God, and cast out all them that sold and bought in the temple, and overthrew the tables of the moneychangers, and the seats of them that sold doves, (KJV)
So, let's see if we can concrete this some more right? How about we go a little deeper to 1 Corinthians 6:19-20:
19 What? know ye not that your body is the temple of the Holy Ghost which is in you, which ye have of God, and ye are not your own?
20 For ye are bought with a price: therefore glorify God in your body, and in your spirit, which are God's. (KJV)
Here we gain an understanding of the temple as its known today, our own body, and Jesus said in John 4:21-24 just what He's after when it comes to the body:
21 Jesus saith unto her, Woman, believe me, the hour cometh, when ye shall neither in this mountain, nor yet at Jerusalem, worship the Father.
22 Ye worship ye know not what: we know what we worship: for salvation is of the Jews.
23 But the hour cometh, and now is, when the true worshippers shall worship the Father in spirit and in truth: for the Father seeketh such to worship him.
24 God is a Spirit: and they that worship him must worship him in spirit and in truth. (KJV)
Specifically take note to this
for the Father seeketh such to worship him.
So, if He's seeking those to worship Him, we can't do that by socializing. Worship is not a form of socializing, worship is a form of praise, thanks, and communication with Jesus that's much deeper than the superficial gatherings of clubs like the Kiwanis or the Boy Scouts.
So, with those scriptures I think we can see that the temple is meant for praise and worship rather than socializing. However, let's not get carried away because that's what we love to do, the Bible does not condemn gathering together to socialize with one another. What we do before and after service, that's socializing, and it's necessary for the saint to stay saved. Take note to 2 Corinthians 6:14-18:
14 Be ye not unequally yoked together with unbelievers: for what fellowship hath righteousness with unrighteousness? and what communion hath light with darkness?
15 And what concord hath Christ with Belial? or what part hath he that believeth with an infidel?
16 And what agreement hath the temple of God with idols? for ye are the temple of the living God; as God hath said, I will dwell in them, and walk in them; and I will be their God, and they shall be my people.
17 Wherefore come out from among them, and be ye separate, saith the Lord, and touch not the unclean thing; and I will receive you,
18 And will be a Father unto you, and ye shall be my sons and daughters, saith the Lord Almighty. (KJV)
There's a lot here but I'm going to try and keep it simple.
Be ye not unequally yoked together with unbelievers is letting us know, we can't commune, or today you might call it socialize, with unbelievers for the vast majority of our time.
Then we see the same theme here that I pointed out previously, and that is that we are in fact the temple,
And what agreement hath the temple of God with idols? for ye are the temple of the living God, and so God is saying we have got to stick together and spend time together with like minded people.
Then you have a clincher with this one,
Wherefore come out from among them, and be ye separate, saith the Lord, and touch not the unclean thing; and I will receive you, clearly stating that we need to come out from among those that God saved us from.
At any rate, it's still not saying you can't spend time with them, it's just letting them know that you have to be spending time with each other because if you get too close to the fire you're going to get burned. You have to spend time with those that aren't in church or you'll never get them in church, but they should be somewhat uncomfortable around you at times even because you're different, you're separate.
Alright, I'm going to have to quit now before I go on forever, but I hope this has helped.