What is heaven?
Within Christianity, it's generally accepted that 1) first comes death, 2) then the intermediate state, 3) then the universal (bodily) resurrection. The exact nature of each of these is (mildly) disputed, but in general, it's said within Protestantism that the body "sleeps" after death until the resurrection. Whether the soul sleeps or not is disputed but often Paul's words, "absent from the body, present with the Lord," (2 Cor 5:8) are taken to mean that we will be conscious before the resurrection; this is referred to as being in heaven, since it is "with the Lord" and God is said to dwell in heaven.
I'd also like to note that I think you were on the right track in discussing the new Jerusalem. The end of Revelation is all about the new heavens and earth. At the resurrection, some will go to everlasting destruction and others to everlasting life (Dan 12:2). The eschatological hope of a Christian culminates in heaven and earth being remade and heaven coming to earth, and it is in that new heaven and earth that he will spend eternity.
Very often, in colloquial language, the intermediate state and the resurrected state are folded into one concept, which is given the name "heaven." I don't think that does any terrible injustice to either concept. So, when people speak of being together "in heaven," they mean being together "after death."
Thus far, I believe I have described the orthodox Protestant position. There is little disagreement in these areas. You can read such ideas of heaven in Randy Alcorn's book Heaven, John MacArthur's book The Glory of Heaven, and N.T. Wright's book Surprised by Hope.
Will we see our loved ones in heaven?
I don't think I can do any better than to quote what John MacArthur says:
Will we recognize and be reunited with our loved ones in heaven?
Yes! In the Old Testament, when a person died, the biblical writers
said he was "gathered to his people" (cf. Gen. 25:8; 35:29; 49:29;
Num. 20:24; Judg. 2:10). In 2 Samuel 12, when David's infant child
died, David confidently said, "I shall go to him, but he shall not
return to me" (v. 23). David evidently expected to see the child
While sharing the Passover meal with His disciples, Christ said, "Take
this [cup] and divide it among yourselves; for I say to you, I will
not drink of the fruit of the vine until the kingdom of God comes"
(Luke 22:17-18). Christ was promising that He and His disciples
would drink the fruit of the vine together again--in heaven. Elsewhere Jesus makes a similar, but even more definite, promise:
"Many will come from east and west, and sit down with Abraham, Isaac,
and Jacob in the kingdom of heaven" (Matt. 8:11).
Describing the Lord's appearing and the resurrection of the saints who
have died, Paul writes, "Then we who are alive and remain shall be
caught up together with them in the clouds to meet the Lord in the
air. And thus we shall always be with the Lord" (1 Thess. 4:17).
Paul's purpose in writing was to comfort some of the Thessalonians who
evidently thought their dying loved ones would miss the return of
Christ. He says in verse 18, "Comfort one another with these words."
The comfort comes from the prospect of reunion. ... Paul's promise
that we will all be "together" forever implies that we shall renew
fellowship with all whom we have known.
I've asked a question on BH about what it means to be "gathered to his people."