As it's been preached to me, justice, here, refers to God's faithfulness to His own promises. It does not refer to giving us what we deserve, which would be Hell. Toward us, justice would necessarily mean Hell.
Instead, "Just" refers to the fact that He has promised to forgive us through the cleansing power of Christ's blood, as long as we have repented and placed our faith in Christ.
Short version: In this verse, we see that because God is Just, He will keep His promises. That would be why the word is combined with "faithful".
For doctrinal support, I refer to Barne's Notes on the Bible
And just to forgive us our sins - The word "just" here cannot be used
in a strict and proper sense, since the forgiveness of sins is never
an act of justice, but is an act of mercy. If it were an act of
justice it could be demanded or enforced, and that is the same as to
say that it is not forgiveness, for in that case there could have been
no sin to be pardoned. But the word "just" is often used in a larger
sense, as denoting upright, equitable, acting properly in the
circumstances of the case, etc. Compare the notes at Matthew 1:19.
Here the word may be used in one of the following senses:
(1) Either as referring to his general excellence of character, or his
disposition to do what is proper; that is, he is one who will act in
every way as becomes God; or,
(2) that he will be just in the sense that he will be true to his
promises; or that, since he has promised to pardon sinners, he will be
found faithfully to adhere to those engagements; or perhaps,
(3) that he will be just to his Son in the covenant of redemption,
since, now that an atonement has been made by him, and a way has been
opened through his sufferings by which God can consistently pardon,
and with a view and an understanding that he might and would pardon,
it would be an act of injustice to him if he did not pardon those who
believe on him.
Viewed in either aspect, we may have the fullest assurance that God is
ready to pardon us if we exercise true repentance and faith. No one
can come to God without finding him ready to do all that is
appropriate for a God to do in pardoning transgressors; no one who
will not, in fact, receive forgiveness if he repents, and believes,
and makes confession; no one who will not find that God is just to his
Son in the covenant of redemption, in pardoning and saving all who put
their trust in the merits of his sacrifice.
Also from Vincent's Word Studies (also available from the same link as Clarke's Commentary)
Rev., righteous. From δίκη right. The term is applied both to God and
to Christ. See Revelation 16:5; John 17:25; 1 John 2:1; 1 John 3:7; 1
Peter 3:18. The two words, faithful and righteous, imply each other.
They unite in a true conception of God's character. God, who is
absolute rightness, must be faithful to His own nature, and His
righteous dealing with men who partake of that nature and walk in
fellowship with Him, is simply fidelity to Himself. "Righteousness is
truth passing into action" (Westcott).