What you're talking about is called "substitutionary atonement" There are several references that support this viewpoint.
Surely he has borne our griefs
and carried our sorrows; yet we esteemed him stricken,
smitten by God, and afflicted. But he was pierced for our transgressions;
he was crushed for our iniquities; upon him was the chastisement that brought us peace,
and with his wounds we are healed. All we like sheep have gone astray;
we have turned—every one—to his own way; and the Lord has laid on him
the iniquity of us all.
2 Corinthians 5:21
For our sake he made him to be sin who knew no sin, so that in him we
might become the righteousness of God.
1 Peter 2:24
He himself bore our sins in his body on the tree, that we might die to
sin and live to righteousness. By his wounds you have been healed.
1 Corinthians 15:3
For I delivered to you as of first importance what I also received:
that Christ died for our sins in accordance with the Scriptures
Romans 4:23-25 (Jesus was delivered up for our sins, and raised for our justification)
But the words “it was counted to him” were not written for his sake
alone, but for ours also. It will be counted to us who believe in him
who raised from the dead Jesus our Lord, who was delivered up for our
trespasses and raised for our justification.
1 John 4:10 (Propitiation: sufficient substitution for our sins)
In this is love, not that we have loved God but that he loved us and
sent his Son to be the propitiation for our sins.
1 Peter 3:18 (Christ suffered for sins, though he was righteous, so that we may be considered righteous)
For Christ also suffered once for sins, the righteous for the
unrighteous, that he might bring us to God, being put to death in the
flesh but made alive in the spirit,
Galatians 3:13 (Christ became the curse for us, taking our penalty)
Christ redeemed us from the curse of the law by becoming a curse for
us—for it is written, “Cursed is everyone who is hanged on a tree”
Galatians 2:20-21 (If salvation is by works, then Christ's death was for nothing)
I have been crucified with Christ. It is no longer I who live, but
Christ who lives in me. And the life I now live in the flesh I live by
faith in the Son of God, who loved me and gave himself for me. I do
not nullify the grace of God, for if righteousness were through the
law, then Christ died for no purpose.
Romans 3:21-26 (The righteousness is apart from the law, and given based on Christ's blood)
But now the righteousness of God has been manifested apart from the
law, although the Law and the Prophets bear witness to it— the
righteousness of God through faith in Jesus Christ for all who
believe. For there is no distinction: for all have sinned and fall
short of the glory of God, and are justified by his grace as a gift,
through the redemption that is in Christ Jesus, whom God put forward
as a propitiation by his blood, to be received by faith. This was to
show God’s righteousness, because in his divine forbearance he had
passed over former sins. It was to show his righteousness at the
present time, so that he might be just and the justifier of the one
who has faith in Jesus.