This answer is given from the perspective of the "Swedenborgian" or "New Church" denominations that accept the Christian theology of Emanuel Swedenborg (1688-1772). Swedenborg rejected penal substitution as completely false and contrary to the plain teachings of the Bible.
The primary question is:
How do opponents of Penal Substitution explain God's declaration that He "will not leave the guilty unpunished"?
However, this and the other verses provided are among the reasons why Swedenborgians reject penal substitution.
As quoted in the question, Proverbs 17:15 says:
Acquitting the guilty and condemning the innocent — the Lord detests them both.
And Exodus 23:7 says:
Have nothing to do with a false charge and do not put an innocent or honest person to death, for I will not acquit the guilty.
And yet, under the penal substitution theory of atonement, that is precisely what happens: the guilty (sinners) are acquitted, and the innocent (Christ) is condemned; an innocent and honest person (Christ) is put to death under a false charge, and the guilty are acquitted.
We do not have to look any farther than these verses to see why the penal substitution theory of atonement is completely false and contrary to the Bible. It violates God's will and God's principles of justice as clearly and explicitly expressed in the Bible itself.
The main verse quoted in the question itself is from Nahum 1:3:
The Lord will not leave the guilty unpunished.
This, too, is violated by the penal substitution theory. In the penal substitution theory, the guilty (sinners) are left unpunished.
Penal substitution theory completely violates all three of these Bible verses. It is the penal substitution theory itself that makes "nonsense or lies of the preceding declarations."
The secondary question is:
If Christ doesn't bear the penalty or punishment for our sins by taking our guilt upon himself, where does the condemnation and punishment go? What is the specific mechanism by which, a sinner can be saved from their sins without making nonsense or lies of the preceding declarations?
The Biblical solution for sin is repentance. This is the clear message of both the Old Testament and the New Testament.
Here are a few key passages in the Old Testament:
From King Solomon's speech at the dedication of the first Temple:
"If they sin against you—for there is no one who does not sin—and you are angry with them and give them to an enemy, so that they are carried away captive to the land of the enemy, far off or near; yet if they come to their senses in the land to which they have been taken captive, and repent, and plead with you in the land of their captors, saying, 'We have sinned, and have done wrong; we have acted wickedly'; if they repent with all their heart and soul in the land of their enemies, who took them captive, and pray to you toward their land, which you gave to their ancestors, the city that you have chosen, and the house that I have built for your name; then hear in heaven your dwelling place their prayer and their plea, maintain their cause and forgive your people who have sinned against you, and all their transgressions that they have committed against you; and grant them compassion in the sight of their captors, so that they may have compassion on them." (1 Kings 8:46-50, italics added)
Excerpts from Isaiah 1, which is all about sin and repentance:
Wash yourselves; make yourselves clean;
remove the evil of your doings
from before my eyes;
cease to do evil,
learn to do good;
rescue the oppressed,
defend the orphan,
plead for the widow.
Come now, let us argue it out,
says the Lord:
though your sins are like scarlet,
they shall be like snow;
though they are red like crimson,
they shall become like wool.
If you are willing and obedient,
you shall eat the good of the land;
but if you refuse and rebel,
you shall be devoured by the sword;
for the mouth of the Lord has spoken.
Zion shall be redeemed by justice,
and those in her who repent, by righteousness.
But rebels and sinners shall be destroyed together,
and those who forsake the Lord shall be consumed.
From Ezekiel 18, which clearly spells out God's laws on sin and repentance:
But if the wicked turn away from all their sins that they have committed and keep all my statutes and do what is lawful and right, they shall surely live; they shall not die. None of the transgressions that they have committed shall be remembered against them; for the righteousness that they have done they shall live. Have I any pleasure in the death of the wicked, says the Lord God, and not rather that they should turn from their ways and live? But when the righteous turn away from their righteousness and commit iniquity and do the same abominable things that the wicked do, shall they live? None of the righteous deeds that they have done shall be remembered; for the treachery of which they are guilty and the sin they have committed, they shall die. (Ezekiel 18:21-24, italics added)
And here are a few among many similar passages in the New Testament:
John the baptizer appeared in the wilderness, proclaiming a baptism of repentance for the forgiveness of sins. (Mark 1:4)
"Be on your guard! If another disciple sins, you must rebuke the offender, and if there is repentance, you must forgive. And if the same person sins against you seven times a day, and turns back to you seven times and says, 'I repent,' you must forgive." (Luke 17:3-4)
Then he opened their minds to understand the scriptures, and he said to them, "Thus it is written, that the Messiah is to suffer and to rise from the dead on the third day, and that repentance and forgiveness of sins is to be proclaimed in his name to all nations, beginning from Jerusalem. You are witnesses of these things. (Luke 24:45-48, italics added)
Peter said to them, "Repent, and be baptized every one of you in the name of Jesus Christ so that your sins may be forgiven; and you will receive the gift of the Holy Spirit." (Acts 2:38, italics added)
The New Testament affirms and teaches the same principle stated in Ezekiel 18:21-22 (as also quoted above):
If the wicked turn away from all their sins that they have committed and keep all my statutes and do what is lawful and right, they shall surely live; they shall not die. None of the transgressions that they have committed shall be remembered against them; for the righteousness that they have done they shall live.
In short, the principle stated in the Bible is that God holds humans guilty of current sins that we continue to commit, not of past sins that we have repented of.
"The specific mechanism by which a sinner can be saved from their sins without making nonsense or lies of the preceding declarations" is "repentance for the forgiveness of sins" (Mark 1:4).
When we have believed in Jesus, and heeded his call to repent from our sins, and no longer commit them, but to live a righteous life instead, giving God the glory rather than claiming it for ourselves, God then forgives our sins, and none of the transgressions that we have committed are remembered against us.
In short, the Biblical mechanism for saving sinners from their sins is the call to repent of our sins, no longer commit them, and begin a new life of love for God and love for the neighbor. We are then no longer guilty because we are no longer sinning; and due to our current righteous life (which is the life of Christ in us, not our own life) God forgives us for all of our previous sins.
This is the clear teaching of the entire Bible, both Old Testament and New.
For a contemporary presentation of penal substitution as a false doctrine that is contrary to the teachings and message of the Bible, and also contrary to the most basic principles of justice, see my article, Did Jesus Really Die to Pay the Penalty for our Sins?!?