I think some information about how the Hebrews thought of water could be of insight here...
Water is used as an image and as a metaphor for many things in the Hebrew Bible, and even the words used for it in Hebrew Scripture and Torah are quite enlightening. The word that is used for what we call "water" is first found in the first reading of the first parsha of the Torah, or what Christians would call Genesis 1:2. The actual word used here is: מָיִם
This word is actually plural... this is why some translations of the Scriptures have the word as "waters"... the problem though is that the singular of this word would be: מַי
but you won't find that word in the Hebrew. It seems that in ancient Hebrew and Rabbinic thought, water was, in its very essence of being inherently plural. IT COULD NOT BE AND IS NOT REFERED TO AS SINGULAR. This is pretty unique, linguistically speaking.
Now, Hebrew is also an interesting language, because like ancient Greek, not only do letters have letter values, they also have numberic values assigned to them (this is the realm of Gematria, which is a whole other taco... I could go on about this, but will later or if someone is interested), and they also can have mystical or what I would call metaphorical meanings ascribed to them, so that simply giving a letter can actually bring to mind a whole bundle of associated concepts. This can be seen with the letter mem, as at this site: http://www.inner.org/hebleter/mem.htm
As you can see there, mem is often shorthand for waters, forgiveness, oneness, mercy, and the unconditional love of God. Most interestingly, the "final mem" or the form of mem that is used at ends of words and either looks like a square or a sort of squashed circle (depending on your handwriting or the font you use) is also a reference to the Messiah himself. As Christians, it should be immediately obvious that these concepts of mem are found within our concepts of Jesus, and point to his identity.
So now we know the basics of "water", lets talk about "ocean" and "sea"... which is: יָם
Notice something there? Yup! You got it - that is "yam" which is "may" or "waters" inreverse/backwards! Hmmm... based on what we've seen with metaphors, symbolism, and imagery, what do you think that means?
Well, first off, "Yam" is a word that is used in other ancient Semitic languages and cultures of the levant and Middle East as the name for a god of waters: rivers, lakes, seas, and oceans. While this is part of the polytheism of the region, and not directly part of the Hebrew religion as found in the Torah and Hebrew Bible... it is important because this god was seen as a god of chaos, disorder, and
entorpy -- and these concepts about "seas" and "oceans" still are found within the meaning and concepts of "sea"/"yam" in Hebrew.
Seas were seen as being large, deep (as an abyss) things that instilled terror in the hearts and minds of Hebrews -- this is why there wasn't really an Israeli navy. The Mediterranean Sea was ABSOLUTELY TERRIFYING to Jews. (Many of the memorable Biblical stories involving the Mediterranean Sea find Jews getting passage on the ships of goyiim/gentiles, not sailing on ships built and manned by Jews!) Smaller, inland seas, such as the Sea of Galilee (the largest, lowest freshwater lake in the region) and the Dead Sea (the largest saltwater lake) were okay... they weren't seen as being horrible, chaotic, watery filled abysses calling for the bodies of dead Jews (which is how the ocean/Mediterranean was conceived of).
So, oceans were quite literally the EXACT, HORRIFYING opposite of waters!
Waters bring life. They symbolize life. In the waters of baptism, we see that all the concepts of mem are tied together and come to symbolise entering into eternal life with God. In human terms: "may" is good, "yam" is bad. Is this how God sees it? Well, we have to remember that God made all of creation, and as we read in Bereshit/Genesis 1:10, the FIRST things that God calls "good" in this new creation of his ARE the Earth and the Seas! So God sees the seas as good -- so like everything in creation, it is good, and he has power over them, and they therefore have some ultimate meaning for humanity other than being a horrible chaotic place of fear. What for an ancient Hebrew would have been the ultimate transformation in the concept of "yam"? Well, for them to become something good.
One way that these vast repositories of horror could be transformed/redeemed by and through the love of God is to be used as a symbol of redemption that unites the concepts of "may" and "yam" into one. This is what I think is behind the concept of ""Sea of Forgetfulness" and "[God] casting all our sins into the depths of the sea". While no one is beyond the love of God, one thing that is outside of it is our sins. As @Bernard R says above, Psalm 103 (Greek Orthodox Psalm 102) contains the idea of the infinite, which is linked to this concept of ultimate forgiveness and the infinity of God and his love. It shows that God chooses to remove our sins and place them as far away as is possible, which is probably more than we can concieve of... one way to understand this would be to imagine them being thrown into that horrible abyss, the ocean/"yam", where they would sink down, down, down, away, never to be seen or heard from again. This would be a way that something horrible would now be seen to have a purpose and to provide a sense of relief - those sins wouldn't come back anymore than you could get to the bottom of the ocean.
Because of this, I would say that this concept, while expressed in English and used as an idiom, is expressing an idea that goes back to the very beginnings of Hebrew thought and the Jewish religion.
I would also say that it is an expression that I would like to see more of in modern English useage and discussion, as I feel that it can be used as a discussion opening to illustrate many truths about God, love, forgiveness, and Jesus.