In the 10th chapter of The Pursuit of God, A. W. Tozer says this:
Here are the facts as I see them. For four hundred years Israel had dwelt in Egypt, surrounded by the crassest idolatry. By the hand of Moses they were brought out at last and started toward the land of promise. The very idea of holiness had been lost to them. To correct this, God began at the bottom. He localized Himself in the cloud and fire and later when the tabernacle had been built He dwelt in fiery manifestation in the Holy of Holies. By innumerable distinctions God taught Israel the difference between holy and unholy. There were holy days, holy vessels, holy garments. There were washings, sacrifices, offerings of many kinds. By these means Israel learned that God is holy. It was this that He was teaching them. Not the holiness of things or places, but the holiness of Jehovah was the lesson they must learn.
Thus, the Law was partly intended to teach the Israelites - who followed the Most Holy God - what was holy and what was not. Like you mention, Jesus took this one step further and made clear that sin included not just one's external actions but also one's own inner, private thoughts. Basically, He took the Law that was extremely hard to keep (being a full-time job) and made it effectively impossible to keep.
The Law that God gave Moses was also part of the covenant between God and Israel. This site covers the topic nicely and succinctly, but to summarize: 1) the Israelites' end of the agreement was to follow the Law completely, 2) God described in great detail how He would punish the nation if they failed to keep the Law, and 3) God promised to establish a new covenant.
As shown in Judges, Israel got themselves into a cycle of falling into idolatry, getting punished, crying out to God for help and repenting, being rescued, having a good time and worshiping God, and then falling back into idolatry a generation or two later. Every time the Israelites disregarded the Law and God's commands, people died. Every time a person was caught willfully breaking the Law, they died.
The Law brings death, not life.
2 Corinthians 3:6 (NIV)
6 He has made us competent as ministers of a new covenant—not of the letter but of the Spirit; for the letter kills, but the Spirit gives life.
God, being who He is, couldn't very well let this go on forever. And so, He sent His Son Jesus, who is life, life, life! Jesus came so the Father could show mercy, which is repeatedly mentioned throughout the New Testament.
On one hand, Jesus made the Law even harder to follow, effectively impossible to keep perfectly. That was to show that no one can satisfy God and His commands completely. On the other hand, Jesus also came to establish a new covenant, and it is one we keep not by good works, but by faith. Thus, peoples' actions mattered less than their hearts for Him. That was why He advocated a more-lenient justice system, one that included forgiveness.
God promised to the Israelites a new covenant, Jesus came to establish it, and in the process, made the old one, the Law, obsolete.
Hebrews 8:13 (NIV)
13 By calling this covenant “new,” he has made the first one obsolete; and what is obsolete and outdated will soon disappear.
In summary, the reasons Jesus advocated these changes from the Old Testament that you mentioned are: 1) to make the Law effectively impossible to keep and 2) to show mercy and give life, and in the process of doing so, He made obsolete the Law based on works and instituted a new one based on faith.