What message did Jesus Christ tried to give by comparing the Kingdom Of God(Isn't that Heaven?) with the mustard seed and yeast?
Here Jesus teaches that He has come to prepare us for the kingdom of heaven (Matt. 13.31) and this kingdom of heaven has to be first created within us if we have to be partners of that kingdom. This kingdom is created within us by His message of Gospel which is simplest of all.
The Gospel message has immense potential to change the medium where it is taught: be it a person, or region or the world. That is what it has done to the world in spite of people severely prosecuting at the onset, the simpleton disciples of Jesus.
The quantity of yeast is minuscule compared to the whole dough, yet it makes full dough to rise with its unique effect. The size of a mustard seed is negligible compared to all other seeds yet it grows into a large plant.
So is the message of Gospel compared to works of other great thinkers and philosophers. It is in simple words and parables yet it is a goldmine full of wisdom and is understandable even to unlearned and lowly. It has the power to change the face of the world and it did. It has the power to win the wisest and make their wisdom appear to be foolish.
Jesus was saying that the Kingdom of God has a huge effect on whatever it interacts with. That is, the effect is disproportionately large in comparison to what we mortals perceive the Kingdom of God to be.
The Kingdom of God refers to more than just Heaven. The Kingdom of God also includes his people, many of whom are still here on Earth.
There are already two good answers here representing the typical interpretation of these parables, that the rapid growth of the mustard plant and yeast are pictures of the rapid growth of Christianity, so I'd like to offer a contrasting view: these parables are actually a warning about corruption within the Church.
This is a minority view (see sources below), and I realize that people much more gifted than I am have drawn out some wonderful lessons based on the standard interpretation, but I think the context and language of that passage of scripture actually point in another direction, while still being consistent with Jesus' other teachings about the Kingdom.
Context: these parables occur together in both Matthew 13 and Luke 13
Language: if we assume that certain symbols are used consistently throughout the Bible, a different picture starts to emerge.
As other answers have already said Jesus taught that the Kingdom of God / Kingdom of Heaven (basically synonyms) is more than just Heaven (in the sense of an afterlife); it is a kingdom composed of everyone who submits to God's rule in their hearts -- in a word, the Church. However, when we look at Christ's followers across history (from a human perspective) we don't see a single unified kingdom -- we see a mess: "political intrigue, lust for power, intolerance, vain superstition, immorality, and greed" -- in other words, false religion.
We should take comfort therefore, that Jesus warned us from the beginning that His Kingdom would be intermingled with the influence of its enemies. And (if you will forgive a little preaching in this Q&A format), we should also put this into perspective by remembering that each of us is susceptible to hypocrisy, sin, and holding false ideas about God. If I want His Kingdom to spread on earth, I need to start by examining how fully that Kingdom really rules in my own life.
Full disclosure: while researching this answer, I've found that this interpretation seems to be typical only among (Dispensational?) Premilennialist teaching.
The Kingdom of God is NOT EXACTLY a place in every parable that has this line. It is the reign or kingship of God, God who rules, rather than a place where God rules.
The Parable of the Mustard Seed signifies the kingdom/reign of God's sudden and surprising transformation from its almost invisible beginning to its full grandeur. It's described to be dynamic or active.
The Parable of the Yeast explains that like the yeast, the reign of God transforms human history by infusing into its activity the invisible spirit of and power of God. The Universal expansion of the reign of God expects not necessarily an acceptance by all but a universal disturbance created in the world by those who do accept. This is a disturbance like that of the leaven that cause the dough to rise. The KOG is the yeast, God is the baker, and we (man) are the dough. It is described as transformative because it allows us to change for the better, so we should do good things from the heart.
I read it from my school textbook by the way.