The Pharisees were trying to trap Jesus into convicting the adulteress on the basis of their incomplete recitation of (what we know as) Leviticus 20:10. In similar fashion, we see in Matthew 4:6 the devil tempting Jesus using an incomplete recitation of Psalm 91:11.
Jesus' response to the Pharisees is to write something in the dust. The Jewish Encyclopedia explains,
Under the Talmudic law the severity of the Mosaic code was in many instances modified, and the laws relating to Adultery came under the influence of a milder theory of the relation of crime and punishment. Indeed, the rabbis went so far as to declare that a woman could not be convicted of Adultery unless it had been affirmatively shown that she knew the law relating to it—a theory that resulted in the practical impossibility of convicting any adulteress. No harm was done by this new view, because the right of divorce which remained to the husband was sufficient to free him from the woman, who, although guilty of the crime, was not punishable by the law. [emphasis added]
Perhaps Jesus is writing the crime for which she has been accused, namely, "Thou shall not commit adultery." From thence it could be said she is aware of the law, and indeed, Jesus tells her in verse 11, "go, and sin no more."
Yet, they press Him further, He challenges them, then Jesus writes something else on the ground. If, as @user3282 points out, the Aramaic wording of the challenge is "He who is without this sin among you, let him be the first to throw a stone at her," then I agree it is very plausible, indeed likely, Jesus is referring verbally to what He then bends to write in the dust, this time identifying the sin that belongs to them. In doing so, Jesus identifies, in writing, in public, the law, so it could be proven without any doubt the transgressors of that law would have had prior knowledge of it during its commission.
So, what was the law?
Now, as legal experts, none of these Scribes and Pharisees would self-indict himself for any crime that could not be proven. Jesus later rebukes their very characters, so we shouldn't expect contrition, self-examination, or a remorseful spirit from them. If it be a crime Jesus has laid at their feet, it is not adultery or sexual immorality (go ahead, prove it), it is not covetousness (again, prove it), it is not bearing false witness (she was caught in the act), nor not keeping the Sabbath holy (they lift no stones), idolatry (except to themselves, but, deny, deny), using the Lord's name in vain (they don't see Jesus as Lord, so, nope), honoring their father and mother (except, you know, Abraham, but whatever), and they haven't stolen anything (the Sadducees, sure, but certainly not the Scribes and Pharisees!).
The only one left is murder.
He who is without the sin of murder among you, let him be the first to throw a stone at her. Meaning, you can't prove she knew the law before I just wrote it down for her. But if you think you can prove stoning her is lawful, then go ahead and do it. Otherwise, if you do it, you are condemning yourself as a murderer. So He may have written:
Thou shalt not murder
Verse 6 says they tried to tempt Jesus. Ultimately, they were hoping to see the adulteress dead by Jesus' hands, and Jesus dead by Roman hands (see verse 37), and they were willing to twist God's word to do it, just as the devil did in the garden (Genesis 3:4). And, just as God put the responsibility squarely on the deceiver's head ("Because thou has done this" Genesis 3:14), Jesus puts the responsibility for this unmerciful, unjust, murderous premeditation squarely on the heads of the Pharisees by naming the sin that was in their jealous hearts before they had a chance to carry it out.
Tragically, in John 8:39-44 Jesus reminds those left to act as Abraham did, who loved God, and tried to obey God, though occasionally losing his faith. But Abraham learned, as he got older, to trust God perfectly, as Jesus instructs in Mark 12:29-30. But these men had not learned. In fact, the older Pharisees left first, being the most respected, and therefore the most culpable among these self-admitted legal experts, had their murderous consciences not been exposed (see verse 12). Nor were their actions born of foolishness, fear for one's life, the desire for children, as in the case of their great patriarch. No, Jesus in verse 44 says their very characters have become evil: "You are of your father the devil, and your will is to do your father's desires. He was a murderer from the beginning, and does not stand in the truth, because there is no truth in him. When he lies, he speaks out of his own character, for he is a liar and the father of lies."
Thus Jesus is rebuking them harshly so that they see themselves for what they are becoming: (1) liars, and (2) murderers, in their vain and growing quest to deny Him, at all costs, as Israel's Messiah.