The important thing here is context. Lets look at the broader passage in a slightly more clear translation:
John 2:18-25 (NIV)
18 Some of his people said to Jesus, "Show us a miracle to prove you have the right to do these things."
19 Jesus answered them, "Destroy this temple, and I will build it again in three days."
20 They answered, "It took forty-six years to build this Temple! Do you really believe you can build it again in three days?"
21 (But the temple Jesus meant was his own body.22 After Jesus was raised from the dead, his followers remembered that Jesus had said this. Then they believed the Scripture and the words Jesus had said.)
23 When Jesus was in Jerusalem for the Passover Feast, many people believed in him because they saw the miracles he did. 24 But Jesus did not believe in them because he knew them all. 25 He did not need anyone to tell him about people, because he knew what was in people's minds.
Jesus was asked to perform a miracle and told the folks there that they could destroy the temple and he would rebuild it in 3 days. John here, with the advantage of having seen the events that followed, gives us a parenthetical telling us the he meant his death and resurrection by this.
I think what the part of this passage you are questioning is saying is that these are the very people who will be calling for his death in just a few short years. He knows their hearts, and even if they appear to be listening and believing they will turn into an angry mob looking to crucify him.
Matthew Henry's commentary on the passage seems to support this:
Henry asserts that Christ, knowing that these were his enemies and the people who would later crucify him, did not trust them.
III. That yet Jesus did not commit himself unto them (v. 24): ouk episteuen heauton autois—He did not trust himself with them. It is the same word that is used for believing in him. So that to believe in Christ is to commit ourselves to him and to his guidance. Christ did not see cause to repose any confidence in these new converts at Jerusalem, where he had many enemies that sought to destroy him, either,
And later
IV. That the reason why he did not commit himself to them was because he knew them (v. 25), knew the wickedness of some and the weakness of others. The evangelist takes this occasion to assert Christ's omniscience.