There is no precedent for infant baptism in Scripture
Infant baptism simply is not found in the Bible. That isn't an argument that it can't be done, but it should be seen as an extrabiblical tradition.
Baptism is commanded with salvation
Jesus commanded the apostles as follows:
Matthew 28:19 (NIV)
19 Therefore go and make disciples of all nations, baptizing them in the name of the Father and of the Son and of the Holy Spirit,
The thrust here (and everywhere baptism is commanded) is that the people that the apostles are making the disciples of, and the people they are baptizing, are the same people. I believe that people must choose to be disciples for themselves, which is a choice babies are unable to make.
Baptism is an outward expression of an inward change
Paul describes the inward change that baptism symbolizes:
Romans 6:2-4 (NIV)
2 By no means! We are those who have died to sin; how can we live in it any longer? 3 Or don’t you know that all of us who were baptized into Christ Jesus were baptized into his death? 4 We were therefore buried with him through baptism into death in order that, just as Christ was raised from the dead through the glory of the Father, we too may live a new life.
It is a symbol of our spiritually putting ourselves to death, so that we may live with Christ. This is not a decision that babies can make, so why perform an act that reflects an inward change that has not yet occurred?
The water is just water
In Acts, we have an example of people who had been baptized with water, and needed to be rebaptized.
Acts 19:2-5 (NIV)
2 and asked them, “Did you receive the Holy Spirit when you believed?”
They answered, “No, we have not even heard that there is a Holy Spirit.”
3 So Paul asked, “Then what baptism did you receive?”
“John’s baptism,” they replied.
4 Paul said, “John’s baptism was a baptism of repentance. He told the people to believe in the one coming after him, that is, in Jesus.” 5 On hearing this, they were baptized in the name of the Lord Jesus.
Clearly not all baptisms are equal. Yet, what changes between right and wrong cases? The water is constant, so I argue that it is the heart of the baptizee. Certainly, whatever knowledge the people were lacking that made their baptism null, babies lack even more. Peter tells us explicitly that the water is not what's important:
1 Peter 3:21 (NIV)
21 and this water symbolizes baptism that now saves you also—not the removal of dirt from the body but the pledge of a clear conscience toward God. It saves you by the resurrection of Jesus Christ,
Babies are easily able to undergo the removal of dirt from flesh - that can be accomplished by the simple act of pouring water on them. What they aren't able to do is the very thing that Peter says is important: "the pledge of a clear conscience toward God".
In regard to your arguments...
I think you bring up some interesting points, but I don't think that they make a compelling case for infant baptism.
Since the sign of inclusion into the covenant under the Old Covenant (circumcision) is so closely tied to the sign under the New Covenant (baptism) Col 2:11-12 and since the Old Covenant sign was offered to children and since the New Covenant is more inclusive in every way than the Old Covenant and since telling a Jewish Christian that they can not baptize their children would have been a big deal to them (and the scriptures show no discussion on this point at all), it has always seemed to me that the burden of proof showing the change in practice lies at the Baptist doorstep.
Is baptism circumcision 2.0?
It sounds like this is considering baptism to have been seen as a replacement for circumcision. I don't know of any Scriptural basis for this, and Colossians 2:11-12 doesn't seem to make the case. The absence of an argument in Galatians to this effect is also very conspicuous. Since in Galatians, Paul is arguing that in Christ, physical circumcision is not important, it seems that the silver bullet argument for his case would have been that baptism is the new circumcision.
Does baptism place us in the new covenant?
I don't know of any indication in Scripture that water baptism includes us in the new covenant. Rather, the indication seems to be that we are included by spiritual baptism, which occurs by faith in Christ, and water baptism is a subsequent act symbolizing that (1 Peter 3:21, Acts 10:44-48, Matthew 3:11).
John Piper makes an excellent point on this topic here. He points out that in the New Covenant, we are not included by physical descent, but spiritual:
Galatians 3:6-7 (NIV)
6 So also Abraham “believed God, and it was credited to him as righteousness.”
7 Understand, then, that those who have faith are children of Abraham.
As an aside
I'm not completely familiar with the doctrine surrounding infant baptism, so I may be off on this point. But one thing I don't understand about infant baptism placing someone in the New Covenant, is this: isn't its efficacy disproven once someone who is baptized as an infant grows up to be a non-Christian? Surely there must be some examples of this... Or is the belief that infant baptism places babies in the New Covenant until they reach an age of accountability, and then they must decide to enter it themselves? In that case, shouldn't they be rebaptized anyway once they re-enter? Neither option seems to have Scriptural support.