I will attempt to answer this question from both viewpoints.
1. What's wrong with instantaneous time traveling for 'soul sleep' advocates?
For a 'soul sleep' advocate, if instantaneous time traveling were a thing, then many passages in the Old Testament that describe death as a state of silence, darkness, devoid of wisdom, thought, worship, etc. would not make much sense.
For example, David dreaded death in a number of occasions:
“What profit is there in my death,
if I go down to the pit?
Will the dust praise you?
Will it tell of your faithfulness?
[Psalm 30:9 ESV]
4 Turn, O Lord, deliver my life;
save me for the sake of your steadfast love.
5 For in death there is no remembrance of you;
in Sheol who will give you praise?
[Psalm 6:4-5 ESV]
If David would be instantaneously moved through time to the day of the Second Coming of Christ, to be instantly in the presence of God, then why did he dread death so much? He should've been happy instead, knowing that he would be moved in a matter of seconds to the presence of the Lord.
Other passages convey a similar idea:
The dead do not praise the Lord, nor do any who go down into silence. [Psalm 115:17]
18 For Sheol does not thank you; death does not praise you; those who go down to the pit do not hope for your faithfulness. 19 The living, the living, he thanks you, as I do this day; the father makes known to the children your faithfulness. [Isaiah 38:18-19 ESV]
More passages here, here and here.
2. What's wrong with instantaneous time traveling for 'intermediate state consciousness' advocates?
If death instantly teleports us to Jesus' Second Coming, there should be no conscious intermediate state. Therefore, for an intermediate state consciousness advocate this would be at odds with the evidence that supports consciousness in an intermediate state.
To the best of my knowledge, the passages that most clearly support a conscious intermediate state are:
- Luke 16:19-31 (the Rich Man and Lazarus)
- 1 Samuel 28 (the spirit of Samuel being invoked from the realm of the dead, most probably Sheol).
- 1 Peter 3:18-20 (Jesus preaching to the spirts in prison, i.e., in Sheol)
Regarding the last point, there is supportive evidence for this interpretation from the teachings of the Apostolic and ante-Nicene Fathers (e.g. see this answer), so if you hold the opinions of the direct (and no so direct) disciples of the Apostles in high esteem, belief in a conscious intermediate state should be worth considering.
Furthermore, if you are open to other forms of extra-biblical evidence, many intermediate state advocates find additional assurance in the endless stream of testimonies from people who have had near-death experiences (NDE) and out-of-body experiences (OBE). I recently found an insightful article making a case from many such reports here. If one is instantly transported to Judgement Day as soon as the spirit departs from the body, most of these testimonies wouldn't make sense at all. In fact, we could raise similar objections in the case of testimonies of resurrections (you may find many of such reports in Craig Keener's 2-volume work on miracles). If a person is instantly moved to Judgement Day upon death, then what happens when that person is resurrected in this life (like Lazarus in John 11)? Does the resurrected person travel back in time from Judgement Day to the present? What if they were dead for several days? What happens to the judgement they already received in the future, before they were resurrected?