In the letter to the Galatians, it is written that in Christ, there is no distinction between Jew and Gentile, men and women, slave and free, because we are all one in Christ. That's generally what is quoted.
However we can also see this in Genesis, where God made man in His image, male and female.
God also tells his people Israel (e.g. in Deuteronomy) that they are to have one law for the native and the sojourner, that they are not to favour either the poor or the rich in a dispute but rather must seek justice, a man cannot simply accuse his wife of adultery and declare her guilty without evidence, etc.
The message conveyed in all this is that God does not want one gender or race or class, etc. to do injustice to / mistreat the others. He wants justice for all. That's because we are all His children and he loves us. This does not however mean that we have no difference in roles; e.g. priests must be men (see the letter to Timothy). St. Paul writes we are the body of Christ and each person is a member of that body; just like in a body, each member has a different role to play but each is vital for perfect function.
Hence the adage: we are all equal but different
I have checked once more and I see that the egalitarianism you've asked about is different from the definition I thought about. I thought you're referring to equality as in our value, but the one you've referred to is about equality in authority and responsibility. My apologies.
The texts written above are used as evidence and further arguments for egalitarianism include the existence of prophetesses like Deborah, and how Aquilla and Priscilla (husband and wife) are listed together as a couple with Aquilla being mentioned first sometimes and Priscilla being mentioned first other times.
However, it must be noted that this doesn't suggest a radical egalitarianism where there is absolutely no difference in authority or responsibility; that egalitarianism isn't supported. Instead it focuses on there being overlap in roles.