This answer will focus on the religious aspects of the Amish rather than the cultural aspects, as far as differentiation is possible. Even then, there are four major orders and each community is self-governing; therefore, all statements made will be generalizations, and will naturally have exceptions.
The Amish find their roots in the Swiss Anabaptists (most still consider themselves to be Anabaptists), and their culture and traditions have been shaped by the persecutions suffered by the Anabaptists in the 1600s.
The Young Center for Anabaptist and Pietist Studies at Elizabethtown College in Pennsylvania does an excellent job of summing up the Amish way of life:
The persecution reinforced the biblical teaching of a cleavage between the church and the larger society. In Amish eyes, the kingdoms of this world, which use coercion, differ from the peaceable kingdom of God. Many Amish practices are based on the religious principle of separation from the world--that the practices of the church should be separate from the larger society.
Scripture such as Romans 12:1-2 are sometimes used to affirm their decision to step back from the world:
I appeal to you therefore, brothers, by the mercies of God, to present your bodies as a living sacrifice, holy and acceptable to God, which is your spiritual worship. Do not be conformed to this world, but be transformed by the renewal of your mind, that by testing you may discern what is the will of God, what is good and acceptable and perfect.
Also note Matthew 13:22, Matthew 16:26, and John 15:18.
The Amish seek to follow Scripture in all they do, even casting lots to select their ministers; and they tend to emphasize practicing their faith more than determining and teaching specific doctrines. They often emphasize the teachings given in the "Sermon on the Mount" and believe the community as a whole is responsible for taking care of each member, to the point where they consider insurance systems to be wrong -- it shows a lack of faith in God and subverts their responsibilities toward each other.
Contrary to popular "knowledge," the Amish do not try to completely ignore the modern world; they only wish to preserve their community-oriented way of life against the highly individualistic cultures around them.
As Peter Seibert, president of the Heritage Centre of Lancaster County puts it*:
It is easy to get it wrong about the Amish. They are not about putting up walls to block out the modern world. What they are about is adapting their community to modernity in order to preserve its essential being as a simple agrarian society.
They will pick and choose what they want from our world.