Previous popes who abdicated lived in seclusion (in fact Celestine V was imprisoned) so it's highly unlikely that there will be any useful or easily-available records of what they did. As Benedict XVI has said that he will do likewise, any answer must necessarily speculate. It is possible to draw on general rules, though.
The pope's priestly ordination and episcopal consecration cannot be expunged, so he is entitled to wear clothes and vestments appropriate to clergy of those ranks. Thus he will continue to wear a clerical collar and cassock, and can continue to wear a greco (long coat), tabarro (long outdoor cape) and pellegrino (short shoulder-cape). Normally the greco is white for the pope; his tabarro and pellegrino are red; everything is black for all other clergy.
Here's Bl John Paul II in a greco (from Dieter Philippi):
Here's a cardinal in a tabarro and pellegrino (ibid). He's also wearing a black cassock with red buttons and a scarlet cincture. The headgear is called a zucchetto.
Because the Pope Emeritus remains a bishop, he is entitled to the "pontificals" of a bishop in liturgical celebrations. However he does not have jurisdiction over a diocese, so will not use anything which would indicate that, such as the mozzetta (the short cape which is buttoned at the front); nor will there be a seventh candle on the altar if he is principal celebrant.
Here's a picture of a mozzetta (Dieter Philippi again). The pope's is fur-lined or fur-trimmed; other ranks are simply cloth of the colour of their rank. The difference between a mozzetta and a pellegrino is that a mozzetta is buttoned in front.
Thus it may be expected that Benedict XVI will continue to wear an amethyst1 ring, and in liturgical celebrations will wear a mitre, pectoral cross and carry a crozier. I would expect that the crozier will be used "facing backwards" — that is, with the hook facing towards himself — as that is the traditional indication that the bearer has no ordinary jurisdiction in the place. The picture reproduced below shows a bishop wearing a mitre (the hat) with a pectoral cross (on his chest) and carrying a crozier in the manner of a bishop without jurisdiction.
1 Bishops wear an amethyst ring, and purple cinctures, because amethyst in Greek means "I am not drunk". It's a reference to Acts 2:15.