In the Catholic Church (at least, according to the Church's teaching today), a validly ordained priest retains the powers of the priesthood for life, even if he is excommunicated or laicized and thus forbidden to use them. Therefore, an excommunicated or laicized priest who celebrates the Mass does so validly (the bread and wine truly become the Body and Blood of Christ as they would were he not excommunicated or laicized) but illicitly (he celebrates the sacrament in a manner against Church law). However, in such cases, it is assumed that the priest celebrates the Mass according to an order of Mass that either is approved or has once been approved by the Church. For example, a validly ordained priest who is a member of the illicit Society of St. Pius X celebrates Mass validly in spite of using the Tridentine order of Mass, which is not the order of Mass endorsed in the current Roman Missal. Regarding the necessity of proper matter and form, true bread and true wine must be used (proper matter), and I believe the part of the anaphora that is integral to proper form (and thus to valid consecration) is the words of institution (in the Catholic Church, "this is my body" and "this is my blood").
Luther's case (and by extension the case for all other priests of the day who would come to celebrate his revised Mass) is unique in that although he was a valid priest who could theoretically continue to celebrate Mass illicitly according to the order of Mass in use in his day, did his consecrations remain valid even after he made notable changes to the Eucharistic liturgy that were not ever approved by the Church? Perhaps most crucial to consider here is that although he retained the words of institution, he translated them into German. Would that be enough to invalidate the sacrament?
Also to be considered are Luther's intentions when celebrating the Mass after his alterations of the Mass order. It is notable that among the early reformers he is one of few, according to Hendrix, who maintained throughout his life that the Body and Blood of Christ are truly "present in" the bread and wine, though he was never too keen on elaborating in what fashion this was so. The biography never explicitly portrays Luther as denying the formulation that the bread and wine "become" the Body and Blood of Christ (transubstantiation), but the former formulation is used by him more frequently. What is certain, however, is that Luther explicitly rejected the sacrificial character of the Mass according to Hendrix (see the second list point below for citation): while he intended, then, to make the body and blood of Christ present in the Mass, he did not intend to make the sacrifice of Christ's body and blood present, which is integral to proper intention according to this report, which elucidates Apostolicae Curae (see quotation of the relevant paragraph in Ken Graham's answer). What is clear from this is that Luther's defective intention would have invalidated his Mass; what I would like answered in the question is whether, all other conditions being adequate for a valid consecration, Luther's revised order of Mass would have invalidated his consecrations.
Other alterations that Luther made to the Mass which may be pertinent to take into consideration include:
- Its translation in whole or in part into German (although he permitted most of the Mass to remain in Latin depending on the scruples of a given congregation, he always spoke the words of institution in German), and
- The removal of the "long prayer of consecration that implied the mass reenacted the sacrifice of Jesus." (Hendrix 128, 129; elsewhere in the biography Hendrix mentions Luther's distaste for the canon of the mass, to which he is likely referring with "long prayer" here. Clearly Luther retained the words of institution, albeit in German, but if I were to guess, he would have at least done away with the preface, the oblation, the epiclesis, and the intercessions [see anaphora]. Of course, if someone else knows more specifically what was changed, that would be good to know.)