The ecclesiology of Augustine and the ecclesiology of the Reformers were both very much products of the times they lived in:
In Augustine's case as well as sourcing a basic understanding on ecclesiology from scripture and tradition, any development of his thinking in this area was greatly influenced by the problems the Church had been facing - especially in it's dealings with false teachers. When he considered 'heretics' and schismatics, it was evident to him that they had wandered into error - their contradictions of the truth guarded by 'the one true church' lead him to believe that there could be no salvation outside the (visible) Catholic church - to be safeguarded from error, it would be necessary to remain within the fold and consequently submit to the established governing authorities within the visible institution.
The Reformers were faced with an entirely different scenario: In standing for their conviction of what the truth was, they were actually put out of the church. Despite largely agreeing with Augustine on what this truth constituted - especially in respect to soteriology - they found themselves in variance with the authority of the visible established church. This lead them to understand that (the Spirit of) Antichrist had not just been in operation within the visible church - leading some into error and becoming heretics and schismatics as in days gone by - but had seized control of the reins as it were - this was a radically different situation. It necessarily generated an ecclesiology that emphasized the invisible nature of the (true) catholic Church organized under Christ's headship against that of a visible human institution lead by men who, while claiming Apostolic succession for their authority, denied this authority through their life conduct and their false doctrine. Consider:
I feel much freer now that I am certain the pope is the Antichrist - Martin Luther
I posit that at least one of the reasons he felt more free is that it helped him clarify his own ecclesiology which in its previous form would have been inadequate to explain the necessity of defending what he saw as essential truth against the attacks of the (visible) Church heirarchy.
It's an interesting hypothetical to ask "what would Augustine do?" given the situation in operation in the 16th century. We can speculate regarding what he taught regarding his understanding of scripture, tradition and the situation he faced in his day, but the reality is that we just don't know.
Summary: The aspect of Augustine's ecclesiology that promoted an unreflective submission to the authority (teaching and governing) of the visible Church regardless of it's condition (he couldn't conceive that the institution could be corrupted to the extent that ancient Israel had been prior to being carried away into exile) was necessarily rejected by the Reformers as they faced the reality of being put out of the visible church while steadfastly holding to the truth that they saw in scripture. This was the departure point, and the ecclesiology they subsequently developed (although varied in visible forms according to varied interpretions of scripture and reliance on tradition) was based on the understanding of the 'true' catholic Church of Christ being invisible in form.
Disclaimer: I imagine there are people on this is site that are much better qualified to answer this question than I, but considering the quality of answers so far, I would like to at least set a minimum standard of answer for this important question. My view has been informed subtly by reading many sources over a number of years, but the only one that stands out as more influential than others in regard to answering this question would be Bruce Shelley's Church History in Plain Language. So I don't claim to be an expert in either Augustine or the Reformers, just that I have a conceptual framework that informs a view of what is the main issue that pertains to answering this question.