TLDR; We are expected to apply our normal human reasoning and communication skills to understand the Pope's (or any Bishop's) intended level of authority.
Consider the context. Consider the phrasing. Consider the look on his face! You've spoken to people before — Use your human reason in earnest good faith to decide whether the Pope or Bishop intends to speak with authority, and in what capacity he intends to do so. (Faith, morals, etc.) When in doubt, cross-reference your interpretation with others, particularly with church authorities. Check to see if the Pope or Bishop has clarified or repeated the message in other tones or similar tones. Etc.
Wikipedia's entry on Magisterium does a reasonable job summarizing this and provides a large set of helpful references. For a more official resource, Lumen Gentium makes it very explicit.
- Among the principal duties of bishops the preaching of the Gospel occupies an eminent place.(39*) For bishops are preachers of the faith, who lead new disciples to Christ, and they are authentic teachers, that is, teachers endowed with the authority of Christ, who preach to the people committed to them the faith they must believe and put into practice, and by the light of the Holy Spirit illustrate that faith. They bring forth from the treasury of Revelation new things and old,(164) making it bear fruit and vigilantly warding off any errors that threaten their flock.(165) Bishops, teaching in communion with the Roman Pontiff, are to be respected by all as witnesses to divine and Catholic truth. In matters of faith and morals, the bishops speak in the name of Christ and the faithful are to accept their teaching and adhere to it with a religious assent. This religious submission of mind and will must be shown in a special way to the authentic magisterium of the Roman Pontiff, even when he is not speaking ex cathedra; that is, it must be shown in such a way that his supreme magisterium is acknowledged with reverence, the judgments made by him are sincerely adhered to, according to his manifest mind and will. His mind and will in the matter may be known either from the character of the documents, from his frequent repetition of the same doctrine, or from his manner of speaking.
(Emphasis mine.)
It's reasonable to give pause to consider with seriousness almost anything a Pope or Bishop says. But, it's unreasonable to do so indiscriminately of how they say it.
Suppose the Pope has a Belgian Lager and exclaims, "This is the most delicious beer of all time!" Clearly, beer is a matter of faith and morals. But, are we really so undiscriminating and silly that we can't distinguish this from his magisterial authority? (Everyone knows that "the best beer of all time" is actually seasonal and is usually a Stout.)