The short answer is “not necessarily.” There can be good reasons to go to work on Sunday or another holy day of obligation. (Note that these reasons do not generally excuse someone from attending Mass on those days, or at least at the vigil Mass the evening before.)
The Catechism of the Catholic Church addresses the issue of how to live out Sunday rest (applicable as well to other holy days of obligation), as follows:
On Sundays and other holy days of obligation, the faithful are to refrain from engaging in work or activities that hinder the worship owed to God, the joy proper to the Lord's Day, the performance of the works of mercy, and the appropriate relaxation of mind and body. Family needs or important social service can legitimately excuse from the obligation of Sunday rest. The faithful should see to it that legitimate excuses do not lead to habits prejudicial to religion, family life, and health (no. 2185, emphasis added).
Therefore, if the family greatly needs it (e.g., poor families that need money just to make ends meet), or if one’s profession is to provide an important service (e.g., doctors, nurses, police officers, clergy), then it is permissible to work even on Sunday or Holy Days.
(I will observe that, in the United States in particular, some of the holy days of obligation do not coincide with national holidays. In many, if not most cases, taking those days off from work will be too disruptive. In such a case, it is not a problem to work. That is one reason why in the dioceses of of the United States many solemnities that are traditionally celebrated in mid-week have been transferred to Sunday.)
The Catechism even goes on to specify that those whose profession is to help others live out their Sunday rest may also work on Sunday, for it is an important service:
Sanctifying Sundays and holy days requires a common effort. Every Christian should avoid making unnecessary demands on others that would hinder them from observing the Lord’s Day. Traditional activities (sport, restaurants, etc.), and social necessities (public services, etc.), require some people to work on Sundays, but everyone should still take care to set aside sufficient time for leisure (no. 2187).
Of course, such persons should be careful to set aside time for leisure (for themselves and their employees) at some other moment.
One final observation: for holy days of obligation that fall on Saturday or Monday, what is abrogated (in dioceses of the United States) is not the holy day, but the obligation to attend Mass. (Incidentally, this norm does not apply to the Solemnity of the Immaculate Conception.)