Matthew 6:16 through 18 NKJV "Moreover, when you fast, do not be like the hypocrites, with a sad countenance. For they disfigure their faces that they may appear to men to be fasting. Assuredly, I say to you, they have their reward. 17 But you, when you fast, anoint your head and wash your face, 18 so that you do not appear to men to be fasting, but to your Father who is in the secret place; and your Father who sees in secret will reward you openly.
According my understanding of these verses, what Jesus said was really 'when you do anything which draws attention the fact that you are fasting that is wrong.'
But actually it goes much further than that;
Matthew 6:1 through 8 NKJV "Take heed that you do not do your charitable deeds before men, to be seen by them. Otherwise you have no reward from your Father in heaven. 2 Therefore, when you do a charitable deed, do not sound a trumpet before you as the hypocrites do in the synagogues and in the streets, that they may have glory from men. Assuredly, I say to you, they have their reward. 3 But when you do a charitable deed, do not let your left hand know what your right hand is doing, 4 that your charitable deed may be in secret; and your Father who sees in secret will Himself reward you openly. 5 "And when you pray, you shall not be like the hypocrites. For they love to pray standing in the synagogues and on the corners of the streets, that they may be seen by men. Assuredly, I say to you, they have their reward. 6 But you, when you pray, go into your room, and when you have shut your door, pray to your Father who is in the secret place; and your Father who sees in secret will reward you openly. 7 And when you pray, do not use vain repetitions as the heathen do. For they think that they will be heard for their many words. 8 "Therefore do not be like them. For your Father knows the things you have need of before you ask Him.
The lesson of all of these Scriptures is that God listens to your Heart, or possibly more accurately to your Spirit, and therefore whatever you do expecting the acclaim of mankind, when they take notice of what you do then you have gotten what you really desired.
On the other hand it seems that Jesus is saying 'After all don't you know that the God who created all things already knows what things you are needing.'
The actual way in which we need to accept the teachings of our Savior, is that what he taught us while on Earth is the bare minimum for how God wants his Children to be.
To put the answer to your question into everyday terms 'Fasting which is a form of worship in that it's purpose is to get nearer to God, If you just fast in the idea that it relieves you of other obligations then when you have your day of rest be happy because you have gotten you heart's desire.'
Those other obligations of which I spoke include work since:
Genesis 3:17 through 19 NKJV Then to Adam He said, "Because you have heeded the voice of your wife, and have eaten from the tree of which I commanded you, saying, 'You shall not eat of it': "Cursed is the ground for your sake; In toil you shall eat of it All the days of your life. 18 Both thorns and thistles it shall bring forth for you, And you shall eat the herb of the field. 19 In the sweat of your face you shall eat bread Till you return to the ground, For out of it you were taken; For dust you are, And to dust you shall return."
Hope this helps.