Yes and no. Calvin said something to the effect of "God cannot be your Father if the Church is not your mother," his point being that the church is very, very important to the spiritual life of the Christian. Most if not all New Testament letters are addressed to specific churches, in Acts orders are given as to the order of worship, and Hebrews 10:24 says "not neglecting to meet together, as is the habit of some, but encouraging one another, and all the more as you see the day drawing near".
Here's some verses on the church pulled from my study bible (don't have to time to check them all)
John 10:16; Eph 2:18, 3:15, 4:6; 1 Pet 5:2-4; Eph 1:22, 23; Gal 4:26; Heb 12:22-24;
and some on what it is
1 Cor 12:12-16; Eph 1:22,23; 1 Cor. 12:27
Also important in this question is a reformed understanding of the Lord's Day (please forgive what's obviously a lot of controversy surrounding this): that the Lord's Day (Sunday) is set apart, as a continuation of OT Sabbath, which was to be kept holy ("set apart"). Isaiah 58 says
If you turn back your foot from the Sabbath,
from doing your pleasure on my holy day,
and call the Sabbath a delight
and the holy day of the Lord honorable;
if you honor it, not going your own ways,
or seeking your own pleasure, or talking idly;
then you shall take delight in the Lord,
yand I will make you ride on the heights of the earth;
I will feed you with the heritage of Jacob your father,
for the mouth of the Lord has spoken.”
I think this definitely carries over to our observance of Sunday now. You obviously don't have to go to church every Sunday, unqualified. The thief on the cross didn't. We certainly don't want to be legalistic about it - Christ's saving work has fulfilled the law for us, we don't earn our own salvation. But just like as God's adopted sons we have to obey him, in that sense we have to go to church. In light of the Isaiah 58 verses, think of it like this: why wouldn't you want to go to church? Worshiping God in the splendor of holiness through singing, receiving his grace and instruction and care through the reading and preaching of his word, talking with him through prayer, and fellowshipping with brothers in Christ - this should all be very important to us, especially on a day that God has set apart for us. We have six other days to get the rest of our life out of the way so we can worship and rest in God on one (just like God did at creation).
Forgive me if this comes as harsh, but in your case i think church attendance is more important than scouts. Scouts (As I see it) isn't an "ox in the ditch."
Perhaps you could find another church to attend. As I see it, church is #1 for me on Sunday, above all else, but the least you can (and should) do is to use as much as your Sunday as possible with God (whether at church or not).