From the Reformed Presbyterian perspective, I would say that the basis for all public worship (i.e., worship with the Lord's people on the Sabbath) is found in the Regulative Principle of Public Worship: all elements of pblic worship must have an express warrant in Scripture. Put another way, you can only do those things in public worship that God has expressly set down in Scripture. The basis for the Regulative Principle is the Second Commandment in Exodus 20:4-6 or Deuteronomy 5:8. I'll quote the first:
4 “You shall not make for yourself an image in the form of anything in
heaven above or on the earth beneath or in the waters below. 5 You
shall not bow down to them or worship them; for I, the Lord your God,
am a jealous God, punishing the children for the sin of the parents to
the third and fourth generation of those who hate me, 6 but showing
love to a thousand generations of those who love me and keep my
Leviticus 10:1-2 shows how seriously God takes worship of Himself. It must be done His way. Now, I am not arguing, obviously, that God kills people who worship Him in unauthorized ways today. However, if something warranted the death penalty in the OT, I think we may infer that it's a bad idea, even in our time.
Now, contrary to some, I believe that the Regulative Principle does not end the matter, because now it becomes a question of "In Scripture, what has God commanded His people to do in public worship?" There are some (the Covenanters, dear brothers in Christ) who say that you should only sing Psalms in worship, and you should not use instruments (the so-called "exclusive psalmody, no-instruments position"). I am not of this view.
The practice of singing in public worship has express warrant in Scripture. Aside from a myriad of Psalms that command the people of God to sing praises to Him in worship, there are passages such as Colossians 3:16:
16 Let the word of Christ dwell in you richly, teaching and
admonishing one another in all wisdom, singing psalms and hymns and
spiritual songs, with thankfulness in your hearts to God.
The context of the phrase "singing psalms and hymns" is "teaching and admonishing one another", which can only realistically happen in the congregation. And there are other passages which command singing together.
Therefore, singing praises to God is considered an "element" of worship, to use the technical term often used in discussions of the Regulative Principle.