It also means a certain collection of values. And having rules for girls and women that include them having to cover their hair and body “because religion” is going against those values.

And yes, it absolutely does matter in a secular state whether people forcing their children into religious beliefs. At least in school the children should learn that these rules only exist in the minds of their parents or communities. Freedom also means to be free to choose. And grooming your children into religious practices is not freedom.