Yes, SSO needs to be set up for every Domain you have if the users should be able to access over SSO. If you want to set up SSO for your domain, you can contact the Frontify Support Team.
Please note that SSO setups are only possible with an Enterprise Plan.