Do I need to apply for or keep a standard VAT registration if I want to use the OSS scheme?

Yes, a standard VAT registration is required for the OSS registration. 

Businesses established in the EU have to register in the country of their establishment.

Non-EU businesses should register in the country from which they send goods to consumers (eg in the country where they keep their stock).