The CDC (Centers for Disease Control) and the FDA (Food and Drug Administration) are both US federal agencies that have overlapping authority over medical testing in the US. In general federal rules prevail over state rules, so yes the CDC can stop the state of New York from testing for COVID-19 if the CDC has not approved the testing protocol.
The primary reason behind the rules is to ensure that any tests provided to the public are reliable. A secondary reason it to ensure that the testing development process followed guidelines on medical ethics. There is a set of procedures for Emergency Use Authorization, but there have been widespread complaints that even that process has grown too cumbersome and time consuming.
There has been more extensive coverage of the CDC and FDA stopping/delaying testing in Washington state.