Click the Edit Products button in the Product Section.
This will take you to the Product Grid layout where you can navigate to the required Category.
Once there you can check the box next to the product (and also select multiple products if needed) then from the options presented on the right-hand side, choose to delete the product.