A sticky adhesive holds the speakers in place in addition to the screws. The speakers will likely be reluctant to detach from the panel due to this cement.
To overcome the adhesive and remove the speakers, place the screw driver in the hole in the speaker mesh and pry out, using the raised metal lip in the center of the speaker for leverage.
A good amount of force may be necessary to completely overcome the adhesive. NOTE: The tweezers or needle nose pliers may be used in place of the screw driver.