To remove the battery, the display will have to be removed. The two screws that were previously removed actually held the display to the frame.
Fingernails will be a perfect tool to just slightly lift the screen out of the frame.
The Display is snapped into place on the side of the dock connector. This end can come loose first by using your fingernails between the frame and the display.