The bumpers are connected by a middle plastic cover that is around the Xbox home button. Remove this plastic piece by lifting the plastic cover over the two pins, pressing the home button in, and pushing the cover towards the top of the controller.
Be aware of the sync button loosely placed under the plastic covering. Ensure it is put back before putting back the bumper/plastic cover assembly.
Remove the current bumper assembly by pulling them off the plastic cover.