There are various third-party iPhone Backup Extractors that should be able to pull the photos from the backup file, just FYI. Both Mac and Windows. Of course, since iTunes overwrote the backup file the OP wanted, doesn't really help in this case.
As long as nothing is actually wrong with your iPhone, you can exit Device Firmware Update (DFU) mode without restoring the device in iTunes. To get out of DFU mode follow the same steps as a forced restart. 1. Hold down the Sleep button and Home button at the same time. 2. After a few seconds the Apple logo appears and the iPhone will boot. 3. Your iPhone is now out of DFU mode. If the procedure doesn't work the first time, try steps 1-2 again. If something is wrong with your iPhone you may need to restore your iPhone 5 using iTunes.
