Blinking question mark, not able to install OS

When I turn on iMac I get the chime and after some time blinking question mark inside the folder appears.

I have tried all combination keys on mac wireless keyboard and Windows USB Keyboard but nothing worked.

Then I replaced hard drive for brand new SSD but that didn't change anything.

What else I have tried:

1) All combination keys I could think of. On original mac wireless keyboard and Windows USB keyboard.

D, N, T, Option, Command + R, Option + Command + P + R, Shift, Command + S, Command + V

Whatever combination I use, doesn't make any impact on the boot process and a question mark appears after while.

2) I plug USB Stick with a bootable system and also System on CD but none of the combination keys worked. I'm unable to eject CD using the Eject button on the keyboard. I can only eject using the mouse while starting.

3) I have replaced batteries on the keyboard and when I turn on with side power button I can see a green LED briefly turn on. Caps lock LED doesn't work but I don't remember this working ever.

4) I have removed all but one RAM memories.

I spent the last few days researching the problem but all point to faulty HDD which I already replaced. What could be a problem? Anything else I could try?