I thank you guys so much. Especially the Tips from Dave and Mart Kroon!
This tiny plastic piece is in my opinion a deliberate bad choice for it's function, it can very easily fall out of place and leave a completely functional device useless.
I simply glued it back on with some standard glue which i aplied with the cut-of end of an q-tip. Be sure to use only a tiny amount, since the button can get glued tight and is not movable anymore.
During the disassembly, the plastic piece from my "Decrease-Volume" Button got lost, so i replaced it with a tiny piece of plastic i cut of the q-tip. Be careful not to make it to big. If it is to big and already glued on, you can leave away some or all of the 4 screws holding the buttons in place and just put a piece of plastic between the battery and the cable so the buttons stay in place but can get pressed further in.