Home button won't work after glass replacement but touch ID does

First try restoring the phone on iTunes, if this has no success then check all the related home button connections as the black foam and easily get trapped inside the connectors, if still no joy try your old screen as the home button extension cable that clips to the logic board is easily torn where it bends through 180deg.

If non of these work then you have probably damaged the homebutton somewhere.

the home button is connected Via the Screen assembly Flex.

make sure the screen you got has a functional flex. (using a second screen is the easiest way to test this)

there may have been damage to the flex. unfortunately replacing this home button is not possible as it pairs to the motherboard, and changing it will cause all function to be lost. a certified technician is your best bet for this procedure.