RPMS fluctuating up while letting off gas

When I’m going about 40 mph and my rpm is 2k it will jump up to 3 when I let off the gas. It’ll eventually shift down but it does shift hard and you can feel a slight jerk. Even when I push the brake you can hear the engine trying to go into overdrive I’m guessing, what are your thoughts on what this could be? ive only seen it idle badly once but when I’m parked or even at a stop sign it doesn’t idle hard at all. (It’s an ‘06)