The shove is a neat feature, but it feels very clunky at the moment.
I think it should be treated as a stab, where you can aim it while the animation is following through. Being locked in makes it feel very clunky and not very fluid.
Some sort of a knockback could be neat as well since I'm assuming it's primary function is to be used defensively (since it cant break block)
Maybe adding some sort of a limit on how many times you can do it would be good as well (since you will land it more often if you can aim it while the animation is going)