The speed seems odd because the animation starts at a "random" middle value.
You can do two things:

First, you can define from explicitly so that you control where the animation starts from.

but for a better and smoother result, it’s even better to add some logic. When the user clicks to change direction while an animation is running, first read the current position of the moving ball. Then, dynamically set from based on this current position for the new direction. Also, recalculate the new duration based on the remaining distance, so the ball always moves with a constant speed (constant velocity).