The button doesn't work as a handle. Instead, use Span or Div, give it button styling, and use it as a handle.

Reason: The Ui sortable gets confused by button functionality. Either it had to take a click or drag, so it does not work.