When updating a pivot table, you are not expected to use attach(), you should use sync or syncWithoutDetaching. This ensure you do not have duplicate entries.

https://laravel.com/docs/11.x/eloquent-relationships#syncing-associations