Call

event.completed();

after your dialog is closed by the user:

        dialog.addEventHandler(Office.EventType.DialogEventReceived, (arg) => {
          if ("error" in arg && arg.error === 12006) {
            event.completed();  
          }
        });

Otherwise your action will be unloaded.