Managed to achieve this by saving the conversation messages and pass these as input when recreating the conversation thread.