It is currently not possible to use DPO (direct preference optimization) of GPT via CLI or Python on Azure. Source: private correspondence with Microsoft.