Download the binary being sure to use the correct architecture
curl -L -o caddy.tar.gz "https://github.com/caddyserver/caddy/releases/latest/download/caddy_2.9.1_linux_amd64.tar.gz"
and install
tar xvf caddy.tar.gz
sudo cp caddy /usr/bin
caddy version