First of all mkl was right to point out that CryptographyClient.signData instead of CryptographyClient.sign had to be used to sign the PDF signed attributes, which is always more than 32 bytes long.

After some investigation, we have also reached the following conclusions, which led to a working solution to the signature invalid issue:

1. A Keys object can never be matched to a Certificates object in Azure Key Vault. This means that signing using a Keys object, while embedding a downloaded Certificates object in the signed PDF will always fail because the public keys of the two objects will never match!
2. A Certificates object is always created with a hidden private key, which means that it can be used to sign content as well. In addition, the Certificates object may be downloaded as a certificate chain to be embedded into the signed PDF and its public key matches that of the signing hidden private key. This is actually the solution to the problem.

In addition, the following were used to implement the PDF signing process using the PDFBox library:

1. Class CreateSignatureBase.java and its supporting classes
2. Implement a custom version of org.bouncycastle.operator.ContentSigner to leverage the CryptographyClient.signData method for signing with Azure Key Vault.