Ok, I figured the solution, Posting anyone who might be stuck with similar issue.

1. The token has to be PublicToken NOT InternalToken. This was causing the 401.
2. The objectId passed to the GetManifestAsync has to be the version of the model that was translated, which means, it is not the URN of the file, but the URN of the version itself. Eg: fileURN can be urn:adsk.wipprod:dm.lineage:C34W6MjMRY-ul8uoPhbRyQ, but the version that I am interested in is let's say v1 of this file. Its URN will be something like urn:adsk.wipprod:fs.file:vf.C34W6MjMRY-ul8uoPhbRyQ?versionId=1 So the versionURN needs to be base64 encoded and passed to GetManifestAsync