I was able to get the url from the same Microsoft graph API

POST /me/drive/items/{itemId}/createLink

Once done I manually added some query parameters which remains same in all embed urls (em=2&wdAr=1.7777777777777777). Hence this url I was able to use in the iframe tag in react.

try {
  const response = await fetch(
    `https://graph.microsoft.com/v1.0/me/drive/items/${fileId}/createLink`,
    {
      method: "POST",
      headers: {
        Authorization: `Bearer ${accessToken}`,
        "Content-Type": "application/json",
      },
      body: JSON.stringify({ type: "embed" }),
    }
  );
  if (response.ok) {
    const result = response.json;
    const embed = result.link.webUrl + "?em=2&wdAr=1.7777777777777777"; //got the correct embed url
  }
} catch (error) {
  console.error("Error creating sharing link:", error);
}