I followed @GuiFalourd's approach, but it did not work for me. It is possibly because my repo deletes PR branch automatically right after PR is merged. So in my case I was successfully able to retrieve PR using github.sha with gh cli.

name: Get PR on Push

on:
  push:
    branches:
      - main

jobs:
  spec:
    name: Prepare spec
    permissions: read-all
    runs-on: ubuntu-latest
    outputs:
      pr-number-closed: ${{ steps.gh-cli.outputs.pr-number-closed }}
    steps:
      - name: Checkout for gh cli
        uses: actions/checkout@v4
        with:
          fetch-depth: 0
      - name: Get PR Number
        id: gh-cli
        env:
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
        run: echo "pr-number-closed=$(gh pr list --search ${{ github.sha }} --state merged --json number -q '.[] | .number' || echo "")" >> $GITHUB_OUTPUT