In my case, I forgot to set the environment name of where my environment secret was defined, which cause my action not being able to access the environment secrets.

jobs:
  build:
    environment: github-pages
    runs-on: ubuntu-latest