Well, in my case, it turned out to be correcting my silly mistakes.

1. The GitHub workflow YML file was located in .github/workflow/deploy.yml not .github/workflows/deploy.yml. ("workflow"(wrong) instead of "workflows"(right)) I first couldn't understand what did @jonsharpe's words that the run didn't use actions-gh-pages. However, I believe this means the actions were not fired due to the incorrect path (in my specific case). After making that change, the workflow for page deployment now works on the GitHub repository with every new push event (as specified in the YML file).

2. I also changed deploy.yml like below.

permissions:
  contents: write    # allow git pushes
  pages:   write     # allow Pages deployment

on:
  push:
    branches: [ main ]

jobs:
  build:
    runs-on: ubuntu-22.04

    strategy:
      matrix:
        node-version: [22.x]

    steps:
      - name: Checkout code
        uses: actions/checkout@v3
        with:
          persist-credentials: true

      - name: Use Node.js ${{ matrix.node-version }}
        uses: actions/setup-node@v3
        with:
          node-version: ${{ matrix.node-version }}

      - name: Install dependencies & build
        run: |
          npm ci
          npm run build

      - name: Deploy to gh-pages
        uses: peaceiris/actions-gh-pages@v3
        with:
          publish_dir: ./_site
          github_token: ${{ secrets.GITHUB_TOKEN }}
          nojekyll: true

There are some changes:

• I upgradedd the Node.js version(strategy.matrix.node-version) from 18.x to 22.x because the older version of Node.js couldn't install NPM dependencies and installed that to continue the deployment procedure. (The text below is the error I got due to insufficient Node.js version)

[11ty] Eleventy Fatal Error (CLI):
[11ty] 1. Error in your Eleventy config file '.eleventy.js'. (via EleventyConfigError)
[11ty] 2. There was a problem importing '.eleventy.js' via import(cjs) (via EleventyImportError)
[11ty] 3. `require("@11ty/eleventy")` is incompatible with Eleventy v3 and this version of Node. You have a few options:
[11ty]    1. (Easiest) Change the `require` to use a dynamic import inside of an asynchronous CommonJS configuration
[11ty]       callback, for example:
[11ty] 
[11ty]       module.exports = async function {
[11ty]         const {EleventyRenderPlugin, EleventyI18nPlugin, EleventyHtmlBasePlugin} = await import("@11ty/eleventy");
[11ty]       }
[11ty] 
[11ty]    2. (Easier) Update the JavaScript syntax in your configuration file from CommonJS to ESM (change `require`
[11ty]       to use `import` and rename the file to have an `.mjs` file extension).
[11ty] 
[11ty]    3. (More work) Change your project to use ESM-first by adding `"type": "module"` to your package.json. Any
[11ty]       `.js` will need to be ported to use ESM syntax (or renamed to `.cjs`.)
[11ty] 
[11ty]    4. Upgrade your Node version (at time of writing, v22.12 or newer) to enable this behavior. If you use a version
[11ty]       of Node older than v22.12, try the --experimental-require-module command line flag in Node. Read more:
[11ty]       https://nodejs.org/api/modules.html#loading-ecmascript-modules-using-require

• I also added permissions above on clause. To let GitHub Action bot creates gh-pages branch from main branch's contents, the bot must have enough permission to make modifications of my repository. Without enough permission configuration, the workflow gets an error like below. (403 forbidden)

Push the commit or tag
  /usr/bin/git push origin gh-pages
  remote: Permission to KnightChaser/music.git denied to github-actions[bot].
  fatal: unable to access 'https://github.com/KnightChaser/music.git/': The requested URL returned error: 403
  Error: Action failed with "The process '/usr/bin/git' failed with exit code 128"

3. I went to the repository's related setting([repository settings] > [Actions] > [General] > [Workflow permissions]) and switched out to the option " Read and write permissions".

After that, the pages are built as desired, on the branch gh-pages.

Finally, I could set the page's branch configuration to "branch: gh-pages, location: /(root)", like below.

Now, when I access to the desired URL(<username>.github.io/<repository_name>), I don't get 404 error anymore since the page contents are well prepared to serve.

In short, everything was because of the typo (paragraph 1) that I couldn't find in the midnight in fatigue.