I had been struggling with this for so long, but I have found a very good solution to this, which suits my needs.
I will add all the information I had to go around look for so it's easier for others, just in case.
It is a combination of these two answers, so the respective credit goes to them:

https://stackoverflow.com/a/57775666/28946680
https://stackoverflow.com/a/44411205/28946680

This works for WIndows Powershell or any similar shells which are built on top of that and can use their own profiles.

To make this, you first need to find out where your profiles are stored for WindowsPowershell
It will be in either of these locations:
1. $UserHome\Documents\WindowsPowerShell\
2. $UserHome\One Drive\Documents\WindowsPowerShell\ (This second one will be your path if you have one drive configured)

Look at this article for more information: https://learn.microsoft.com/en-us/powershell/module/microsoft.powershell.core/about/about_profiles?view=powershell-7.5

Then you can make your script with the name profile.ps1

function Write-BranchName () {

    $git_cmd = "git rev-parse --abbrev-ref HEAD"
    Invoke-Expression $git_cmd 2> $null | Tee-Object -Variable git_branch | Out-Null
    $git_branch_text = $None
    
    # If we are inside a git repo
    if  ($git_branch -And -Not $git_branch.StartsWith($git_cmd)) {

        if ($git_branch -eq "HEAD") {
            $git_branch = git rev-parse --short HEAD
            if ($git_branch) {
                # we're probably in detached HEAD state, so print the SHA
                Write-Host " ($git_branch)" -NoNewline -ForegroundColor "red"
            } else {
                # we'll end up here if we're in a newly initiated git repo
                Write-Host " (no branches yet)" -NoNewline -ForegroundColor "yellow"
            }
        }
        else {
            # we're on an actual git_branch, so print it
            Write-Host " ($git_branch)" -NoNewline -ForegroundColor "blue"
        }
    }
}

function prompt {
    $base = "PS "
    $path = "$($executionContext.SessionState.Path.CurrentLocation)"
    $userPrompt = "$('>' * ($nestedPromptLevel + 1)) "

    Write-Host "`n$base" -NoNewline

    Write-Host $path -NoNewline -ForegroundColor "green"
    Write-BranchName

    return $userPrompt
}

This handles the case for if there's no current branch, no branch selected as well a particular commit in different colours (The colours are easily configurable from the code)

The screenshots can be found in the above link where I took this from.