The VS Code Quick Pick item has its own custom sorting and filtering behavior. I set sortByLabel to false, matchOnDetail to true and added custom fields to the data component of my item which made it easier to search for.

Entirely custom logic has not been yet implemented in VS Code, check out this issue: https://github.com/microsoft/vscode/issues/90521#issuecomment-589829788