Another thing to check is whether your tests modify the current directory (for example with `os.chdir`)

When running tests from the command line directly, tests changing dir work fine. When running them from vscode's UI, it breaks things apparently.