I'm currently having the same issue. After researching i found that Github Classroom autograder only shows the markers after the student submission when all tests pass. Otherwise, even though the student might pass 4/5 tests, it is considered a fail, therefore the markers won't show up. Using python-grader module will solve this. If you still want to manage your own custom test suite, you might want to take a look at this discussion: (it's old, not everything can be considered) https://github.com/orgs/community/discussions/77361 https://github.com/classroom-resources/autograding-python-grader