Turned out RedirectResponse didn't contain the cookies header, because we set them in response. This is the correct version of the code:

@router.post("/login")
async def login(response: RedirectResponse, credentials: UserLoginSchema = Form()):
    if credentials.email == ADMIN_EMAIL and credentials.password == "123":
        token = auth.create_access_token(uid=credentials.email)
        redirect_response = RedirectResponse(url="/", status_code=status.HTTP_302_FOUND)
        redirect_response.set_cookie(
            key=config.JWT_ACCESS_COOKIE_NAME,
            value=token,
        )
        return redirect_response
    
    raise HTTPException(401, detail={"message": "Invalid credentials"})

Thanks to C3roe's comment for a lead.