It looks like lightning creates a different runid for each invocation. However, if I noticed if I start a run (in my case by logging hyperparameters):
mlflow_logger = MLFlowLogger(experiment_name=MLFLOW_EXPERIMENT_NAME, tracking_uri=MLFLOW_TRACKING_URI, run_name=MLFLOW_RUN_NAME) mlflow_logger.log_hyperparams(hyperparams)
then lightning reuses the same runId for emitting metrics for fit and test stages.