Reports

Ok here is what was happening.

This notation in CLI script did not work

  python preprocessing_script.py
  --Y_df ${{outputs.Y_df}} 
  --S_df ${{outputs.S_df}}

Thats because hydra does not like that notation (I think)

Instead this notation worked:

  python data_processing.py '+Y_df=${{outputs.Y_df}}' '+S_df=${{outputs.S_df}}'

What this does is that it adds those 2 new variables - Y_df and S_df into the config file These variables can be accessed in the program just like all other variables in the config file by doing cfg.Y_df or cfg.S_df

79363322