Ok. I finally figured out how to fix this issue. I had to rewrite the code as follows:
# Iterate through all .xlsx files in the current directory
for file in os.listdir(script_dir):
if file.endswith('.xlsx'):
# Load the spreadsheet
df = pd.read_excel(file)
# Remove rows with "cancelled by system" or "cancelled by trader" in any cell
df = df[~df.apply(lambda row: row.astype(str).str.contains('cancelled by system|cancelled by trader|Maximum position exceeded', case=False).any(), axis=1)]
# Remove unwanted columns: A, B, C, D, K, Q
df = df.drop(df.columns[[0, 1, 2, 3, 4, 6, 7, 8, 10, 11, 12, 16]], axis=1)
# Save the updated spreadsheet without header\title row
output_file_path = os.path.join(output_folder, file)
df.to_excel(output_file_path, index=False, header=False)
print("Reformated data in xlsx files")