If anyone is facing this issue specific to one drive folder.
You can loop through and delete all the files inside the folder, but while trying to delete folder seems to be causing this issue in One drive location.