Is it producing any exceptions or is it running completely?

make sure the file is deleted after delete line using:

Debug.Assert(!File.Exists(filePath), "File was not deleted successfully.");

Also check if the file is in read only mode, you can change that using like this:

var fileInfo = new FileInfo(filePath);
if (fileInfo.Exists && fileInfo.IsReadOnly)
{
    fileInfo.IsReadOnly = false;
}