79119661

Date: 2024-10-23 20:47:58
Score: 0.5
Natty:
Report link

Usually it is better to append datetime to the filename when you get the same file. If you want to generate the sequence number in the filename here you go

The pipeline design look like below Pipeline design

  1. Get Metadata activity(Get Metadata Source) to read the source file from file share and configure it as shown below GetMetadata

  2. For Each activity The for each loop through the source file from the previous get metadata activity. The for each items expression under setting tab shown below

@activity('Get Metadata Source').output.childItems

Under for each activity tab add the below activities For Each Activity

  1. Lookup activity This activity will call a stored procedure and increment the sequence for filename and return the filename for the destination.

Create a table using below script

CREATE TABLE [dbo].[FileNameSequence](
    [OriginalFileName] [varchar](100) NOT NULL,
    [SequenceNo] [smallint] NOT NULL,
 CONSTRAINT [PK_FileNameSequence] PRIMARY KEY CLUSTERED 
(
    [OriginalFileName] ASC
)WITH (STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF, OPTIMIZE_FOR_SEQUENTIAL_KEY = OFF) ON [PRIMARY]
) ON [PRIMARY]
GO

Stored procedure script

SET ANSI_NULLS ON
GO
SET QUOTED_IDENTIFIER ON
GO

CREATE PROCEDURE dbo.GetFileNameSequence
(
    @FileName varchar(100)
)
AS
BEGIN
    
    SET NOCOUNT ON
    IF EXISTS
    (
        SELECT 1 FROM dbo.FileNameSequence
        WHERE OriginalFileName = @FileName
    )
    BEGIN
        UPDATE dbo.FileNameSequence
        SET 
        SequenceNo = SequenceNo+1
        WHERE OriginalFileName = @FileName

        SELECT SUBSTRING(OriginalFileName,1,CHARINDEX('.',OriginalFileName)-1)+'_'+Cast(SequenceNo As varchar)+SUBSTRING(OriginalFileName,CHARINDEX('.',OriginalFileName),len(OriginalFileName)) AS FileName FROM dbo.FileNameSequence WHERE OriginalFileName = @FileName;

    END
    ELSE
    BEGIN
        INSERT INTO dbo.FileNameSequence([OriginalFileName],  [SequenceNo])
        VALUES(@FileName,0)

        SELECT OriginalFileName AS FileName FROM dbo.FileNameSequence WHERE OriginalFileName = @FileName;

    END

    
END
GO

Configure the lookup activity to call the stored procedure which return the filename for destination as shown below

Lookup activity

  1. Copy activity Copy activity take the filename output and move the file to destination

Source settings Sourcesetting

Sink Setting

The sink parameter pmFile expression is

@activity('GetFileName').output.firstRow.FileName

sinksetting

I tested the initial run with 2 input files the pipeline ran sucessfully and copied the file as it is as there is no file in the destination.

Pipelineinitial run

the table which generate the sequence would look like this

Initial run table

the Source(before execution) and destination(after execution) for initial run shown as below Initialrun

For the next run i added 2 same file and 1 different file

the table data look like this now

tabledatanext run

the Source(before execution) and destination(after execution) for next run shown as below

next run

For the subsequent run same files with same name will keep on appending the incremental sequence number.

Also please add a error logic if any error happened during file movement decrease the sequence counter for that file.

Hope this will work for you.

thanks

Reasons:
  • Blacklisted phrase (0.5): thanks
  • Probably link only (1):
  • Long answer (-1):
  • Has code block (-0.5):
  • Low reputation (0.5):
Posted by: SQL006