I found that Marshal.ReleaseComObject can release the Marshaled data, Modified the Managed C# code like below, now the out proc exe is closing after release.

object obj;
 int hr = Ole32.CoCreateInstance(Constants.MyImplClassGuid,
     IntPtr.Zero, Ole32.CLSCTX_LOCAL_SERVER, typeof(IMyInterface).GUID, out obj);
 if (hr < 0)
 {
     Marshal.ThrowExceptionForHR(hr);
 }

 var server = (IMyInterface)obj;
 string output = server.MyMethod();

 Marshal.ReleaseComObject(obj);