The reason was my lack of knowledge how Apple handles installed apps.
They are actually folders, where the executable is somewhere within. Treating the dropped app as a folder solved the problem.