For debug and release mode, use this way until Flutter provides an official way

if (g_file_test("assets", G_FILE_TEST_IS_DIR))
  {
    gtk_window_set_icon_from_file(window, "assets/images/icon.png", NULL); // For debug mode
  }
  else
  {
    gtk_window_set_icon_from_file(window, "data/flutter_assets/assets/images/icon.png", NULL); // For release mode
  }

You can also try with hover https://github.com/flutter/flutter/issues/53229#issuecomment-660040567