According to this documentation:
https://en.cppreference.com/w/cpp/container/map/emplace_hint.html
which is not exactly the std::set function member you wanted, but it stand.
You are on the right direction.

Anyway, it is better to emplace every new member to speedup the execution. If you don't do that, at every insertion the compiler will create a new temporary element that lasts just the time to be copied into your std::map<int, std::set<int> > container.

So your problem was not the complexity of the insertion itself, but the execution time for creating the objects. Emplace the new elements of your containers if they are not trivial!