A couple issues off the top of my head:
When you call the function if the library is large it might slow down your algorithm. Whether it matters or not depends on the context and end user. This drawback could be a gain if the goal is to reduce initial script loading time.
If the library isn't installed your code may only fail when the function is called, which could cause a delay in failure. It is often better to fail as soon as the script is loaded so you immediately know there is a problem.
It's easier to read and debug code that adheres to formatting standards.
Potential linter implications.
In the end the drawbacks depend on the context entirely. I think the more important thing to consider is what you can accomplish by doing this, which often is very little.