As suggested by @furas, we can use git url.
ubuntu@ubuntu:~/hello$ poetry add scikit-learn git+https://github.com/scikit-learn-contrib/imbalanced-learn.git Creating virtualenv hello in /home/ubuntu/hello/.venv Using version ^1.7.1 for scikit-learn
Updating dependencies Resolving dependencies... (0.1s)
Package operations: 6 installs, 0 updates, 0 removals
Writing lock file ubuntu@ubuntu:~/hello$