From experience - it is not so simple. You need to have take into account:

1. engine used for inference (TGI? pure transformers? llama-cpp)
2. card type (really it matters whether it is H100 or L40S or A100)
3. batch size
4. is it a chatbot like experience or maybe you need to process offline?
5. What is the maximum context you would like to process?

On basis of this you need to run some benchmark and generalize it