One can also look for the file uvernum.h and inside it there is a constant called U_ICU_VERSION.

For example in Linux:

locate uvernum.h | xargs grep U_ICU_VERSION

Outputs the following result (after filtering non relevant lines):

/usr/include/unicode/uvernum.h:#define U_ICU_VERSION "74.2"