Turns out we had different versions of Elasticsearch running for test and production for some reason. Test was on version 7 while production was on version 6. So i changed the versions of rest high level client and elastic search dependencies in pom.xml to version 7 and was able to get correct hits for testing by using hits.getTotalHits().value.