Feels a bit weird to find an answer after going through the trouble to document this question, but maybe someone can improve upon it. After finding out "Matchers" is already part of the Hamcrest library, I renamed it from my last post to "MatcherMethods". I found this question about child elements and stitched its answer together with the getText matcher from the answer here. The resulting renamed helper class looks like this:

public class MatcherMethods {

    public static String getText(final Matcher<View> matcher) {
        final String[] stringHolder = { null };
        onView(matcher).perform(new ViewAction() {
            @Override
            public Matcher<View> getConstraints() {
                return isAssignableFrom(TextView.class);
            }

            @Override
            public String getDescription() {
                return "getting text from a TextView";
            }

            @Override
            public void perform(UiController uiController, View view) {
                TextView tv = (TextView)view; //Save, because of check in getConstraints()
                stringHolder[0] = tv.getText().toString();
            }
        });
        return stringHolder[0];
    }

    public static int getChildCount(final Matcher<View> matcher, String className) {
        final int[] intHolder = { -1 };
        onView(matcher).perform(new ViewAction() {
            @Override
            public Matcher<View> getConstraints() {
                return isAssignableFrom(View.class);
            }

            @Override
            public String getDescription() {
                return "Getting count of child elements from a View for type " + className;
            }

            @Override
            public void perform(UiController uiController, View view) {
                ViewGroup group = (ViewGroup)view;
                int counter = 0;
                final int totalChildCount = group.getChildCount();
                for(int i = 0; i < totalChildCount; i++) {
                    //Count only if child element matches the desired type
                    if(group.getChildAt(i).getClass().getSimpleName().equals(className)) counter ++;
                }
                intHolder[0] = counter;
            }
        });
        return intHolder[0];
    }
}

The second matcher counts the child elements with the fitting className. Those matchers are used like this:

//Table interactions
        int tableId = context.getResources()
                .getIdentifier("randomInts","id",context.getPackageName());
        ViewInteraction table = onView(withId(tableId));

        table.check(matches(isDisplayed()));
        table.check(matches(hasMinimumChildCount(1)));
        Log.d("tag","Table is: " + table);

        int rowCount = MatcherMethods.getChildCount(withId(tableId),
                TableRow.class.getSimpleName());

        Log.d("tag","Rowcount is: " + rowCount);
        onView(allOf(
                withParent(withId(tableId)),
                withClassName(containsString("TableRow")),
                withParentIndex(rowCount - 1)
        )).perform(scrollTo(), click());

        //Extract table entry string
        String tableText = MatcherMethods.getText(allOf(
                withParent( allOf(
                        withParent(withId(tableId)),
                        withClassName(containsString("TableRow")),
                        withParentIndex(rowCount - 1)
                )),
                withClassName(containsString("TextView"))
        ));
        Log.d("tag", "Table entry is " + tableText);

My code has two issues now: First, if the table has children, but none of type TableRow. In that case, the withParentIndex(...) statements look for -1 and causes an error.Does Espresso have a matcher for that out of the box? Otherwise, I could use standard asserts. Second, I need to know the class name of the children. Looking simply for View.class finds 0 children. That can be resolved by creating a getChildCount method with only the first parameter and simply returning Viewgroup.getChildcount.