The dimensions of the output from AveragePooling2D can differ from the dimensions of the input, since default value for padding is 'valid'. Maybe setting padding='same' would fix your issue.