According to this documentation, templates can also be defined under custom_dir (relative to the root directory--not the docs_dir) in the theme like this:

theme:
  name: material
  custom_dir: templates/

I'm still not sure what extra_templates is for, but once I configured theme:custom_dir it worked.