Yes - you can absolutely use Node.js + Express.js without a template engine. Template engines (like EJS, Pug, or Handlebars) are just convenience tools for embedding dynamic data into HTML, but they're not mandatory.
Serve static HTML files directly
Send raw HTML with res.send ()
Send JSON data to frontend JavaScript