Ok, so here is the my answer to the question. I did not succeed to make diagram work with the beamer output (but the code is working with reealjs output, which was the original output). So at the end, this is what I did:

---
title: Test
author: 
  - John Due
format:
  beamer:
    pdf-engine: lualatex
    keep-tex: true
    header-includes:
      - \usepackage{tikz}
      - \usetikzlibrary{arrows.meta}
      - \tikzset{arrow/.style = {> = {Latex[length = 1.2mm]}}}
---

## Test
::: {.cell}

```{=latex}
\begin{tikzpicture}
  \node (i) at (0, 0) {i};
  \node (j) at (2, 0) {j};
  \draw[->, arrow] (i) -- (j);
\end{tikzpicture}
```

:::

Then, thanks to the comments of @SamR I could also fix the problems of nbformat and nbclient. Note that this was unrelated to the compilation problem of today (but I was also having problems with other documents...).