Thank you very much @β.εηοιτ.βε. Your answer pointed me to the good documentation.

And I found vars[item] to complete my research (here Ansible: how to construct a variable from another variable and then fetch it's value).

Thus my last version, that works with a list of variables :

---
- name: Playbook to try assert module
  hosts: all
  gather_facts: false

  vars:
    mandatories:
      - variableA
      - variableB
      - variableThatDoesNotExist

  tasks:
    - name: Check mandatory variableA
      assert:
        quiet: false
        that:
          - variableA is defined
          - variableA | length > 0

    - name: Check mandatory variableB
      assert:
        quiet: false
        that:
          - variableB is defined
          - variableB | length > 0

    - name: Check mandatory variables with loop
      assert:
        quiet: false
        that:
          - vars[item] is defined
          - vars[item] | length > 0
      loop: "{{ mandatories }}"

    - meta: end_play