From the view of the project Eclipse/Californium (Scandium), that will be a very bad idea and may end up in a denial of service. You will at least need something to filter that incoming "TOFU" handshakes, otherwise anyone may use Californium's Benchmark client to fill up your device stores very fast.

In my case it's long ago that I was up-to-date with LwM2M, but in order to have something implemented in Leshan it will be much easier if that is part of the spec.

What I implemented in Eclipse/Californium in order to an auto-provisioning is to use a specific, temporary key to establish a dtls connection, which only allows to call the "add device credential" API. The idea is to generate a key-pair and use that for a "production charge", which at the "end of line" does an functionality check and execute that auto-provisioning.

Anyway, regardless which way you go, you will need something additional in order to prevent a DoS from provisioning with "TOFU".