After some experimenting, I found that the code generates the desired output without the need to write files to the file system, delayed expansion and without a for loop.

set myvar="aaa^<bbb"

set xxx=%myvar:<=^^<%
set zzz=%xxx:"=%
echo %zzz%

I would appreciate some comments with theories why the %myvar:<=^^^<% did not work and the %myvar:<=^^<% works.