I found the following solution, which works in my use-case:

(funcall
 (lambda (x)
   (let ((a 2))
     (declare (special a))
     (funcall
      (lambda (x)
        (declare (special a))
        (+ a (* x 3)))
      x)))
 3)