In your code, parent method Processor.excute() is annotated with @Transactional which is responsible for rollback of the method execution in case of any exceptions. But @Transactional is also added on both the children service methods StudentHandler.process() and TeacherHandler.process(). This will start separate transaction for the child methods and may not get rolled back when the parent method initiates rollback.
You can remove @Transactional from both the child methods.
@Transactional(rollback= CustomException.class) can be added only on the Parent method.
Also, all DAO methods have to be configured with default propogation (Required) to get rolled back along with parent transaction.
Refer to Transaction Management using @Transactional :