Answering here, incase any one else facing the same issue.

1. The HttpSession was not getting invalidated cause the JSESSIONID was not coming in the logout request due to the SameSite changes happened. So need to configure this.
2. The Logout response was not being sent cause of the X-Frame-Options: Deny set by the spring security. https://docs.spring.io/spring-security/reference/features/exploits/headers.html#headers-frame-options so cause of this it was not setting the responses in my case.