I have tried using JLayeredPane to hold the splitpane/ tabbedpane/ Awt panel... but nothing worked.

Any suggestions or workaround is highly appreciated. Any suggestions on alternate way to get handle for JNI in this hierarchy is also highly appreciated.

It's mot a question of z-order or layer. AWT components have heavyweight peers that come from the operating system that Java is running on'S GUI toolkit. On the other hand, Swing is lightweight - defined in Java and not the OS. OS components always take precedence in screen display and with UI events.