Let RecyclerView handle views ,you only handle data and then notice recyclerview data are changed.

adapterNotCheckedList = RecyclerViewListAdapter(clickListener = { view: View, listItem: Any, position: Int ->
    //Here, besides visibility i want to move it to the top everytime unCheckedList gets smaller.
    checkedList.add(unCheckedList.removeAt(position))
    // you need get adapter.
    topAdapter.notifyItemRemoved(position)
    bottomAdapter.notifyItemInserted(checkedList.lastIndex)
})

if unchecked list is too large ,you will only see top recyclerview.