I was having trouble updating my RecyclerView after scanning or importing a new PDF document. The logs were showing that the new document was being loaded, but the UI wasn't reflecting these changes. Here's a breakdown of the problems I faced and how I addressed them:

Problem:

1. UI Not Updating After Scan/Import: The RecyclerView was not reflecting new documents after scanning or importing despite logs showing successful loading.
2. Delayed Updates: The UI was not updating instantly, it was lagging.
3. DiffUtil Not Called: The DiffUtil was not being called when a new document was added to the RecyclerView.

Solutions:

• To fix this I load the documents inside the onResume() method of my HomeFragment, and in onActivityResult() of my HomeFragment and MainActivity, which means that the list of document is loaded when the user is coming back to the screen or when a new PDF is imported or scanned.
• Also I was updating the adapter outside the main thread, which was causing issues with the animations, to fix this I wrapped the adapter list update in withContext(Dispatchers.Main) to make sure that the updates are done in the main thread.

Code Snippet:

HomeFragment.kt

@RequiresApi(Build.VERSION_CODES.R)
    override fun onResume() {
        super.onResume()
        //Use a coroutine to import the PDFs asynchronously
        loadDocumentsWithAnimation() // Load documents with animation
        requireActivity().onBackPressedDispatcher.addCallback(
            this, true // Handle even if other fragments have higher priority
        ) {
            onBackPressed()
        }
    }

    @Deprecated("Deprecated in Java")
    override fun onActivityResult(requestCode: Int, resultCode: Int, data: Intent?) {
        super.onActivityResult(requestCode, resultCode, data)
        //Use a coroutine to import the PDFs asynchronously
        if (requestCode == PdfImportUtils.REQUEST_CODE_IMPORT_PDF && resultCode == Activity.RESULT_OK) {
            val urisToImport = if (data?.clipData != null) {
                (0 until data.clipData!!.itemCount).map {
                    data.clipData!!.getItemAt(it).uri
                }
            } else {
                listOfNotNull(data?.data)
            }
            //Use a coroutine to import the PDFs asynchronously
            uiScope.launch {
                progressIndicator.visibility = View.VISIBLE
                withContext(Dispatchers.IO){
                    val importJobs =    urisToImport.map { uri ->
                        async { PdfImportUtils.importPdfFromUri(requireContext(), uri, documentViewModel) }
                    }
                    importJobs.awaitAll()
                }
                withContext(Dispatchers.Main){
                     loadDocumentsWithAnimation() // Load documents with animation
                }
                  progressIndicator.visibility = View.GONE
            }
            try {
                analyticsManager.logEvent(eventName = "pdf_imported", params = Bundle().apply {
                    putString("file_uri", data?.data?.toString() ?: "Multiple Files")
                })
            }catch(e:Exception){
                Timber.e(e, "Exception in pdf_imported:")
            }
        } else {
            try {
                analyticsManager.logEvent(eventName = "pdf_import_failed", params = Bundle().apply {
                    putInt("request_code",requestCode)
                    putInt("result_code", resultCode)
                })
            }catch(e: Exception){
                Timber.e(e,"Exception in pdf_import_failed:")
            }
        }
    }

MainActivity.kt

   private fun Context.savePdfToAppStorage(pdfUri: Uri) {
        try {
            // Get the directory for app's files
            val appFilesDir = filesDir

            // Find the next available filename with auto-increment
            val newFileName = createUniqueFileInDirectory(appFilesDir)
            // Create the output file path
            val outputFile = File(appFilesDir, newFileName)

            val inputStream = contentResolver.openInputStream(pdfUri) ?: return
            val outputStream = FileOutputStream(outputFile)
            inputStream.copyTo(outputStream)
            Timber.d("PDF saved successfully to: $outputFile")
            Toast.makeText(this, "PDF saved as: $newFileName", Toast.LENGTH_SHORT).show()
            uiScope.launch{
                documentViewModel.refreshDocuments(this@savePdfToAppStorage)
            }
        } catch (e: Exception) {
            Timber.e(e, "Error saving PDF to App Storage:")
        }
    }

By using this, the list is correctly updated and the DiffUtil is called which animates the list correctly.

Addressing the Lagging UI:

• To fix this I had to make sure all the reads from the file system are done in an IO thread which I achieved by using withContext(Dispatchers.IO), this ensures that the UI is not blocked and is updated smoothly, then update the UI in the main thread using withContext(Dispatchers.Main).

Code Snippet: (from HomeFragment's loadDocumentsWithAnimation()):

    private fun loadDocumentsWithAnimation() {
        documentViewModel.viewModelScope.launch {
            progressIndicator.visibility = View.VISIBLE
            val documentData = documentViewModel.loadAllDocumentsFromStorage(requireContext())
            withContext(Dispatchers.Main) {
                recentDocumentsAdapter.updateRecentDocuments(documentData.second)
                recentDocumentsVisible = documentData.second.isNotEmpty()
                updateViewVisibility()
            }
            progressIndicator.visibility = View.GONE
        }
    }

Thanks to everyone how commented on the question!