You can try using a basic Convolutional Neural Network program for the digit recognition. Using MNIST to demonstrate how a CNN can identify patterns is the most basic application of a CNN. It's so basic that it's taught in courses on Artificial Intelligence. You can search on Google or use one of these links

If you're allergic to clicking on links (like I am) here's a basic explanation:

You probably already know what a Neural Network is. You probably also know what a CNN is. You can simply build a CNN using the tensorflow library for the various layers, then mix and match the layers as to your liking. For the text input, just use OpenCV and PyTesseract for scanning an image and extracting text using OCR

Here's a sample code for you (for the OCR)

import cv2
import pytesseract
pytesseract.pytesseract.tesseract_cmd = r'C:\Program Files\Tesseract-OCR\tesseract.exe'
import matplotlib.pyplot as plt

def extractTextFromID(image_path):
    """

    Parameters
    ----------
    image_path : str
        Path to the image that is going to be analysed. Extracts all read text
        NOTE: vertically arranged text may not be read properly

    Returns
    -------
    extracted_text : str
        str of all the text as read by the function.

    """
    img = cv2.imread(image_path)
    image_rgb = cv2.cvtColor(img, cv2.COLOR_BGR2RGB)

    plt.figure(figsize=(10, 6))
    plt.imshow(image_rgb)
    plt.title("Original Image")
    plt.axis("off")
    plt.show()
    
    extracted_text = pytesseract.image_to_string(image_rgb)
    return extracted_text

This code returns the extracted text as a single large string. If you just want to read from the image directly you can just use a CNN:

from keras.models import Sequential, load_model
from keras.layers import Dense, Dropout, Flatten
from keras.layers import Conv2D, MaxPooling2D, BatchNormalization
from keras.optimizers import Adam


model = Sequential()

model.add(Conv2D(filters=32, kernel_size=(3, 3), activation='relu', strides=1, padding='same', data_format='channels_last',
                 input_shape=(28,28,1)))
model.add(BatchNormalization())
model.add(Conv2D(filters=32, kernel_size=(3, 3), activation='relu', strides=1, padding='same', data_format='channels_last'))
model.add(BatchNormalization())
model.add(MaxPooling2D(pool_size=(2, 2), strides=2, padding='valid' ))
model.add(Dropout(0.25))

model.add(Conv2D(filters=64, kernel_size=(3, 3), activation='relu', strides=1, padding='same', data_format='channels_last'))
model.add(BatchNormalization())
model.add(Conv2D(filters=64, kernel_size=(3, 3), strides=1, padding='same', activation='relu', data_format='channels_last'))
model.add(BatchNormalization())
model.add(MaxPooling2D(pool_size=(2, 2), padding='valid', strides=2))
model.add(Dropout(0.25))

model.add(Flatten())
model.add(Dense(512, activation='relu'))
model.add(BatchNormalization())
model.add(Dropout(0.25))
model.add(Dense(1024, activation='relu'))
model.add(BatchNormalization())
model.add(Dropout(0.5))
model.add(Dense(10, activation='softmax'))

#Optimizer
optimizer = Adam(lr=0.001, beta_1=0.9, beta_2=0.999 )
#Compiling the model
model.compile(optimizer=optimizer, loss="categorical_crossentropy", metrics=["accuracy"])

#defining these prior to model to increase readability and debugging
batch_size = 64
epochs = 50
# Fit the Model
history = model.fit_generator(datagen.flow(x_train, y_train, batch_size = batch_size), epochs = epochs, 
                              validation_data = (x_test, y_test), verbose=1, 
                              steps_per_epoch=x_train.shape[0] // batch_size,
                              callbacks = [reduce_lr]) 

Let me know if you need additional help