I was able to do it using the SDKv2 and the following code:

from azureml.core import Workspace, Datastore, Dataset
import pandas as pd

pandas_df = pd.read_csv('<path to your csv file>')
ws = Workspace.from_config()
datastore = Datastore.get(ws, '<name of your datastore>')
dataset = Dataset.Tabular.register_pandas_dataframe(pandas_df, datastore, "dataset_from_pandas_df", show_progress=True)

Official documentation for the code source: https://learn.microsoft.com/en-us/azure/machine-learning/how-to-create-register-datasets?view=azureml-api-1&viewFallbackFrom=azureml-api-2#create-a-dataset-from-pandas-dataframe