The error occurs because AWS Glue 3.0 PythonShell jobs have specific Python package version requirements that must be met. To fix this, you can create a requirements.txt file with compatible package versions. Then,

create a Python script that uses these packages

1. Create a Glue job with the following configuration

2. Upload your script to S3

3. Create the Glue job using AWS CLI