Due to Firefox updating its security module, FirefoxDriver in Selenium has become legacy. You now need to use geckodriver as a local and remote communication server. (See more details). As a result, the setup and containerization process is more complex, with additional constraints to consider. If you only need simple automation, Chrome might be a more convenient choice.

Solution

The issue mainly stems from FirefoxDriver not initializing correctly in Selenium. About deploying on AWS Fargate, you might need to open a separate question. Furthermore, I can't access the site in your question. Based on your description(purpose, python version, webdriver in using) I'll focus on how to:

1. Containerized Selenium setup using Firefox
2. Access stackoverflow page
3. Locate user page button and click
4. Take a screenshot to verify success

Folder Structure

-- SO-79483362
 |-- Dockerfile
 |-- README.md
 |-- script.py

Dockerfile

FROM ubuntu:20.04
ARG DEBIAN_FRONTEND=noninteractive

# Install dependencies and driver
RUN apt-get update \
  && apt install --no-install-recommends -y curl wget build-essential gpg-agent software-properties-common unzip firefox\
  && add-apt-repository ppa:deadsnakes/ppa \
  && apt install -y python3.9-venv python-is-python3 python3-pip \
  && ln -sf /usr/bin/python3.9 /usr/bin/python \
  && ln -sf /usr/bin/python3.9 /usr/bin/python3 \
  && wget https://github.com/mozilla/geckodriver/releases/download/v0.36.0/geckodriver-v0.36.0-linux64.tar.gz \
  && tar -xvf geckodriver-v0.36.0-linux64.tar.gz \
  && chmod +x geckodriver \
  && apt clean \
  && rm -rf /var/lib/apt/lists/*

ENV PATH="${PATH}:/geckodriver" \
    PYTHONPATH=/usr/lib/python3.9/site-packages:${PYTHONPATH}

# Copy your project files
COPY . /app

# Set the working directory
WORKDIR /app

# Install Python dependencies
RUN pip install selenium==3.141.0
RUN pip install urllib3==1.26.16
RUN pip install webdriver-manager

VOLUME ["/app"]

CMD ["python3", "script.py"]

script.py

from selenium import webdriver
from selenium.webdriver.firefox.options import Options
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.by import By
from selenium.webdriver.common.action_chains import ActionChains

# Launch Driver
firefoxOptions = Options()
firefoxOptions.binary_location = r"/usr/bin/firefox"
firefoxOptions.add_argument("--headless")
firefoxOptions.add_argument("--no-sandbox")

driver = webdriver.Firefox(
    executable_path=r"/geckodriver",
    options=firefoxOptions
)

# Access StackOverflow and take a screenshot
driver.get("https://stackoverflow.com/")
driver.get_screenshot_as_file("Homepage.png")
wait = WebDriverWait(driver, 20)

# Click user page navigation button and take a screenshot
users_page_btn = wait.until(EC.element_to_be_clickable((By.ID, "nav-users")))
users_page_btn.click()
driver.get_screenshot_as_file("Users.png")

driver.quit()

Run the following commands

# Build docker image
docker build -t selenium-docker .

# Run docker container
docker run --rm -d -p 4444:4444 -v .:/app selenium-docker

Expected Result See two pictures in the SO-79483362 folder:

1. Homepage.png: "Open the broswer" and "take a shot".
2. Users.png: "Click user page navigation button" and "take a shot".