Ok, I think I reached a satisfactory solution. I realised I need to do two things:

1. Generate the static website using schemaspy
2. Run a web server to serve the static website so I can actually access it using the ip and the port

So I created two docker containers; one to do each of these steps. I would prefer if I can do both these things in one container, but I kept running into errors trying to achieve that so I'm letting it be like this for now.

Here's my current working setup in docker-compose.yaml:

    docs:
        image: schemaspy/schemaspy:latest
        container_name: Docs
        environment:
            DB_USER: root
            DB_PASS: pass111
            DB_NAME: EduSys
            DB_PORT: "1026"
            DB_HOST: 138.150.210.254
        depends_on:
            - "mysql"
        volumes:
            - schemaspy-data:/output
        entrypoint: >
            sh -c "schemaspy -t mysql -host 138.150.210.254 -port 1026 -db EduSys -s EduSys -u root -p pass111"

    zcedudocs_web:
        image: nginx:latest
        container_name: nginx
        ports:
            - "5558:80"
        volumes:
            - schemaspy-data:/usr/share/nginx/html:ro
        depends_on:
            - docs
    mysql:
        image: mysql:8.2
        container_name: MySQL
        environment:
            MYSQL_ROOT_PASSWORD: pass111
            MYSQL_OPTIONS: --sort_buffer_size=2M
        ports:
            - "1026:3306"
        volumes:
            - mysql_db:/var/lib/mysql

    volumes:
       mysql_db:
       schemaspy-data:

I used docker compose up -d to run everything, and it works now! Just remember to use http not https. And the url looks like this http://138.150.210.254:5558/index.html

Note: I can still see the same Graphviz errors if I use docker logs Docs, but turns out it doesn't actually prevent the website generation. I still don't know why it doesn't go away when I use -nohtmlimages and -nodot. Any input on that would be appreciated.