• What happens if my ADF jobs start to exceed the interval and offset? Will they queue up?

AFAIK, as you are using Self-Dependency tumbling window triggers, the current trigger run won't start until the previous trigger is completed. Even if the current pipeline run fails, the queues will wait for the dependency trigger run to get succeeded. By this way it will ensure that current operation executed only after the successful completion of its previous run.

What do I need to do to "clear" a queue? Do I need to redeploy the pipeline?

As per this Github issue by @jrimada,

If you want to clear the queue, you can directly cancel the current trigger run. To do this, go to monitor -> Trigger runs -> Tumbling window triggers -> current running trigger run -> cancel.

This will ensure that it won't trigger the pipeline run anymore.

If there are any pipeline failures, you can stop the next trigger runs and stop the trigger and update the pipeline as per your requirement.

You can refer this documentation and this SO answer by @Alain to know more about the Dependency of the Tumbling window triggers.