The issue was fairly simple: I forgot to set the protocol in the broker and backend, i.e. redis://a-redis-server.abcdef.0001.use1.cache.amazonaws.com:6379/0. Seems a bit obvious in retrospect lol.
broker
backend
redis://a-redis-server.abcdef.0001.use1.cache.amazonaws.com:6379/0