I need to get better at reading!

It's not highlighted very well, but the docs explain that you must import their base css file in the file that contains your calendar component. Specifically:

When you include Big Calendar in your interface, you will need the core styles. We provide a precompiled style sheet (react-big-calendar/lib/css/react-big-calendar.css) for you, or you can directly import the SASS into your implementation.

Simply adding import "react-big-calendar/lib/css/react-big-calendar.css"; to my file fixed my issues. Thanks @DaveNewton for pointing this out.