Thanks everyone for the answers!
Turns out it was me being silly; I hadn't moved the .htaccess
file into the new document root, public. I also had to change the .htaccess
rule a little bit, as it was rewriting requests to be index.php/request/here:
RewriteEngine On
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule ^ index.php [QSA,L]