Silly mistake, I needed to feed the blob name I wanted to create into .get_blob_client. And I didn't need to create blob_service_client = BlobServiceClient(client)
Amended code:
blob_name = "testCsv.csv"
blob_client = container.get_blob_client(blob_name)
blob_client.upload_blob(csv_bytes, overwrite=True)