A modification to @Carlo Zanocco's answer:

This solution didn't really work for me, and there were a couple of key changes I made to make it look like the solution desired, described in the comments below.

# It's good practice to import what you need, instead of getting the entirety of openpyxl and trying to navigate to them
from openpyxl import load_workbook
from openpyxl.chart import LineChart, Reference

filename = "chart.xlsx"

workbook = load_workbook(filename)
sheet = workbook["chart"]
chart = LineChart()

# This makes the x axis and y axis values actually appear
chart.x_axis.delete = False
chart.y_axis.delete = False
chart.legend = None # This removes the ugly legend from the right of the graph

chart.title = "Chart Title"

# Changed to min_row=2 so the row value can match with the row label 
#                                                           \/
labels = Reference(sheet, min_col=1, min_row=2, max_row=sheet.max_row) 
data = Reference(sheet, min_col=2, min_row=2, max_row=sheet.max_row)

chart.add_data(data, titles_from_data=False)
chart.set_categories(labels)

# Right now, the values are all technically in their own series, hence being labeled separately. 
# This makes them all the same color, so we can pretend they're the same line
for s in chart.series:
    s.graphicalProperties.line.solidFill = "4472c4" # Microsoft Blue

sheet.add_chart(chart, "D1")
workbook.save(filename)

Before and after

(big credit goes to this thread, which got me on a great path to solving all my problems: https://groups.google.com/g/openpyxl-users/c/khC6BTqaH3Y)