Per Polars Support:
First, you need way more iterations than just 100 for such a small timewindow. With 10,000 iterations I get the following:
avg polars run: 0.0005123567976988852 avg pandas run: 0.00012923809615895151 But we can rewrite the polars query to be more efficient:
df_filtered = ( df.lazy() .with_columns(abs_diff = (pl.col.column_0 - target_value).abs()) .filter(pl.col.abs_diff == pl.col.abs_diff.min()) .collect() )
Then we get:
avg polars run: 0.00018435594723559915 Ultimately Polars isn't optimized for doing many tiny tiny horizontally wide datasets though.
Unfortunately, I didn't experience much of a performance boost when I tried the version above. It does seem the speeds are very machine dependent. I will continue with pandas for this specific use case. Thanks all for looking.