Spectral.js is the best algorithm I found. MixBox is second place.

Comparing the two, when mixing blue (0,0,255) with yellow (255,255,0): spectral.js: 56, 143, 84 mixbox: 78, 150, 100

As you can see, spectral.js tends to be more vibrant and less grey. When I tested both of them side by side with multiple colors, spectral.js also felt a lot more natural and intuitive, mixbox felt a little disappointing and grey.

Spectral.js is only officially implemented in JS and Python, so I transcribed the script into C++.

Spectral.js still isn't perfect, though. I imagine the best algorithm would be one using supervised machine learning, if someone wanted to take the time to make that training data.