Maybe you can refer to this gist: RGB Color Mixing Algorithm: A Mathematical Approach to Digital Color Blending

Here's a real-world use case: Color Mixing Puzzle