How can I optimize this further for better performance and lower memory allocation?
There are no obvious algorithmic improvements I can spot. So improvements will likely come down to microoptimizations and possibly parallelism.
A good first step would be to run the code in the built in visual studio profiler to see if you can spot anything unexpected. A second step would be to inspect the assembly code. Even if you know nothing about assembly, just checking the number of instructions should give you some idea about the runtime.
If you really want to micro optimize you might want to use a more detailed profiler like vTune.
Parallelism is easiest to exploit if you have different messages. But if you have a single large message composed of a very large number of key-value pairs you could probably just create evenly distributed indices, find the separator following each of these indices, and use that to split your string between threads.
Is there a faster or more efficient algorithm to extract key-value pairs from a FIX message?
Your algorithm looks linear to me, and that is the best complexity that can be had.
Are there alternative data structures or parsing techniques that would improve lookup speed while keeping allocations low?
Depends on your requirements, if you need a dictionary containing strings you will have allocations. If you have an upper bound of key sizes you might be able to use a plain array. Possibly containing Memory<char>
to avoid string allocations.
Would SIMD, Span-based parsing, or other low-level optimizations be beneficial in this case?
The main use case for SIMD would be IndexOf
. I'm not sure if this has been SIMD optimized or not in your version of .net. This should be fairly obvious if you inspect the assembly.
Are there any other approaches or data formats that could improve parsing and performance for high-throughput FIX messages?
You might be able to get better performance from C/C++, but in my experience that only helps if you have a very good understanding of both the language and the underlying hardware.