Update 2024.10. Something looks change based on the new IDE. Here is the setup page below:

File -> Settings -> Languages & Frameworks -> Jupyter -> Markdown font scale

You need to change the percentage to increase or decrease the font size of your markdown font size(like what I did to set it up as 150% to make the font 1.5 times than the original.)