79191870

Date: 2024-11-15 10:03:22
Score: 3
Natty:
Report link

I am assuming you are referring Spark static streaming joins. In Spark Streaming, when joining a streaming DataFrame with a static DataFrame, the static DataFrame remains unchanged throughout the streaming query's execution. This means that any updates or changes to the underlying data source of the static DataFrame won't be reflected in the join results unless the programmed for the same.

Please refer Stack Overflow discussion here - https://stackoverflow.com/questions/66154867/stream-static-join-how-to-refresh-unpersist-persist-static-dataframe-periodic#:~:text=You%20can%20trigger%20the%20refreshing,that%20refreshes%20the%20static%20Dataframe

Reasons:
  • Blacklisted phrase (1): stackoverflow
  • RegEx Blacklisted phrase (0.5): any updates
  • No code block (0.5):
  • Low reputation (1):
Posted by: Gurunandan Rao