Most of the data caching strategies in next.js server components are implemented as an extension of the native fetch function. The supabase SDK uses mostly postgres connections to get data so the caching will likely not work out of the box.
A good strategy would be to create an API route where you fetch your data from postgres and call this endpoint in your server component using fetch. Most of the magic happens in headers so this way you would have them implemented automatically.
https://nextjs.org/docs/app/building-your-application/data-fetching/fetching
Another alternative could be to use the cache function from react that is native to server components (not a next.js specific)
https://react.dev/reference/react/cache
This could wrap your call and provide caching.