79297964

Date: 2024-12-20 17:25:36
Score: 1
Natty:
Report link

it depends on what you do, and how many columns data have.

  1. Batch Job in data of many columns -> Parquet, Polars is best. it can read few of specific columns you need. it cnsumes less memory. So, faster.

  2. Batch Job in data of few columns -> choose the more comfortable for you. But, if your machine's performance(CPU, memory) is fairly lower than server, I think database would be faster.

  3. Batch Job in data of Large columns -> Server(Database) would be faster. But, if the server already stored data in parquet, Polars would be faster.

  4. ACID Job with single transaction -> only Database

Reasons:
  • Long answer (-0.5):
  • No code block (0.5):
  • Low reputation (1):
Posted by: Sung Eun An