I agree with XQ Hu. You might consider exploring the Bigtable Beam Connector (BigtableIO), which allows you to perform both batch and streaming operations on Bigtable data within a Dataflow pipeline. Additionally, you can create a custom container image using Docker to launch containerized SDK processes.