cloud-dataflow

Exporting data from Storage to Memorystore using Cloud Dataflow

Recently, I got a chance to do an R&D on a requirement where I would need to read files stored in a Cloud Storage bucket, which would be processed and transformed in the desired format and stored in an in-memory data store, i.e., Memorystore for faster access. Well, honestly, it took several days to figure […]

Exporting data from Storage to Memorystore using Cloud Dataflow Read More »

Cloud Dataflow: A Unified Model for Batch and Streaming Data Processing

Dataflow is a fully managed service to execute pipelines within the Google Cloud Platform ecosystem. It is a service that is fully dedicated to transforming and enriching data in stream (real-time) and batch (historical) modes. It is a serverless approach where users can focus on programming instead of managing server clusters, can be integrated with

Cloud Dataflow: A Unified Model for Batch and Streaming Data Processing Read More »