Google Cloud Data Fusion

What Google Cloud Data Fusion is

Google Cloud Data Fusion is a managed cloud service for building and running data integration pipelines. Its visual point-and-click interface supports code-free ETL and ELT workflows to cleanse, prepare, blend, transfer, and transform data. More than 150 preconfigured connectors and transformations are included at no additional cost, and lineage tracking follows integrated data at dataset and field level to support root cause and impact analysis.

The service integrates with Cloud Storage, Managed Service for Apache Spark, BigQuery, and Spanner. Its replication feature can send changes from SQL Server, Oracle, and MySQL databases into BigQuery, and it integrates with Datastream for change streams. Security integrations include IAM, Private IP, VPC Service Controls, and customer-managed encryption keys. REST APIs, schedules, pipeline state triggers, logs, metrics, and monitoring dashboards support operations.

Built on open-source CDAP, Data Fusion supports pipeline portability across hybrid and multi-cloud environments. Basic provides the first 120 pipeline-development hours per month per account at no charge, and new customers receive $300 in credits. Pipeline execution also incurs separate charges for Managed Service for Apache Spark clusters and other Google Cloud resources. Developer is $0.35 per instance hour; Basic is $1.80 per instance hour after the free hours; Enterprise is $4.20 per instance hour. All are billed by the minute.

Who it suits

Data teams that want to design ETL or ELT pipelines visually, connect cloud and database sources, and track lineage may find Data Fusion suitable. Its Developer, Basic, and Enterprise editions address development, testing, and production workloads, respectively.

Limits

Basic includes the first 120 pipeline-development hours per month per account; additional instance time is $1.80 per hour, billed by the minute. Pipeline execution also incurs separate charges for Spark clusters and other Google Cloud resources. Each environment version is supported for 18 months after release; unsupported versions receive no further fixes, including security fixes.

Getting started

1. Visit https://cloud.google.com/data-fusion.
2. Choose a Developer, Basic, or Enterprise edition for the intended workload.
3. Create an instance in the cloud; its edition cannot be changed later.
4. Use the visual interface to build and deploy a pipeline.
5. Connect data sources and configure schedules, triggers, or monitoring as needed.

Verdict

Choose Google Cloud Data Fusion if visual pipeline design, lineage tracking, and Google Cloud integrations fit your data workflows. The free Basic development hours and new-customer credits provide a starting point. Account for separate execution-resource charges, and select an edition carefully because an instance's edition cannot later be changed.