Ingest Greenpixie data into CloudHealth

This guide takes you from an enriched CUR file in your Amazon S3 bucket through to a fused cost and carbon dataset inside CloudHealth. It is written for a platform, FinOps or data engineer who already has the enriched file, with standard AWS CUR columns plus Greenpixie's numeric sustainability columns (kWh, kgCO₂e, litres).

The integration uses CloudHealth Data Connect (Bring Your Own Data) to ingest the file as a custom dataset, then Custom Datasets to join it onto the native cost model on shared keys such as resource ID and SKU.

Before you start

You need an enriched CUR file landing in an S3 bucket you control, carrying both the standard AWS CUR columns and Greenpixie's numeric sustainability columns.

A few constraints shape the whole design, so read them first:

• Data Connect, part of the newer CloudHealth experience, is relatively new. Confirm the tenant is entitled to it.
• The schema cannot be edited after creation. Getting the sample file right the first time is critical.
• Maximum 100 MB per file (CSV or Parquet). Partition large CURs.
• Ingested data is queryable within about 24 hours. Treat it as daily, not real-time.
• Native CUR ingestion requires Legacy CUR, not CUR 2.0. Keep the enriched file in legacy CUR column naming so the join keys line up with CloudHealth's own CUR dataset.

Prepare the enriched file

Before touching CloudHealth, make the file join-ready.

1. Confirm the identifier columns are present and clean. The join back to the cost model depends on columns that also exist in CloudHealth's native CUR dataset. Include at minimum:
   • lineItem/ResourceId, the strongest join key.
   • lineItem/UsageType and/or product/sku, for SKU-level joins and where resource IDs are blank, such as some data-transfer or tax lines.
   • lineItem/UsageAccountId, the linked account.
   • A period key, bill/BillingPeriodStartDate or lineItem/UsageStartDate truncated to the day or month.
2. Fix the grain. CUR is granular to line item, resource and hour, while Greenpixie metrics are usually per-resource and often daily. Decide the join grain now (resource ID + SKU + day or month is the natural meeting point) and pre-aggregate the coarser side so you have exactly one row per key. A one-to-many match will multiply rows and inflate both cost and emissions.
3. Keep sustainability columns strictly numeric so CloudHealth classifies them as measures, not dimensions, for example gp_energy_kwh, gp_emissions_kgco2e, gp_water_litres.
4. Partition to stay under 100 MB per file. Split by billing period and account, for example s3://<bucket>/greenpixie/2026/07/account=1234/part-000.parquet. Parquet is preferred for size and typed columns.
5. Produce a representative sample file containing every column with realistic values. This is what defines the immutable schema.

Grant CloudHealth access to your S3 bucket

Data Connect authenticates to S3 through an IAM assume-role, using an External ID that CloudHealth generates. Collect three values from CloudHealth first.

1. In CloudHealth, go to Setup > Accounts > AWS, open the target account, and copy the External ID (a roughly 30-character hex string) and the exact Account Name. These are two of the four fields Data Connect will ask for.

2. In the AWS IAM console, create an IAM role for this connection with a trust policy that lets CloudHealth's account assume it, conditioned on that External ID. Verify the CloudHealth principal against current CloudHealth docs before applying.

   {
     "Version": "2012-10-17",
     "Statement": [{
       "Effect": "Allow",
       "Principal": { "AWS": "arn:aws:iam::<CLOUDHEALTH_AWS_ACCOUNT_ID>:root" },
       "Action": "sts:AssumeRole",
       "Condition": { "StringEquals": { "sts:ExternalId": "<EXTERNAL_ID_FROM_CLOUDHEALTH>" } }
     }]
   }

3. Attach a least-privilege read policy scoped to the Greenpixie prefix in the bucket.

   {
     "Version": "2012-10-17",
     "Statement": [
       {
         "Effect": "Allow",
         "Action": ["s3:GetObject"],
         "Resource": "arn:aws:s3:::<BUCKET_NAME>/greenpixie/*"
       },
       {
         "Effect": "Allow",
         "Action": ["s3:ListBucket"],
         "Resource": "arn:aws:s3:::<BUCKET_NAME>",
         "Condition": { "StringLike": { "s3:prefix": ["greenpixie/*"] } }
       }
     ]
   }

4. Copy the new role's ARN. This is the Assume Role ARN that Data Connect needs.

5. If the bucket uses SSE-KMS, grant the role kms:Decrypt on the relevant key.

You now have all four connection values: Assume Role ARN, Assume Role External ID, Assume Role Account Name, and S3 Bucket Name.

Create the Data Connect integration

1. In CloudHealth, open Data Connect and choose Add New Connection.
2. In Select Connection Type, choose Custom File Upload. The automated connector option currently only covers Databricks, so custom files from S3 go through this path.
3. Under Metadata, enter a dataset name and description, for example greenpixie_sustainability, then choose Next.
4. Under Authentication, set Data Location to AWS S3 and enter the four values from the previous step.
5. Under File path, point to the object or prefix of your sample file, then choose Get Schema.

Define the schema

The schema is set once and locked, so confirm it carefully.

1. CloudHealth reads the sample file and auto-classifies each column as a measure or a dimension, and assigns a data type.
2. Correct the classification before saving:
   • gp_energy_kwh, gp_emissions_kgco2e, gp_water_litres become measures (numeric).
   • lineItem/ResourceId, product/sku, lineItem/UsageType, account and period become dimensions.
3. Set Mapping Identifiers. These are CloudHealth's cloud-agnostic FOCUS fields (ResourceId, ServiceName, Billingaccount, ChargePeriod). Map your identifier columns onto them. This is what lets the dataset line up with other CloudHealth data, and is the foundation for the join.
4. Review and finish. The schema is locked after creation. To change it, you create a new connection with a corrected sample file.
5. The dataset imports and, within about 24 hours, appears as a Data Connect dataset in the Reports section and in classic FlexReports.

Keep it fed

1. Write each new period's enriched file to the same S3 prefix, using the same schema and column names.
2. Data Connect reads from the configured S3 location, so new files under the path are picked up on the ingestion cycle (roughly daily). Keep every file schema-identical and under 100 MB.
3. Monitor row counts per period as a smoke test that ingestion is landing.

Join onto the cost model

This is where the sustainability data fuses with cost.

1. Go to Custom Datasets and create a new dataset.
2. Add two source datasets, the native CloudHealth CUR / cost dataset and your greenpixie_sustainability Data Connect dataset.
3. Add a JOIN operation, and choose the type deliberately:
   • A LEFT join with the CUR as the left (first) dataset keeps every cost line and attaches Greenpixie measures where a match exists. This is the recommended default, since you never want to drop cost rows.
   • Use INNER only if you want to restrict to resources Greenpixie has measured.
4. Select the join columns, which must match on both sides at the same grain:
   • The primary key is resource ID, lineItem/ResourceId.
   • Add SKU / usage type, product/sku or lineItem/UsageType, for SKU-level precision and to help on lines without a resource ID.
   • Add account and period to keep the match in scope and avoid cross-period fan-out. More keys make for a tighter, safer match.
5. Validate immediately. Compare the joined dataset's total cost against the unjoined CUR total; for a LEFT join they should be identical. If cost has inflated, a one-to-many match is fanning rows out, so revisit the grain or add join keys.
6. Add calculated columns for the metrics that matter, for example carbon_intensity_per_dollar = gp_emissions_kgco2e / cost, emissions_by_service, or kwh_per_service.

Report and dashboard

1. Build a FlexReport or Report on the joined custom dataset, grouping cost and emissions by service, account, team tag or SKU on the same rows.
2. Embed one or more of these reports into a CloudHealth dashboard for a unified cost-and-carbon view.
3. For programmatic extraction, the GraphQL reporting API can pull the joined dataset into a BI tool.

What this does and does not cover

The fusion lives at the reporting and dashboard layer. The joined dataset powers Reports, FlexReports and dashboards, but the metrics are not pushed into CloudHealth's deeper native machinery. Perspectives, budgets and anomaly detection still operate on cost alone. For showback, chargeback and sustainability reporting this is more than enough, but "carbon per Perspective" is not available out of the box.

A note on accuracy

The exact AWS trust-policy principal and account ID, and any file-path conventions, are tenant-specific and change over time. Confirm them against the live Data Connect setup screen and current CloudHealth docs before applying.

References

• Bring Your Own Data (BYOD) with Data Connect
• BYOD Setup Quick Start
• Custom Datasets (JOIN / MERGE, calculated columns)
• FlexReports
• GraphQL API guides
• CloudHealth API docs

Next steps