Load Greenpixie data into CloudHealth (AWS)

This guide takes you from an enriched Cost and Usage Report (CUR) file in your Amazon S3 bucket through to a fused cost and carbon dataset inside CloudHealth. It is written for a platform, FinOps or data engineer who already has the enriched file. That file carries the standard AWS CUR columns plus Greenpixie's numeric sustainability columns for energy in kWh, carbon in tonnes CO₂e and water in litres. Where the procedure is Broadcom's, this guide links to the CloudHealth documentation and covers what sustainability data changes.

The integration uses CloudHealth Data Connect, the Bring Your Own Data feature, to load the file as a custom dataset. Custom Datasets then joins it onto the native cost model on shared keys such as resource ID and SKU.

For an Azure estate, follow the Azure version of this guide.

Before you start

You need an enriched CUR file landing in an S3 bucket you control, carrying both the standard AWS CUR columns and Greenpixie's numeric sustainability columns.

Data Connect's own limits shape that file before you build it, and Bring Your Own Data with Data Connect is the authority on the current numbers. The limits that change what you produce:

• Data Connect is available only at the top-level organisational unit (TLOU) of the CloudHealth tenant. Create the connection there. A FlexOrg administrator without TLOU access will not find the feature.
• The schema cannot be edited after creation. The sample file you supply defines it, so get that file right the first time.
• Maximum 100 MB per file, as CSV or Parquet, and only one file per account per billing month. Together these mean the CUR has to be pre-aggregated down to one file per account-month.
• Column names must match ^[a-zA-Z][a-zA-Z0-9_-]*$, so letters, numbers, underscores and hyphens only, starting with a letter. Legacy CUR column names contain a slash and fail this rule, so they have to be renamed.
• Native CUR loading requires Legacy CUR, not CUR 2.0, so the join keys line up with CloudHealth's own CUR dataset.

Prepare the enriched file

1. Confirm the identifier columns are present and clean. The join back to the cost model depends on columns that also exist in CloudHealth's native CUR dataset. Include at minimum:

   • lineItem/ResourceId, the strongest join key.
   • lineItem/UsageType and/or product/sku, for SKU-level joins and where resource IDs are blank, such as some data-transfer or tax lines.
   • lineItem/UsageAccountId, the linked account.
   • A period key, bill/BillingPeriodStartDate or lineItem/UsageStartDate truncated to the day or month.

2. Rename the CUR columns to pass the naming rule. Legacy CUR uses a slash in every prefixed column name, which CloudHealth rejects at schema inference. Replace the slash with an underscore across the whole file.

   CUR column	Rename to
   lineItem/ResourceId	lineItem_ResourceId
   lineItem/UsageType	lineItem_UsageType
   lineItem/UsageAccountId	lineItem_UsageAccountId
   product/sku	product_sku
   bill/BillingPeriodStartDate	bill_BillingPeriodStartDate

   Greenpixie's own enrichment columns contain only letters and underscores, so they need no change.

3. Convert the period columns to YYYY-MM-DD. CloudHealth infers a column as a Date only in YYYY-MM-DD form, and as a Timestamp only in YYYY-MM-DDTHH:MM:SS form. Anything else is stored as a string, and a string period key cannot be used as a date in reports or matched cleanly against CloudHealth's own period fields.

4. Fix the grain. CUR is granular to line item, resource and hour, while Greenpixie metrics are usually per-resource and often daily. Decide the join grain now, where resource ID, SKU and day or month is the natural meeting point, and pre-aggregate the coarser side so you have exactly one row per key. A one-to-many match will multiply rows and inflate both cost and emissions.

5. Keep sustainability columns strictly numeric so CloudHealth classifies them as measures, not dimensions, for example usage_electricity_consumption_kwh, total_tonnes_co2e, total_water_litres.

6. Get each account-month under 100 MB. Because the folder structure allows one file per account per month and nothing finer, a single account-month that exceeds 100 MB in Parquet cannot be split. Reduce it instead, either by pre-aggregating to a coarser grain, so resource ID and month where you currently have resource ID and day, or by dropping columns that serve neither the join nor the report. Parquet is preferred for size and typed columns.

7. Produce a representative sample file containing every column with realistic values, which is what defines the immutable schema.

Lay out the files in S3

Data Connect reads one fixed folder structure, set out on Broadcom's Bring Your Own Data page, and it does not collect a file written to a free-form prefix. Accepted extensions are .csv, .csv.gz and .parquet. A worked Greenpixie path is below.

BYOD/greenpixie_sustainability/123456789012/2026-07/1/gpx-enriched.parquet

Read against that structure, BYOD is a literal prefix, greenpixie_sustainability the Data Connect dataset name you enter under Metadata, 123456789012 the AWS account ID the data belongs to, 2026-07 the billing month, and 1 an incrementing version folder used when you re-upload the same month.

Grant CloudHealth access to your S3 bucket

Data Connect authenticates to S3 through an IAM assume-role, using an External ID that CloudHealth generates. CloudHealth's own account setup documentation carries the trust policy and the CloudHealth principal to trust, both of which change, so take them from Broadcom at the time you build the role.

1. In CloudHealth, go to Setup > Accounts > AWS, open the target account, and copy the External ID, a roughly 30-character hex string, and the exact Account Name.

2. Create the IAM role with the trust policy from Broadcom's documentation, conditioned on that External ID.

3. Attach a least-privilege read policy scoped to the BYOD prefix in the bucket. That scoping is the part specific to this integration.

   {
     "Version": "2012-10-17",
     "Statement": [
       {
         "Effect": "Allow",
         "Action": ["s3:GetObject"],
         "Resource": "arn:aws:s3:::<BUCKET_NAME>/BYOD/*"
       },
       {
         "Effect": "Allow",
         "Action": ["s3:ListBucket"],
         "Resource": "arn:aws:s3:::<BUCKET_NAME>",
         "Condition": { "StringLike": { "s3:prefix": ["BYOD/*"] } }
       }
     ]
   }

4. If the bucket uses SSE-KMS, grant the role kms:Decrypt on the relevant key.

5. Note the new role's ARN, the bucket's name, the region it is in, and the path to the BYOD prefix. Data Connect asks for all four.

Set up Data Connect

Broadcom's Data Connect documentation covers the wizard under Setup & Configuration > Data Connect. Choose Custom File Upload as the connection type, since the automated connector option currently covers Databricks alone. The values it asks for that have to match what you wrote to S3:

• The dataset name under Metadata is the <datasetName> folder in the path, so greenpixie_sustainability in the example above.
• The Account Identifier can be any unique identifier, and the AWS account ID is the natural choice, matching the <accountId> folder.
• The Usage Month is given in YYYY-MM format and matches the month folder you wrote.

CloudHealth verifies it can reach the storage location before it opens the Schema section, so a bucket policy, KMS or path error shows up at that point. In the Schema section, enter the data location file path to your sample file and choose Get Schema.

Define the schema

CloudHealth reads the sample file, auto-classifies each column as a measure or a dimension, assigns a data type, and locks the result once you save. To change a saved schema, create a new connection with a corrected sample file. What needs correcting before you save:

• usage_electricity_consumption_kwh, total_tonnes_co2e and total_water_litres have to be numeric measures, and lineItem_ResourceId, product_sku, lineItem_UsageType, account and period have to be dimensions.
• The period column's inferred type has to be Date. Where it shows as a string, the file's date format is wrong and the sample needs regenerating before you save.
• Mapping Identifiers have to be set. These are CloudHealth's cloud-agnostic FOCUS fields, from the FinOps Open Cost and Usage Specification, being ResourceId, ServiceName, Billingaccount and ChargePeriod. Map your identifier columns onto them, since that mapping is what lets the dataset line up with other CloudHealth data, and what the join depends on.

The dataset imports and, within about 24 hours, appears as a Data Connect dataset in the Reports section and in classic FlexReports.

Load each new month

Write each new month's file into a new <YYYY-MM> folder under the same dataset and account path, using the same schema and column names. Data Connect collects on a 12-hour schedule, and Trigger Collection on the connection pulls a new file without waiting for the next scheduled run.

To correct a month already uploaded, write a complete replacement file containing every row for that month into a new <version> folder. CloudHealth does not merge partial updates, so the latest file must include all previously uploaded data for that month. Monitor row counts per period as a smoke test confirming the load is working.

Join onto the cost model

In Custom Datasets, create a new dataset over two sources, the native CloudHealth CUR / cost dataset and your greenpixie_sustainability Data Connect dataset, then add a JOIN operation. Broadcom's Custom Datasets page covers the operations and the calculated columns.

Use a LEFT join with the CUR as the left dataset, which keeps every cost line and attaches Greenpixie measures wherever the keys match. INNER restricts the result to resources Greenpixie has measured, which is rarely what you want.

Select join columns that match on both sides at the same grain.

• The primary key is resource ID. Your renamed lineItem_ResourceId does not share a name with CloudHealth's native lineItem/ResourceId, so match the two through the ResourceId mapping identifier you set on the schema. Confirm the join behaves as expected on your tenant before building reports on top of it, since the join across renamed columns is not covered by Broadcom's documentation.
• Add SKU / usage type, product_sku or lineItem_UsageType, for SKU-level precision and to help on lines without a resource ID.
• Add account and period to keep the match in scope and avoid cross-period fan-out.

Compare the joined dataset's total cost against the unjoined CUR total as soon as the join runs; for a LEFT join, they should be identical. If cost has inflated, a one-to-many match is fanning rows out, so revisit the grain or add join keys. Then add calculated columns for the metrics that matter, for example carbon_intensity_per_dollar = total_tonnes_co2e / cost, emissions_by_service, or kwh_per_service.

Report and dashboard

Build a FlexReport or Report on the joined custom dataset, grouping cost and emissions by service, account, team tag or SKU on the same rows, and embed those reports in a CloudHealth dashboard for a unified cost-and-carbon view. For programmatic extraction, the GraphQL reporting API pulls the joined dataset into a BI tool.

References

• Bring Your Own Data (BYOD) with Data Connect
• BYOD Setup Quick Start
• Custom Datasets, covering JOIN, MERGE and calculated columns
• FlexReports
• GraphQL API guides
• CloudHealth API docs

Next steps