# Load Greenpixie data into CloudHealth (AWS) Source: https://greenpixie.com/docs/guides/cloudhealth-integration-aws How to integrate your enriched AWS cloud usage report data into CloudHealth. This guide takes you from an enriched Cost and Usage Report (CUR) file in your Amazon S3 bucket through to a fused cost and carbon dataset inside CloudHealth. It is written for a platform, FinOps or data engineer who already has the enriched file. That file carries the standard AWS CUR columns plus Greenpixie's numeric sustainability columns for energy in kWh, carbon in tonnes CO₂e and water in litres. Where the procedure is Broadcom's, this guide links to the CloudHealth documentation and covers what sustainability data changes. The integration uses CloudHealth **Data Connect**, the Bring Your Own Data feature, to load the file as a custom dataset. **Custom Datasets** then joins it onto the native cost model on shared keys such as resource ID and SKU. For an Azure estate, follow [the Azure version of this guide](/guides/cloudhealth-integration-azure). ## Before you start [#before-you-start] You need an enriched CUR file landing in an S3 bucket you control, carrying both the standard AWS CUR columns and Greenpixie's numeric sustainability columns. Data Connect's own limits shape that file before you build it, and [Bring Your Own Data with Data Connect](https://techdocs.broadcom.com/us/en/ca-enterprise-software/it-operations-management/cloudhealth/saas/index/exploring-cloudhealth-new-experience-exploring-cloudhealth-index/exploring-cloudhealth-new-experience-getting-started/bring-your-own-data--byod--with-data-connect.html) is the authority on the current numbers. The limits that change what you produce: * Data Connect is available only at the **top-level organisational unit (TLOU)** of the CloudHealth tenant. Create the connection there. A FlexOrg administrator without TLOU access will not find the feature. * **The schema cannot be edited after creation.** The sample file you supply defines it, so get that file right the first time. * **Maximum 100 MB per file**, as CSV or Parquet, and **only one file per account per billing month**. Together these mean the CUR has to be pre-aggregated down to one file per account-month. * **Column names must match `^[a-zA-Z][a-zA-Z0-9_-]*$`**, so letters, numbers, underscores and hyphens only, starting with a letter. Legacy CUR column names contain a slash and fail this rule, so they have to be renamed. * Native CUR loading requires **Legacy CUR, not CUR 2.0**, so the join keys line up with CloudHealth's own CUR dataset. ## Prepare the enriched file [#prepare-the-enriched-file] 1. **Confirm the identifier columns are present and clean.** The join back to the cost model depends on columns that also exist in CloudHealth's native CUR dataset. Include at minimum: * `lineItem/ResourceId`, the strongest join key. * `lineItem/UsageType` and/or `product/sku`, for SKU-level joins and where resource IDs are blank, such as some data-transfer or tax lines. * `lineItem/UsageAccountId`, the linked account. * A period key, `bill/BillingPeriodStartDate` or `lineItem/UsageStartDate` truncated to the day or month. 2. **Rename the CUR columns to pass the naming rule.** Legacy CUR uses a slash in every prefixed column name, which CloudHealth rejects at schema inference. Replace the slash with an underscore across the whole file. | CUR column | Rename to | | ----------------------------- | ----------------------------- | | `lineItem/ResourceId` | `lineItem_ResourceId` | | `lineItem/UsageType` | `lineItem_UsageType` | | `lineItem/UsageAccountId` | `lineItem_UsageAccountId` | | `product/sku` | `product_sku` | | `bill/BillingPeriodStartDate` | `bill_BillingPeriodStartDate` | Greenpixie's own enrichment columns contain only letters and underscores, so they need no change. 3. **Convert the period columns to `YYYY-MM-DD`.** CloudHealth infers a column as a Date only in `YYYY-MM-DD` form, and as a Timestamp only in `YYYY-MM-DDTHH:MM:SS` form. Anything else is stored as a string, and a string period key cannot be used as a date in reports or matched cleanly against CloudHealth's own period fields. 4. **Fix the grain.** CUR is granular to line item, resource and hour, while Greenpixie metrics are usually per-resource and often daily. Decide the join grain now, where resource ID, SKU and day or month is the natural meeting point, and pre-aggregate the coarser side so you have exactly one row per key. A one-to-many match will multiply rows and inflate both cost and emissions. 5. **Keep sustainability columns strictly numeric** so CloudHealth classifies them as measures, not dimensions, for example `usage_electricity_consumption_kwh`, `total_tonnes_co2e`, `total_water_litres`. 6. **Get each account-month under 100 MB.** Because the folder structure allows one file per account per month and nothing finer, a single account-month that exceeds 100 MB in Parquet cannot be split. Reduce it instead, either by pre-aggregating to a coarser grain, so resource ID and month where you currently have resource ID and day, or by dropping columns that serve neither the join nor the report. Parquet is preferred for size and typed columns. 7. **Produce a representative sample file** containing every column with realistic values, which is what defines the immutable schema. ## Lay out the files in S3 [#lay-out-the-files-in-s3] Data Connect reads one fixed folder structure, set out on Broadcom's [Bring Your Own Data](https://techdocs.broadcom.com/us/en/ca-enterprise-software/it-operations-management/cloudhealth/saas/index/exploring-cloudhealth-new-experience-exploring-cloudhealth-index/exploring-cloudhealth-new-experience-getting-started/bring-your-own-data--byod--with-data-connect.html) page, and it does not collect a file written to a free-form prefix. Accepted extensions are `.csv`, `.csv.gz` and `.parquet`. A worked Greenpixie path is below. ``` BYOD/greenpixie_sustainability/123456789012/2026-07/1/gpx-enriched.parquet ``` Read against that structure, `BYOD` is a literal prefix, `greenpixie_sustainability` the Data Connect dataset name you enter under Metadata, `123456789012` the AWS account ID the data belongs to, `2026-07` the billing month, and `1` an incrementing version folder used when you re-upload the same month. ## Grant CloudHealth access to your S3 bucket [#grant-cloudhealth-access-to-your-s3-bucket] Data Connect authenticates to S3 through an IAM assume-role, using an External ID that CloudHealth generates. CloudHealth's own account setup documentation carries the trust policy and the CloudHealth principal to trust, both of which change, so take them from Broadcom at the time you build the role. 1. In CloudHealth, go to **Setup > Accounts > AWS**, open the target account, and copy the **External ID**, a roughly 30-character hex string, and the exact **Account Name**. 2. Create the IAM role with the trust policy from Broadcom's documentation, conditioned on that External ID. 3. Attach a least-privilege read policy scoped to the `BYOD` prefix in the bucket. That scoping is the part specific to this integration. ```json { "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Action": ["s3:GetObject"], "Resource": "arn:aws:s3:::/BYOD/*" }, { "Effect": "Allow", "Action": ["s3:ListBucket"], "Resource": "arn:aws:s3:::", "Condition": { "StringLike": { "s3:prefix": ["BYOD/*"] } } } ] } ``` 4. If the bucket uses SSE-KMS, grant the role `kms:Decrypt` on the relevant key. 5. Note the new role's **ARN**, the bucket's name, the **region** it is in, and the **path** to the `BYOD` prefix. Data Connect asks for all four. ## Set up Data Connect [#set-up-data-connect] Broadcom's Data Connect documentation covers the wizard under **Setup & Configuration > Data Connect**. Choose **Custom File Upload** as the connection type, since the automated connector option currently covers Databricks alone. The values it asks for that have to match what you wrote to S3: * The dataset name under **Metadata** is the `` folder in the path, so `greenpixie_sustainability` in the example above. * The **Account Identifier** can be any unique identifier, and the AWS account ID is the natural choice, matching the `` folder. * The **Usage Month** is given in `YYYY-MM` format and matches the month folder you wrote. CloudHealth verifies it can reach the storage location before it opens the Schema section, so a bucket policy, KMS or path error shows up at that point. In the **Schema** section, enter the data location file path to your sample file and choose **Get Schema**. ## Define the schema [#define-the-schema] CloudHealth reads the sample file, auto-classifies each column as a measure or a dimension, assigns a data type, and locks the result once you save. To change a saved schema, create a new connection with a corrected sample file. What needs correcting before you save: * `usage_electricity_consumption_kwh`, `total_tonnes_co2e` and `total_water_litres` have to be numeric **measures**, and `lineItem_ResourceId`, `product_sku`, `lineItem_UsageType`, account and period have to be **dimensions**. * The period column's inferred type has to be **Date**. Where it shows as a string, the file's date format is wrong and the sample needs regenerating before you save. * **Mapping Identifiers** have to be set. These are CloudHealth's cloud-agnostic FOCUS fields, from the FinOps Open Cost and Usage Specification, being ResourceId, ServiceName, Billingaccount and ChargePeriod. Map your identifier columns onto them, since that mapping is what lets the dataset line up with other CloudHealth data, and what the join depends on. The dataset imports and, within about 24 hours, appears as a Data Connect dataset in the **Reports** section and in classic **FlexReports**. ## Load each new month [#load-each-new-month] Write each new month's file into a new `` folder under the same dataset and account path, using the same schema and column names. Data Connect collects on a 12-hour schedule, and **Trigger Collection** on the connection pulls a new file without waiting for the next scheduled run. To correct a month already uploaded, write a complete replacement file containing every row for that month into a new `` folder. CloudHealth does not merge partial updates, so the latest file must include all previously uploaded data for that month. Monitor row counts per period as a smoke test confirming the load is working. ## Join onto the cost model [#join-onto-the-cost-model] In **Custom Datasets**, create a new dataset over two sources, the native CloudHealth CUR / cost dataset and your `greenpixie_sustainability` Data Connect dataset, then add a **JOIN** operation. Broadcom's [Custom Datasets](https://techdocs.broadcom.com/us/en/ca-enterprise-software/it-operations-management/cloudhealth/saas/index/exploring-cloudhealth-new-experience-exploring-cloudhealth-index/exploring-cloudhealth-new-experience-nx-analyzing-costs-and-trends/custom-datasets-.html) page covers the operations and the calculated columns. Use a **LEFT** join with the CUR as the left dataset, which keeps every cost line and attaches Greenpixie measures wherever the keys match. INNER restricts the result to resources Greenpixie has measured, which is rarely what you want. Select join columns that match on both sides at the same grain. * The primary key is resource ID. Your renamed `lineItem_ResourceId` does not share a name with CloudHealth's native `lineItem/ResourceId`, so match the two through the **ResourceId** mapping identifier you set on the schema. Confirm the join behaves as expected on your tenant before building reports on top of it, since the join across renamed columns is not covered by Broadcom's documentation. * Add SKU / usage type, `product_sku` or `lineItem_UsageType`, for SKU-level precision and to help on lines without a resource ID. * Add account and period to keep the match in scope and avoid cross-period fan-out. Compare the joined dataset's total cost against the unjoined CUR total as soon as the join runs; for a LEFT join, they should be identical. If cost has inflated, a one-to-many match is fanning rows out, so revisit the grain or add join keys. Then add calculated columns for the metrics that matter, for example `carbon_intensity_per_dollar = total_tonnes_co2e / cost`, `emissions_by_service`, or `kwh_per_service`. ## Report and dashboard [#report-and-dashboard] Build a FlexReport or Report on the joined custom dataset, grouping cost and emissions by service, account, team tag or SKU on the same rows, and embed those reports in a CloudHealth dashboard for a unified cost-and-carbon view. For programmatic extraction, the GraphQL reporting API pulls the joined dataset into a BI tool. ## References [#references] * [Bring Your Own Data (BYOD) with Data Connect](https://techdocs.broadcom.com/us/en/ca-enterprise-software/it-operations-management/cloudhealth/saas/index/exploring-cloudhealth-new-experience-exploring-cloudhealth-index/exploring-cloudhealth-new-experience-getting-started/bring-your-own-data--byod--with-data-connect.html) * [BYOD Setup Quick Start](https://docs.vmware.com/en/VMware-Tanzu-CloudHealth/SaaS/getting-started-with-vmware-tanzu-cloudhealth/GUID-bring-your-own-data-quick-start.html) * [Custom Datasets](https://techdocs.broadcom.com/us/en/ca-enterprise-software/it-operations-management/cloudhealth/saas/index/exploring-cloudhealth-new-experience-exploring-cloudhealth-index/exploring-cloudhealth-new-experience-nx-analyzing-costs-and-trends/custom-datasets-.html), covering JOIN, MERGE and calculated columns * [FlexReports](https://docs.vmware.com/en/VMware-Tanzu-CloudHealth/SaaS/working-with-reports-recommendations-vmware-tanzu-cloudhealth/GUID-flexreports.html) * [GraphQL API guides](https://help.cloudhealthtech.com/graphql-api/) * [CloudHealth API docs](https://apidocs.cloudhealthtech.com/) ## Next steps [#next-steps] - [Enrich an AWS cloud usage report using the API](https://greenpixie.com/docs/guides/enrich-cur-api): Produce the enriched file this guide starts from. - [Load Greenpixie data into CloudHealth (Azure)](https://greenpixie.com/docs/guides/cloudhealth-integration-azure): The same integration for an Azure estate. - [Read the API documentation](https://api.greenpixie.com/v1/docs): Full endpoint reference for the Greenpixie rate card.