You can bring health data exports into one local analytics workflow without replacing the systems that created them: keep original files unchanged, use DuckDB to query and store analytical tables, and use dbt to document and test repeatable transformations. This is for personal exploration—not a clinical record, medical advice, or proof that your data is secure.
What this setup does—and what it does not
DuckDB is an embedded database designed for analytical workloads, and the dbt-duckdb adapter lets dbt run transformations against it. The adapter can persist relations in a local DuckDB database file and work with CSV, JSON, and Parquet inputs. DuckDB also supports querying Parquet files directly. See the dbt-duckdb project documentation and DuckDB’s guides.
As an Amazon Associate I earn from qualifying purchases.
The result is a manageable workspace for questions such as how a measurement changes over time or how often a source has gaps. It is not a replacement for your provider’s chart, a validated clinical system, or a tool for deciding whether to start, stop, or change treatment.
Support status differs by environment. dbt’s supported-platform page lists DuckDB as generally available for “DuckDB (CLI only)” and says the listing applies to dbt v2.0 and later; it also cautions that the local adapter lifecycle may differ from the dbt platform lifecycle. The adapter README retrieved for this workflow describes support targets of dbt-core 1.8.x or later and DuckDB 1.0.0 or later. Check the current dbt supported-platform listing and adapter README before installing because supported versions can change.
#1 Best Overall
Choose a local workflow or a managed FHIR service
| Consideration | DuckDB and dbt CLI on your computer | Managed FHIR service, such as AWS HealthLake |
|---|---|---|
| Where data lives | Files and a DuckDB database you manage locally; direct file querying is supported by the tools. | In a cloud-managed FHIR datastore and related cloud storage configured for the service. |
| Setup and upkeep | Requires comfort with files, a command-line workflow, dbt project configuration, and your own backups. | Requires a cloud account, permissions, service and storage configuration, and ongoing cloud administration. |
| Access control | You manage computer access, database copies, backups, credentials, and any sync behavior. | Access depends on cloud identity and service configuration; AWS HealthLake documentation describes specific IAM and SMART-on-FHIR requirements. |
| Interoperability | Works with exported files such as CSV, JSON, and Parquet; FHIR data may need to be obtained and prepared through the source’s supported export route. | Designed around FHIR data and provides service-specific export capabilities, including NDJSON output to S3. |
| Cost and scale | No cloud service is required for the basic local workflow; hardware, backups, and any other services remain your responsibility. No price is established here. | Cloud account and service usage may incur ongoing costs; no price is established here. It is an adjacent managed-service option, not a necessary choice for an individual. |
For one person doing exploratory analysis, local DuckDB plus dbt generally means less infrastructure and more responsibility for protecting and maintaining copies. A managed FHIR service is a different operating model and should not be adopted just because a provider’s export is inconvenient.
Prepare exports without losing the originals
Inventory what you can actually export
Start by listing each source, the export method it offers, its file format, and what the export contains. Record whether dates include a timezone, what units measurements use, which identifiers are stable, and whether the export contains the full history or only a selected period. Keep unknowns explicit rather than assuming two similarly named fields mean the same thing.
Rank #2
Separate raw files from transformed data
Save untouched downloads in a raw area, for example data/raw/<source>/, and place your dbt project and DuckDB database in a separate working area. This layout is a practical design recommendation, not a requirement imposed by DuckDB or dbt. Add source and export date to folder or filenames where possible, and do not overwrite an older export when refreshing.
For each export, keep a small record of its source, export date, date range, file format, timezone and units when known, and any known missingness. This makes it easier to distinguish a genuine change from a changed export window, a unit difference, or an incomplete file.
Build a small, reviewable dbt project
Connect dbt to a persistent DuckDB file
Configure the dbt-duckdb adapter with a profile path that points to the DuckDB file where you want relations stored. Use the adapter’s current installation and profile instructions rather than copying an old version-specific configuration: the adapter’s supported dbt and DuckDB targets can change. The adapter README documents persistence through the profile’s path setting as well as access to external CSV, JSON, and Parquet inputs.
Move from source-shaped data to analysis models
- Source and staging models: expose the source data with its original field meanings and retain source-specific columns. Avoid renaming away distinctions you do not understand.
- Intermediate models: normalize timestamps, units, and identifiers only where the conversion rule is known. Preserve the original value or field when interpretation is uncertain.
- Analysis models: create focused tables for a clearly stated personal question, such as plotting a measurement by date. Avoid combining records merely because they appear related.
Keep transformations small enough to inspect. Record assumptions beside the models: for example, how a timestamp without an explicit timezone is treated, whether a conversion applies to every record, or which source fields were excluded.
Test structure, not medical meaning
Use dbt tests for assumptions that can be stated precisely, such as a key expected to be unique, a timestamp expected to be present, an allowed set of units, or a reasonable row-count range for a known export. A passing test means the tested data-shape condition held; it does not establish that a measurement is accurate, that units were interpreted clinically correctly, or that a transformation is medically valid.
The Tool Desk
Outbyte PC Repair FREEClear out junk files and repair common Windows errorsFree Scan →Outbyte Driver Updater FREEScan for outdated or missing drivers - takes under a minuteDriver Scan →Do not silently fill missing values or smooth gaps just to make a chart look complete. Make nulls and missing periods visible, and document when a source did not provide a value. dbt produces machine-readable artifacts from invocations that can support project documentation, test coverage, timing analysis, and other run-history uses; exact artifact names and behavior depend on the dbt version. See dbt’s artifact documentation.
Best Value
Use FHIR bulk export only when your source supports it
FHIR bulk export is an optional route, not a universal personal-data export feature. The server must support the capability, and your authorization determines what you can retrieve. HL7’s current Bulk Data Access Implementation Guide describes authenticated requests and asynchronous retrieval; it says servers should return only resources the client is authorized to access and recommends respecting individual sharing preferences. The export scope therefore depends on both the server’s implementation and your permissions.
A provider’s supported patient export is usually the more direct starting point when it meets your needs. AWS HealthLake is a specific cloud-service example: its documentation describes FHIR bulk export to S3 as NDJSON objects organized by resource type, with IAM and SMART-on-FHIR requirements. Those details apply to HealthLake, not to every FHIR server. See AWS HealthLake’s export documentation.
Protect the copies you create
A local database is not inherently secure. Anyone who can access the computer, an unprotected backup, or a synced copy may be able to access its contents. Minimize the number of copies and consider who can reach the device and its backups. Use the operating system’s available access controls and encryption options, protect credentials, understand whether a sync folder uploads files and how long it retains them, and keep health data out of public repositories and screenshots. Decide how you will delete old exports, database files, and backups when they are no longer needed.
Do these 3 things before closing this tab:
1Clear out junk files and repair common Windows errors2Fix the driver behind crashes, sound loss and screen glitches3Repair Windows errors before they cause bigger problemsIn the United States, HIPAA generally does not protect health information stored on a personal phone or tablet, and for most apps it does not protect data downloaded or entered for personal use unless the app is provided by a covered entity or business associate. That is distinct from information held by a covered provider or a provider’s business associate. HHS also warns that other entities may access information in some consumer-app settings, including for advertising or data-broker uses. See HHS guidance on health information and personal devices. This U.S. guidance does not establish what laws apply in other jurisdictions.
Quick Recap
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
