v0.17.0-beta

⚠️ Beta release. Review potential breaking changes before upgrading.

---

Table 1

Table 1 is now generated directly inside D2E. Table 1 is the standard baseline characteristics table included in virtually every observational study. A new configuration dialog lets you define which concept sets and variables to include, and the underlying backend now supports concept sets natively, so your Table 1 stays tied to the same cohort logic you've already defined elsewhere in the platform. Results can be exported directly as Parquet.

Previously, producing this table meant exporting cohort data and building it manually in R, Excel, or another external tool which was an extra step, and a risk that the table used in a manuscript quietly drifted from the platform's actual cohort definition. With Table 1 built into D2E, that step and that risk goes away.

Open the Table 1 dialog using the "Analyze" button on the cohort builder to get started.

---

AI chat assistant improvements

The assistant can also now manage concept sets directly, via MCP (Model Context Protocol). Instead of navigating the concept set UI by hand, you can ask the assistant to look up, create, or refine concept sets as part of a conversation.

This is part of our continued efforts to improve the user experience through judicious use of AI features.

---

ATLAS 3.0 preview (Experimental)

This release includes an early preview of ATLAS 3.0. It's an opportunity to explore what's coming next in the OHDSI toolchain directly from within D2E, ahead of a full rollout. Access it through your account page and try it out.

As a preview, this is best suited to teams who want early visibility into ATLAS 3.0 rather than day-to-day production use just yet.

---

Improvements

Datasets

• i2b2 datasets can now be created through a specialised, dedicated creation flow.
• BigQuery connections can now be scoped to an entire project rather than a single dataset.

FHIR & TREX

• A dedicated TREX FHIR server has been added, and the FHIR transform package has been upgraded.
• Database variables in dataflow are now configurable.

Authentication & roles

• A custom OIDC connector is now available as a sign-in option.
• User roles are now synced to WebAPI via Logto scopes.
• Database password encryption has been enabled in WebAPI.

Concept sets & search

• A feature flag now controls concept-set record counts.
• Semantic-search flows can now run against HANA datasets.
• concept_synonyms are now included in full-text search.
• Concept-search scoring has been improved for better accuracy.

UI components

• New alert/dialog and alert-toast feedback components are available across the platform.
• Patient-list limits are now configurable in PA config.
• Result-viewer templates for new studies are now configurable.

HANA & TREX engine

• Cohort materialisation now runs via trex_hana_materialize_cohort, routed to the MDC tenant.
• Data Characterisation and DQD now run through the TREX pgwire passthrough.
• The hdb packet-size limit has been increased to support larger SQL statements.
• TREX has received multiple updates, including a new arm64 build and a connection pool size increase from 64 to 1024.

Query, logging & tooling

• Cohort-definition query performance has been improved.
• The audit logger is now asynchronous.
• The Hugging Face model version has been updated.
• Deno has been updated to 2.7.14, and Helm charts refreshed for develop.

Branding & labels

• The UI now uses IBM Plex Sans, and D2E branding has been refined.
• External fonts have been removed from the chat interface.
• Translations have been unified and improved; the "Email" column in the Permissions dialog is now labelled "Name"; permission grant/revoke text and analysis/overview labels have been updated.

Access & roles

• Dashboard/Viewer access can now be managed per dataset type.
• pid is now the default patient-list sorting attribute.
• Deletion is now disallowed when confirmation text doesn't match.
• Unused scope/role creation in TREX has been removed.

Fixes

PA & Cohort Builder

• Fixed cohort-builder layout, navigation, and concept-set filtering regressions.
• Fixed the "first time" filter defaulting incorrectly.
• Fixed the Patient Cohort filter-card dropdown not opening.
• Fixed the Wizards save button not revalidating after a cohort name change.
• Fixed mismatching patient counts between the materialised cohort and its bar chart.

Cohort Materialisation

• Fixed materialise failures.
• Fixed admin-credential HANA writes.
• Fixed raw (non-URL-encoded) HANA password passing.

Data Characterisation

• Retained partial results with better failure diagnostics.
• Fixed treemap-table node selection and name parsing.
• Fixed PATIENT_ID casing in attrition plots.
• Fixed the HANA results-table DDL for the pgwire passthrough.

FHIR & Dataflow

• Fixed general FHIR bugs.
• Fixed source-dataset WebAPI sync.
• Reverted sqlalchemy-hana installation during flow runs.

Charts & Export

• Fixed empty CSV / error toast on PA file export.
• Fixed overlapping-chart legends.
• Fixed attrition-plot dialog behaviour.

Misc

• Fixed base64 encoding replacements.
• Fixed an unauthorised Docker logs issue.
• Fixed notebook kernels disconnecting on unmount.
• Fixed non-string structure_map handling.
• Fixed a local k8s issue.

Potential breaking changes

None flagged in this release's changelog. As always, with beta software, review your deployment configuration before upgrading.

Full Changelog: v0.16.0-beta...v0.17.0-beta

---

Questions? Join the Data2Evidence Slack or open an issue.

v0.16.0-beta

⚠️ Beta release. Review potential breaking changes before upgrading.

---

WebR notebooks

WebR is now available as a UI plugin. Researchers can run R-based analysis directly in the browser without a server-side R kernel. This is intended for ad-hoc exploration and lightweight analytics that don't require a full notebook environment.

Open the Notebooks section to find and launch a WebR notebook.

---

Cohort builder improvements

Two new chart types are now available for distribution analysis: an overlapping histogram and a KDE (Kernel Density Plot). The overlapping histogram lets you compare distributions across cohort groups in a single view without switching between charts. The KDE plot renders a smoothed probability density curve if you want to see the shape of a continuous variable's distribution without the binning artefacts of a standard histogram. Both chart types are saved per bookmark, so your preferred view persists across sessions.

The attrition plot is also new in this release. It visualizes how the patient population reduces at each inclusion and exclusion criterion, giving you a clear picture of where and why patients drop out of a cohort. This is useful for validating cohort logic and communicating study design to collaborators.

---

Save concept mappings between sessions

One of the most common frustrations in concept mapping work is having to redo the same decisions when returning to a dataset. Starting in v0.16.0, mappings can be saved directly to the dataset database via a new "Save to database" action in the mapping workflow.

Once saved, mappings appear in a new "Saved Mappings" tab in the concept mapping overview, so your team can review and build on previous work without starting from scratch each session.

---

FHIR-to-OMOP data lineage

For teams running FHIR-to-OMOP transformations, a new set of data lineage tables now tracks each transformation from source FHIR resource to CDM record making it convenient to trace exactly how a CDM observation, condition, or measurement was derived from the source data.

---

Cross-platform binaries

The D2E CLI is now available as a standalone binaries for each platform. Node.js is no longer required. You can download the binary and run it directly.

---

Sign in with Microsoft Entra External ID

Organisations using Microsoft Entra External ID can now use it as a sign-in provider in D2E. The new Logto connector handles the integration, enabling enterprise SSO without additional identity infrastructure on the D2E side.

---

Improvements

• Authentication
  • Role sync runs automatically at startup, paginates large scope lookups, and reports skipped entries. An admin prompt guides initial setup.
  • The login page logo can now be customized per deployment.
• Concept sets: Columns in concept set tables can now be sorted.
• Concept search
  • Standard search now correctly handles descendant concepts.
  • Search accuracy on HANA has been improved, and hybrid search is now supported on HANA datasets.
  • Terms containing commas now return correct results.
• Cohort building
  • Condition filters now offer an option to explicitly exclude descendant concepts.
  • Materialisation retries automatically on transient failures, reducing the need for manual re-runs.
  • Cohort counts throughout the UI now include thousand separators.
  • Multi-parent interaction structures are now supported, expanding coverage for datasets with complex domain hierarchies.
• Chat: Ollama and other local LLM backends are now supported in addition to existing providers, giving operators the option to run inference entirely within their own environment.
• Population Analytics: Filter card reset and advanced time behaviour now work predictably across filter types.
• Dashboards: Ethnicity and race values are now synced from required fields, keeping dashboard displays consistent.
• Notebooks
  • Navigating back to a notebook now starts with a fresh kernel, preventing stale state from affecting new analyses.
  • Duplicate notebook names are resolved automatically on creation.
• Study management
  • Administrators can now delete a study directly from the platform UI.
  • The dataset selection step has been streamlined in the study deletion flow.
• Datasets
  • Source vocabulary schema can now be updated and the token study code is shown for reference.
  • Datasets managed by WebAPI are now better aligned with the platform cache.
  • Strategus results are stored in a dedicated DuckDB file that is created and attached automatically.
  • The SSL mode field now defaults to "allow" when a CA certificate is configured but no SSL mode has been set.
• ETL
  • Flow templates are validated before execution begins, surfacing configuration errors early.
  • Exported flow JSON now follows the flow template format for easier versioning and reuse.
• HANA
  • Drivers are auto-installed during deployment, removing a manual setup step.
  • A default query hint has been added to improve performance on HANA-backed datasets.
• Platform
  • Loading indicators are shown during user management actions and permission updates.
  • General performance improvements, restructured caching, and enhanced SQL tracing across services.

---

Fixes

• Admin: Fixed the admin console becoming inaccessible in certain configurations.
• Patient List: Fixed failing to load when Entry/Exit columns were enabled.
• Data Characterisation
  • Fixed failures on rerun when a custom results schema was configured.
  • Fixed the flow on HANA datasets.
• Cache
  • Fixed the result schema cache not updating correctly after schema changes.
  • Fixed snapshot copies incorrectly duplicating the result schema.
• Result viewer: Fixed permissions not applying correctly.
• Cohort building
  • Fixed the "Materialise cohort" button being available when it should not be.
  • Fixed the advanced time option appearing on basic filter cards.
• Scope management: Fixed approve/reject actions affecting the wrong row.
• Notebooks: Fixed an infinite recursion problem.
• Bookmarks: Fixed bookmark state being lost when a bookmark was renamed.
• Datasets: Fixed conversion to WebAPI format failing in some cases.
• HANA: Fixed schema casing inconsistencies during dataset creation.
• UI
  • Fixed translations on the concepts page and corrected the password change label.
  • Fixed dropdown menus rendering behind other UI elements.
• Connections: Fixed a database connection error that could prevent certain dataset queries from completing.
• Authentication: Fixed Logto console access being blocked in certain configurations.

---

Potential breaking changes

• use_cache_db removed: The use_cache_db configuration setting has been removed. Update your deployment configuration before upgrading if you set this value.

---

Questions? Join the Data2Evidence Slack or open an issue.

v0.15.0-beta

What's Changed

Key Features

• Wizards & PA Configuration: Wizards now read from PA configuration, enabling per-deployment customization of the wizard experience.
• Researcher Portal Customization: Customizable researcher banner image and customizable Data Quality static text via public configs endpoint, plus a public legal page rendered from configuration.
• Notebook Integration: Added notebook support directly within the researcher workflow.
• ETL Enhancements: Enabled the Terminology window in Admin and ETL views, enforced unique node names in ETL, and prevented creation of flows with duplicate names. Added import for ETL artifacts.
• Dashboards & Wizards: Dynamic Shiny dashboard configuration based on study analysis specifications. Added studyId selection dropdown in analysis UI.
• Cohort Builder UX: Column header tooltips and sorting, a reset button to clear bar chart position and selection, per-bar coloring by x-axis category for non-stacked bar charts, attrition stats recalculation on reorder, and removal of the soft 10-bookmark cap.
• Concept Sets: Empty-state messaging for the Concept Sets list and improved validation for non-empty names.
• Editor Tooling: Monaco editor syntax highlighting for SQL, Python, and R.
• Notifications: Send navigation notifications to Jobs when creating a dataset, datamart, or update-cache. Auto-close notebook notification bar after 5 seconds. Allow deletion of an individual flow run.
• Navigation: Breadcrumb navigation for config pages and an Analysis UI study dropdown.

Performance & Infrastructure

• HANA Performance: Implemented HANA performance improvements for query and dataset operations.
• Cohort Loading: Improved loading time of the cohort management tab and optimized the cohortdefinitions?source=pa endpoint.
• State Management: Added Pinia for state management in the Vue application.
• Connection Cleanup: Removed direct Postgres connection code from analyze services, consolidating database access through Trex.
• Plugin Size: Reduced the functions plugin size for faster deploys.
• TREX Engine Updates: Multiple TREX updates with query and runtime improvements; fixed TREX Docker build.
• DevOps: Added build context in docker-compose-local.yml. Always run CI steps for PRs against release. Make demo dataset setup idempotent. Added platform setting for Supabase.
• Supabase Storage: Fixed retrieval of files from Supabase storage.

Fixes

• Patient Analytics: Fixed misc chart and button UI issues, stack bar state reset on empty plotly selection, attrition plot exclusion rules numbers and display, drilldown values when multiple selections occur, barchart formatData ignoring 0 in truthy check, the "No patient group" message on chart filter, and stack bar chart colors and default config.
• Cohort Builder: Fixed filtercard rename dialog feedback, double-fetch on load, inconsistent filter/group behavior, date input hydration in bookmarks, copy-SQL button state when charts are busy, and tooltip text for disabled cohort materialization.
• Data Characterisation: Fixed selection border behavior on treemap charts and updated attrition/treemap CSV export.
• FHIR & ETL: Fixed FHIR-to-OMOP transform, added rollback on FHIR project creation failure, fixed ETL feature flag (with e2e), fixed pyqe incorrect function argument, and updated study dataset database code on execution.
• Plugins: Fixed the loyalty-score plugin's eligible person id query, the semantic search plugin, and the concept recommendation flow.
• UI Resets: Reset dataset resources to empty on error, reset form / restore default button on banner config, and fix metadata attribute edit/delete notifications.
• Misc: Fixed Open Dashboard disease/condition dropdown not appearing, job run page pagination, and bundled bug fixes.

Maintenance

• Authentication & Roles: Propagate user-management roles to Logto roles, reference the new Logto image, read email from Logto JWT, and removed the deprecated study manager role.
• i18n: Sentence case across labels, fixed untranslated labels, and removed unused translations.
• UI Polish: Thousands separators, node title word-break, aligned chart toolbar buttons, repositioned Data Characterisation tab, and added static text on the Data Quality page.
• Testing: Added e2e for hybrid search embedding, used DuckDB for HTTP tests (with file-change conditional execution), playwright requests without cache, e2e CDW validation fixes, and updated e2e test code owners.
• HADES & Strategus: Updated Strategus commit in HADES packages and updated renv.lock. Updated build ref for PLP SQL translation to DuckDB format.
• CLI: Removed console logs in CLI output.

v0.14.0-beta

What's Changed

Key Features

• Cohort Inclusion Report: New inclusion report endpoint and UI showing step-by-step patient attrition, with support for explicit exclusions and basic data filters. Added feature flag for gradual rollout.
• Dashboards: Enabled dashboard viewing directly from the Cohort Builder, including Shinylive dashboard restoration and Wizards integration.
• Strategus Result Viewer: Integrated the researcher portal Strategus result viewer for viewing population-level effect estimation results.
• FHIR Cache Updates: Added support for cache updates on FHIR datasets, with project-scoped cache instead of system-level, and fixed race conditions between project creation and cache flow triggers.
• ETL Concept Mapping: Exposed ETL concept mapping functionality via a dedicated researcher role, enabling researchers to review and manage vocabulary mappings.
• MCP Improvements: Enhanced Model Context Protocol integration for AI-assisted clinical data exploration.

Performance & Infrastructure

• HANA Streaming: Implemented streaming for materializing cohorts on SAP HANA, improving performance for large cohort generation.
• DuckDB Memory Management: Added configurable DuckDB memory limits for Data Characterisation to prevent out-of-memory issues.
• HANA TLS Support: Added option to pass HANA TLS certificates via environment variable and explicit TLS input fields in database configuration.
• Offline Installation: Added support for offline installation of Supabase storage components.
• TREX Engine Update: Updated to latest TREX version with query improvements.
• CLI Version Command: Added d2e version command for easier deployment verification.
• Java Heap Configuration: Configurable JAVA_TOOL_OPTIONS with heap size set via environment variable in .Rprofile.

Fixes

• Cohort Builder: Fixed Advanced Time days field rejecting < and > inputs, inconsistent filter/group behavior, nested filtercard splitting into individual exclusions, and filtercard rename dialog feedback.
• Data Characterisation: Fixed DC/DQD when database code matches schema name. Updated tooltip and legend for treemaps. Fixed StackBarChart selection and x-axis label truncation.
• UI/UX: Fixed stale results on failed concept search, import notebook button hover effect, PA patient list row borders, missing user feedback on bookmark deletion, and concept code handling with hyphens.
• Caching: Fixed cache creation when vocab schema and CDM schema are the same. Fixed FHIR project creation race condition.
• Backend: Fixed regex path typo, inclusion report @deadperson placeholder, basic data filter for group-by queries, and Table 1 result viewer template.
• Authentication: Added email claim support in custom JWT and Logto JWT email reading.

Maintenance

• Feature Flags: Show wizards in feature flags, added visible and defaultEnabled properties for feature flag configuration, disabled add-to-cohort button for incompatible datasets.
• UI Improvements: Improved BaseTagInput option selection and dropdown behavior, enhanced TypeaheadField filtering and sorting logic, added AnalyticsIcon component. Set default result drawer to hidden for Cohort Method Node.
• Data & Schema: Added cohort_definition table to demo_cdm_results schema. Added not-null constraints separately. Enforced type selection for HANA dataset creation. Fixed MIMIC dataflow. User-accessible dataset with researcher role.
• Studies: Study table overhaul with improved structure and organization.
• Dataflow: UI changes for dataflow management. Show disclaimer dialog only once per session. Support multiple places for Enter key form submission. Transform legacy Internal in db_extra.
• Testing: Added end-to-end tests for materialize cohort workflow.

v0.13.0-beta

What's Changed

Key Features

• Data Export: Added support for exporting data in both Parquet and JSON formats, with dynamic request body filters.
• Dashboards & Workspaces: Added support for configuring multiple dashboards, introduced a researcher portal dashboard, and enabled notebook tabs for HANA datasets.
• Strategus & Results: Enabled upload/download functionality for Strategus results, added a Table 1 flow (saving to the TREX database), and introduced a Table 1 module to the Shiny result viewer.

Performance & Infrastructure

• Optimization: Optimized the snapshot metadata endpoint, removed direct DuckDB connections, and eliminated dataset metadata caching in analytics-svc.
• Prefect Flows: Added d2e to the Prefect base path, configured environment-variable-driven timeouts for cache flows, and added user_name to standalone phenotype flows.
• Visualization: Optimized axes for Data Characterisation charts, updated treemap and table drilldowns, and improved TypeaheadField filtering and sorting.
• TREX Engine: Updated TREX, adjusted health checks, enabled debug logging for dev environments, and configured automatic clearing of TEMP tables.

Fixes

• UI/UX: Fixed rendering issues with the Portal, Stackbar Chart, and Shinylive. Resolved bugs related to hiding filter panels, refreshing the concepts page, and FHIR project creation.
• Backend & API: Fixed multiple Tokio channel and V8 isolate issues, resolved error propagation from failed validation checks in cdw-svc, and handled missing text items in fetchAttributeValues.
• Auth & Integrations: Added a fallback for empty Entra groups claims and fixed the JWT token issue for Starboard embeds.
• Caching & Results: Resolved bugs in the result viewer and fixed caching issues during the creation of results schemas.

Maintenance

• Clean Up: Aligned dashboard implementations, removed NiFi-related codes, renamed resultSchema to resultsSchema, and configured single-spa to unload when the researcher renderer unmounts.
• DevOps: Migrated PA to Vite, updated the Logto base image, fixed CLI composeFile, improved HTTP test actions, and set up E2E for easier local runs.
• Quality of Life: Ensured disclaimer dialogs show only once per session, maintained active concept sets during pagination, and mapped the "Enter" key to trigger the AddCohort component.

Full Changelog: v0.12.1-beta...v0.13.0-beta

v0.12.1-beta

What's Changed

• Update logto base image by @satish-a0 in #1682
• [Fix] fix plugins ver in cli by @mwaiyee in #1676
• [Analyze 1680] Add catch on error when db operations fail for concept records count by @brandantck in #1686
• [1680] [1693] hot fix for v0.12 by @suwarnoong in #1694
• [Fix] cli composeFile for v0.12 by @mwaiyee in #1709

Full Changelog: v0.12.0-beta...v0.12.1-beta

v0.12.0-beta

What's Changed

Key Features

• Deep Linking: Added support for sharing cohort definitions via deep links.
• UI Enhancements: Enabled drag-and-drop for Patient Analytics (PA) filter cards and added Action Dialogs for Studies.
• Analysis & AI: Integrated the Model Context Protocol (MCP) client and added the CohortSurvival node to analysis flows.
• MRI Query: Added mriquery support to cohort definition syntax.

Performance & Infrastructure

• Concurrency: Switched analyze services to tokiochannel for improved backend performance.
• Optimization: Improved cache plugin (chunked copying), optimized database connections (retries/reconnection), and added SSL support for test DBs.
• Visualization: Enhanced data characterization charts and optimized concept search performance.

Fixes

• Integration: Added OMOP 5.4 support for HANA, fixed Entra connector for duplicate usernames, and resolved White Rabbit file upload errors.
• Cohorts: Fixed UI validation for save names and resolved type mismatches when updating concept sets from PA.
• Schema: Fixed schema naming issues during Data Model creation and addressed data seeding bugs.

Maintenance

• Breaking Change: Renamed environment variables from ALP to D2E.
• Clean Up: Removed legacy code (cleansed schema, built-in plugins) and reconciled Portal/Single SPA concept sets.
• DevOps: Updated trex dependencies (Deno 2.5.6) and refined CI/CD workflows.

v0.11.0-beta

What's Changed

New Features

• Dashboards: Integrated Shiny dashboards into datasets based on result viewer functionality.
• Architecture: Implemented Single-SPA architecture for Analysis Flow, Notebook UI, and React Portal.
• Data Transformation: Added a new transform node to map FHIR to OMOP using structure maps.
• MCP Tools: Added MCP tools for cohort definition creation and Strategus-based code generation context.
• Search: Implemented descendant variant 2 search and domain value search for cohort definitions.

Enhancements

• Cohort Management: Materialized cohorts to both cache and source DB, improved definition list retrieval, and switched to counting distinct subject IDs.
• Patient Analytics (PA): Added execute side panel for PA Atlas, configurable standalone logos, and tooltips for icon buttons.
• Validation: Enhanced input validation for filter criteria, cohort names, and datamart token dataset codes.
• Data Integration: Enabled async execution for Python nodes, supported token dataset code display, and added cache refresh for OMOP datasets.
• Security: Introduced auto-grant researcher roles by dataset codes and a new role for study results.

Bug Fixes

• Stability: Stabilized Strategus websocket connections and handled binary data support.
• UI/UX: Fixed app crashes on chart selection, errors when filtering binned attributes, and notebook chat scrollbar issues.
• Logic: Corrected date conversion in query generation, White Rabbit flows, and concept search filters.
• Configuration: Fixed issues with opening CDM configs and repeated axis prefixes in tooltips.

Infrastructure & DevOps

• Package Management: Replaced Yarn with Bun.
• Trex: Updated Trex integration, including SQL usage for embedding search and storage plugin conversion.
• CI/CD: Updated to stable enterprise gateway base image and added environment support for Codespaces.
• Testing: Expanded test coverage for cohort creation, filter cards, outcomes nodes, and E2E scenarios.

v0.10.2-beta

What's Changed

• Fixes dc bq by @satish-a0 in #1137
• [Integrate-1075] update term-of-use and privacy-policy URL in Logto i… by @satish-a0 in #1138
• Responsive dropdown by @satish-a0 in #1139
• Patch version & skip existing schema by @satish-a0 in #1140

Full Changelog: v0.10.1-beta...v0.10.2-beta

v0.10.1-beta

What's Changed

• [Analyze-793] Add results schema to trex connection and translation by @brandantck in #1043
• Fix functions envs by @satish-a0 in #1048
• Patch v0.10.1 by @mwaiyee in #1067

Full Changelog: v0.10.0-beta...v0.10.1-beta