Skip to content

feat(kepler-jupyter): version 0.4.0 - #3345

Merged
lixun910 merged 7 commits into
masterfrom
xli-update-kepler-jupyter
Mar 10, 2026
Merged

feat(kepler-jupyter): version 0.4.0#3345
lixun910 merged 7 commits into
masterfrom
xli-update-kepler-jupyter

Conversation

@lixun910

@lixun910 lixun910 commented Feb 24, 2026

Copy link
Copy Markdown
Collaborator

This pull request introduces a major refactor and modernization of the KeplerGL Jupyter Python package

  • use modern tools (uv for Python, esbuild for TypeScript)
  • use widget implementation based on anywidget
  • only need to publish to pypi (keplergl-jupyter npm package is no longer needed)

Note: the function save_to_html() will not be supported anymore:

  • Old approach: The old binding shipped a complete standalone keplergl.html file (a full kepler.gl build) specifically for HTML export. This is separate from the widget rendering.
  • New anywidget approach: The widget uses index.js and index.css which are designed for the Jupyter widget protocol, not standalone HTML. They expect the anywidget runtime and communication layer.
  • Users will see a helpful message explaining that the method is no longer supported and directing them to use the "Share" button in the map config panel instead.

Copilot AI review requested due to automatic review settings February 24, 2026 21:55

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Modernizes the keplergl Jupyter Python package by migrating to an anywidget-based widget implementation, updating the frontend build to esbuild, switching Python tooling to uv, and adding serialization/test scaffolding to support the new widget data flow.

Changes:

  • Introduces a new anywidget-based KeplerGl Python widget and a React/Redux frontend that renders kepler.gl in Jupyter.
  • Adds Python↔JS dataset serialization (DataFrame/GeoDataFrame/CSV/GeoJSON) and corresponding pytest coverage.
  • Updates packaging, build tooling, docs, notebooks, and CI to build/publish a PyPI-only distribution.

Reviewed changes

Copilot reviewed 29 out of 32 changed files in this pull request and generated 4 comments.

Show a summary per file
File Description
bindings/python/tsconfig.json Adds TypeScript compiler config for the new widget frontend.
bindings/python/tests/test_widget.py Adds widget construction and basic behavior tests.
bindings/python/tests/test_serializers.py Adds serializer tests for multiple dataset input types.
bindings/python/tests/conftest.py Adds pandas/geopandas fixtures used by widget/serializer tests.
bindings/python/src/widget.ts Implements the widget runtime: mount/unmount, config/data loading, kepler.gl dispatches.
bindings/python/src/utils/serialization.ts Adds base64 helpers intended for binary payload handling.
bindings/python/src/utils/data.ts Adds dataset processing for CSV/GeoJSON/DF and Arrow/GeoArrow payloads.
bindings/python/src/types.ts Defines widget model and dataset/config payload types.
bindings/python/src/styles.css Adds layout CSS to ensure kepler.gl/maplibre sizing works in Jupyter.
bindings/python/src/store.ts Adds a minimal Redux store with a mounted kepler.gl reducer instance.
bindings/python/src/index.ts Anywidget entry: wires model changes to widget instance lifecycle.
bindings/python/src/components/App.tsx Renders injected kepler.gl container and observes size changes.
bindings/python/pyproject.toml Adds modern Python packaging config and hatch-jupyter-builder integration.
bindings/python/package.json Adds JS build/typecheck/lint scripts and frontend dependencies.
bindings/python/notebooks/hex_config.py Adds example configuration asset for notebook demos.
bindings/python/notebooks/hex-data.csv Adds example CSV dataset for notebook demos.
bindings/python/notebooks/geojson-data.json Adds example GeoJSON data for notebook demos.
bindings/python/notebooks/GeoJSON.ipynb Adds a GeoJSON usage notebook example for the new widget.
bindings/python/notebooks/GeoDataFrame.ipynb Adds a GeoDataFrame usage notebook example for the new widget.
bindings/python/notebooks/DataFrame.ipynb Adds a DataFrame usage notebook example for the new widget.
bindings/python/keplergl/widget.py Adds the Python KeplerGl anywidget implementation and APIs.
bindings/python/keplergl/serializers.py Adds dataset serialization logic for Python→JS transfer.
bindings/python/keplergl/_version.py Adds package version source.
bindings/python/keplergl/init.py Defines package public API exports.
bindings/python/esbuild.config.mjs Adds the esbuild bundling configuration to emit static widget assets.
bindings/python/README.md Updates end-user installation/usage documentation for the new package.
bindings/python/DEVELOPMENT.md Adds contributor/development workflow documentation (uv/esbuild/testing/publishing).
.gitignore Ignores Python caches, notebook checkpoints, and generated widget static assets.
.github/workflows/build-publish-pypi.yml Updates CI to build/test with uv + esbuild and publish only to PyPI.
Comments suppressed due to low confidence (5)

bindings/python/src/utils/data.ts:12

  • DEBUG is set to true, which will log dataset contents/Arrow buffers to the console in normal operation. This is noisy and can significantly hurt performance for large datasets. Default this to false and/or gate it behind an environment/build flag.
const DEBUG = true;
function debug(...args: unknown[]) {
  if (DEBUG) {
    console.log('[keplergl-data]', ...args);
  }
}

bindings/python/keplergl/widget.py:53

  • add_data() / _add_data_dict() use copy.deepcopy(self.data) before inserting the new dataset. Since self.data can contain large DataFrame/GeoDataFrame objects, deep-copying can be very expensive and can duplicate underlying arrays. A shallow copy of the dict (copying only the mapping) is sufficient to trigger traitlets change detection without copying the datasets themselves.
    def add_data(self, data, name="data"):
        """Add data to the map."""
        updated = copy.deepcopy(self.data)
        updated[name] = data
        self.data = updated

    def _add_data_dict(self, data_dict):
        """Add multiple datasets from a dict."""
        updated = copy.deepcopy(self.data)
        for name, data in data_dict.items():
            updated[name] = data
        self.data = updated

bindings/python/src/widget.ts:51

  • this.store.subscribe(() => this.syncConfigToPython()); creates a persistent Redux subscription, but unmount() never unsubscribes. This can leak listeners when widgets are created/destroyed. Store the unsubscribe function and call it from unmount() (or avoid subscribing until syncConfigToPython is implemented).
    this.loadData();
    this.loadConfig();
    this.store.subscribe(() => this.syncConfigToPython());
  }

  unmount() {
    debug('unmount() called');
    this.root?.unmount();
    this.root = null;
  }

bindings/python/src/widget.ts:135

  • waitForInstance() always schedules the 5s timeout, and it will still fire (logging a timeout warning) even if the instance registers earlier via the store subscription. Track/clear the timeout when checkInstance() succeeds, and only warn/resolve on timeout if registration still hasn’t happened.
      // Otherwise subscribe and wait
      const unsubscribe = this.store.subscribe(() => {
        if (checkInstance()) {
          unsubscribe();
        }
      });

      // Timeout after 5 seconds
      setTimeout(() => {
        unsubscribe();
        console.warn('[keplergl-widget] Timeout waiting for instance registration');
        resolve();
      }, 5000);

bindings/python/src/utils/serialization.ts:16

  • This file defines base64 encode/decode helpers, but it isn’t imported anywhere in src/ (and data.ts reimplements base64 decoding directly). Consider either using these helpers from the data pipeline or removing the unused module to avoid dead code/duplication.
export function decodeBase64ToBytes(base64: string): Uint8Array {
  const binaryString = atob(base64);
  const bytes = new Uint8Array(binaryString.length);
  for (let i = 0; i < binaryString.length; i++) {
    bytes[i] = binaryString.charCodeAt(i);
  }
  return bytes;
}

export function encodeToBase64(bytes: Uint8Array): string {
  let binary = '';
  for (let i = 0; i < bytes.byteLength; i++) {
    binary += String.fromCharCode(bytes[i]);
  }
  return btoa(binary);
}

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread bindings/python/keplergl/serializers.py Outdated
Comment thread bindings/python/src/components/App.tsx
Comment thread bindings/python/tests/test_serializers.py Outdated
Comment thread bindings/python/pyproject.toml
@lixun910 lixun910 changed the title [Kepler-Jupyter]: version 0.4.0 feat(kepler-jupyter): version 0.4.0 Feb 25, 2026

@chrisgervang chrisgervang left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey @lixun910, this looks great. I have a couple thoughts

I'd be interested in seeing if we could modernize pydeck in a similar way, and ideally get some help from you to do it given how great this looks.

As I recall, the "Share" button has an export HTML feature that's more or less feature equivalent to save_to_html() except that it must be triggered by the user in the UI. I agree with the decision to remove it given the complexity and lack of jupyter protocol support for this kind of feature.

Also, I think removing it also resolves a long-standing issue outlined in #3139. Could you take a look and let me know if that is true? In practice, this means users will need to supply their own token to use Mapbox-provided tiles - mostly concerning satellite tiles. Mapbox is nice enough to donate a token for use in kepler's hosted apps and docs, and has requested we not embed them in exports going forward since people re-host them and increase costs.

We've received feedback over the years whenever satellite tiles or the html export has broken, my 2cents is to make a note of these intentional changes in an upgrade guide.

Have you considered making this a major release considering it comes with a breaking change to the API?

@lixun910

Copy link
Copy Markdown
Collaborator Author

@chrisgervang ​Thanks for the approval! I’d be happy to help modernize pydeck as well. This PR involved a lot of work with Claude Opus 4.5 in cursor, and I’d love to share that process with you to streamline the next steps.

​Regarding your points:

​Yes, removing save_to_html is significant. Maybe we lead with a Release Candidate (RC) to gauge community impact before the final release? Since there are no "new" features, a major announcement might not be necessary, but I agree we should make this a Major Release given the API changes.

​For Mapbox token, yes, I agree with you. I’ll implement it in this PR, so users with both versions have the same solution.

​I’ll put together a Upgrade Guide to explain these intentional changes and the transition to the UI-based "Share" button.

Thank you! 🙏

@lixun910
lixun910 merged commit faa000c into master Mar 10, 2026
7 of 8 checks passed
@lixun910
lixun910 deleted the xli-update-kepler-jupyter branch March 10, 2026 18:37
bdjulbic pushed a commit to bdjulbic/kepler.gl that referenced this pull request Mar 16, 2026
* update kepler-jupyter to 0.4.0

* version to 0.4.0rc1

* add tests; update uv.lock

* address comments

* fix lint

* Update test.yml

* Update duckdb-table.ts

Signed-off-by: bdjulbic <bdjulbic@foursquare.com>
@karosc

karosc commented Apr 9, 2026

Copy link
Copy Markdown
Contributor

Just noticing this PR now. I use kepler.gl to automatically generate map artifacts directly from python. It seems like that kind of workflow would no longer supported in this version of kepler for python?

@lixun910, is that correct? In other words, to generate keplergl map artifacts in this new version requires user interaction in a jupyter widget?

@lixun910

lixun910 commented Apr 9, 2026

Copy link
Copy Markdown
Collaborator Author

@karosc Yes, it is for now in the 0.4.0rc1. Is the save_to_html is critical feature for you?

@karosc

karosc commented Apr 9, 2026

Copy link
Copy Markdown
Contributor

@lixun910 : Yes, I use the python library to generate map reports automatically outside of jupyter entirely. If the official version of keplergl won't support that functionality anymore, I'll need to resort to maintaining my own fork of the older version.

@lixun910

lixun910 commented Apr 9, 2026

Copy link
Copy Markdown
Collaborator Author

@karosc I see. Let me work on bring the save_to_html function to v0.4.0

@lixun910

Copy link
Copy Markdown
Collaborator Author

@karosc Can you help to test if version 0.4.0rc2 works as you expected. Thanks! https://pypi.org/project/keplergl/0.4.0rc2/

@karosc

karosc commented Apr 23, 2026

Copy link
Copy Markdown
Contributor

@lixun910, I can take a look tomorrow and let you know if I have any issue.

@karosc

karosc commented Apr 24, 2026

Copy link
Copy Markdown
Contributor

@lixun910 here is my feedback:

Possible Bugs

  1. My map config does not seem to be working. I works fine with the older version of kepler. When I load up my geodataframes in this new version, they are loaded in the map, but the styles in my config are not applied. I cannot see where the problem is. Are you able to apply a visconfig successfully using the config dict?
  2. serialize_geodataframe breaks when the gdf is empty and thus ga.as_geoarrow cannot resolve the geometry type. In that case, I want the layer to still show up in my map (indicating that my analysis was run, but no meaningful outputs were found). Adding this kind of logic resolves the issue for me:
    geom_series = gdf.geometry
    if geom_series.notna().any():
        geom_array = ga.as_geoarrow(geom_series)
    else:
        # GeoArrow cannot infer a concrete geometry spec from an empty/all-null series.
        geom_array = ga.as_geoarrow(geom_series, type=ga.wkb())
  3. _dataset_to_geojson does not work when a geodataframe as datetime values in it. This was not a problem in the last version. I can pre-process my own data prior to inserting to the html, but it feels like if I can successfully run map.add_data, I should be able to run map.export_to_html. Although it's a bit dangerous, you can change data.to_json() to data.to_json(default=str), which will just push the datetime values through str. Or we could build some kind of argument into export_to_html that would allow us to customize data serialization.

Possible Improvements

  1. I also think it would be nice if users could specify the app theme here. I personally use the mapbox light theme, and having the Kepler UI also be light would also be nice. Event being able to add a title to the html file other then kepler-gl by changing appName would be a nice touch.

I am happy to assist with a PR on any of these if you'd like.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants