feat(kepler-jupyter): version 0.4.0 - #3345
Conversation
There was a problem hiding this comment.
Pull request overview
Modernizes the keplergl Jupyter Python package by migrating to an anywidget-based widget implementation, updating the frontend build to esbuild, switching Python tooling to uv, and adding serialization/test scaffolding to support the new widget data flow.
Changes:
- Introduces a new
anywidget-basedKeplerGlPython widget and a React/Redux frontend that renders kepler.gl in Jupyter. - Adds Python↔JS dataset serialization (DataFrame/GeoDataFrame/CSV/GeoJSON) and corresponding pytest coverage.
- Updates packaging, build tooling, docs, notebooks, and CI to build/publish a PyPI-only distribution.
Reviewed changes
Copilot reviewed 29 out of 32 changed files in this pull request and generated 4 comments.
Show a summary per file
| File | Description |
|---|---|
| bindings/python/tsconfig.json | Adds TypeScript compiler config for the new widget frontend. |
| bindings/python/tests/test_widget.py | Adds widget construction and basic behavior tests. |
| bindings/python/tests/test_serializers.py | Adds serializer tests for multiple dataset input types. |
| bindings/python/tests/conftest.py | Adds pandas/geopandas fixtures used by widget/serializer tests. |
| bindings/python/src/widget.ts | Implements the widget runtime: mount/unmount, config/data loading, kepler.gl dispatches. |
| bindings/python/src/utils/serialization.ts | Adds base64 helpers intended for binary payload handling. |
| bindings/python/src/utils/data.ts | Adds dataset processing for CSV/GeoJSON/DF and Arrow/GeoArrow payloads. |
| bindings/python/src/types.ts | Defines widget model and dataset/config payload types. |
| bindings/python/src/styles.css | Adds layout CSS to ensure kepler.gl/maplibre sizing works in Jupyter. |
| bindings/python/src/store.ts | Adds a minimal Redux store with a mounted kepler.gl reducer instance. |
| bindings/python/src/index.ts | Anywidget entry: wires model changes to widget instance lifecycle. |
| bindings/python/src/components/App.tsx | Renders injected kepler.gl container and observes size changes. |
| bindings/python/pyproject.toml | Adds modern Python packaging config and hatch-jupyter-builder integration. |
| bindings/python/package.json | Adds JS build/typecheck/lint scripts and frontend dependencies. |
| bindings/python/notebooks/hex_config.py | Adds example configuration asset for notebook demos. |
| bindings/python/notebooks/hex-data.csv | Adds example CSV dataset for notebook demos. |
| bindings/python/notebooks/geojson-data.json | Adds example GeoJSON data for notebook demos. |
| bindings/python/notebooks/GeoJSON.ipynb | Adds a GeoJSON usage notebook example for the new widget. |
| bindings/python/notebooks/GeoDataFrame.ipynb | Adds a GeoDataFrame usage notebook example for the new widget. |
| bindings/python/notebooks/DataFrame.ipynb | Adds a DataFrame usage notebook example for the new widget. |
| bindings/python/keplergl/widget.py | Adds the Python KeplerGl anywidget implementation and APIs. |
| bindings/python/keplergl/serializers.py | Adds dataset serialization logic for Python→JS transfer. |
| bindings/python/keplergl/_version.py | Adds package version source. |
| bindings/python/keplergl/init.py | Defines package public API exports. |
| bindings/python/esbuild.config.mjs | Adds the esbuild bundling configuration to emit static widget assets. |
| bindings/python/README.md | Updates end-user installation/usage documentation for the new package. |
| bindings/python/DEVELOPMENT.md | Adds contributor/development workflow documentation (uv/esbuild/testing/publishing). |
| .gitignore | Ignores Python caches, notebook checkpoints, and generated widget static assets. |
| .github/workflows/build-publish-pypi.yml | Updates CI to build/test with uv + esbuild and publish only to PyPI. |
Comments suppressed due to low confidence (5)
bindings/python/src/utils/data.ts:12
DEBUGis set totrue, which will log dataset contents/Arrow buffers to the console in normal operation. This is noisy and can significantly hurt performance for large datasets. Default this tofalseand/or gate it behind an environment/build flag.
const DEBUG = true;
function debug(...args: unknown[]) {
if (DEBUG) {
console.log('[keplergl-data]', ...args);
}
}
bindings/python/keplergl/widget.py:53
add_data()/_add_data_dict()usecopy.deepcopy(self.data)before inserting the new dataset. Sinceself.datacan contain largeDataFrame/GeoDataFrameobjects, deep-copying can be very expensive and can duplicate underlying arrays. A shallow copy of the dict (copying only the mapping) is sufficient to trigger traitlets change detection without copying the datasets themselves.
def add_data(self, data, name="data"):
"""Add data to the map."""
updated = copy.deepcopy(self.data)
updated[name] = data
self.data = updated
def _add_data_dict(self, data_dict):
"""Add multiple datasets from a dict."""
updated = copy.deepcopy(self.data)
for name, data in data_dict.items():
updated[name] = data
self.data = updated
bindings/python/src/widget.ts:51
this.store.subscribe(() => this.syncConfigToPython());creates a persistent Redux subscription, butunmount()never unsubscribes. This can leak listeners when widgets are created/destroyed. Store the unsubscribe function and call it fromunmount()(or avoid subscribing untilsyncConfigToPythonis implemented).
this.loadData();
this.loadConfig();
this.store.subscribe(() => this.syncConfigToPython());
}
unmount() {
debug('unmount() called');
this.root?.unmount();
this.root = null;
}
bindings/python/src/widget.ts:135
waitForInstance()always schedules the 5s timeout, and it will still fire (logging a timeout warning) even if the instance registers earlier via the store subscription. Track/clear the timeout whencheckInstance()succeeds, and only warn/resolve on timeout if registration still hasn’t happened.
// Otherwise subscribe and wait
const unsubscribe = this.store.subscribe(() => {
if (checkInstance()) {
unsubscribe();
}
});
// Timeout after 5 seconds
setTimeout(() => {
unsubscribe();
console.warn('[keplergl-widget] Timeout waiting for instance registration');
resolve();
}, 5000);
bindings/python/src/utils/serialization.ts:16
- This file defines base64 encode/decode helpers, but it isn’t imported anywhere in
src/(anddata.tsreimplements base64 decoding directly). Consider either using these helpers from the data pipeline or removing the unused module to avoid dead code/duplication.
export function decodeBase64ToBytes(base64: string): Uint8Array {
const binaryString = atob(base64);
const bytes = new Uint8Array(binaryString.length);
for (let i = 0; i < binaryString.length; i++) {
bytes[i] = binaryString.charCodeAt(i);
}
return bytes;
}
export function encodeToBase64(bytes: Uint8Array): string {
let binary = '';
for (let i = 0; i < bytes.byteLength; i++) {
binary += String.fromCharCode(bytes[i]);
}
return btoa(binary);
}
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
chrisgervang
left a comment
There was a problem hiding this comment.
Hey @lixun910, this looks great. I have a couple thoughts
I'd be interested in seeing if we could modernize pydeck in a similar way, and ideally get some help from you to do it given how great this looks.
As I recall, the "Share" button has an export HTML feature that's more or less feature equivalent to save_to_html() except that it must be triggered by the user in the UI. I agree with the decision to remove it given the complexity and lack of jupyter protocol support for this kind of feature.
Also, I think removing it also resolves a long-standing issue outlined in #3139. Could you take a look and let me know if that is true? In practice, this means users will need to supply their own token to use Mapbox-provided tiles - mostly concerning satellite tiles. Mapbox is nice enough to donate a token for use in kepler's hosted apps and docs, and has requested we not embed them in exports going forward since people re-host them and increase costs.
We've received feedback over the years whenever satellite tiles or the html export has broken, my 2cents is to make a note of these intentional changes in an upgrade guide.
Have you considered making this a major release considering it comes with a breaking change to the API?
|
@chrisgervang Thanks for the approval! I’d be happy to help modernize pydeck as well. This PR involved a lot of work with Claude Opus 4.5 in cursor, and I’d love to share that process with you to streamline the next steps. Regarding your points: Yes, removing save_to_html is significant. Maybe we lead with a Release Candidate (RC) to gauge community impact before the final release? Since there are no "new" features, a major announcement might not be necessary, but I agree we should make this a Major Release given the API changes. For Mapbox token, yes, I agree with you. I’ll implement it in this PR, so users with both versions have the same solution. I’ll put together a Upgrade Guide to explain these intentional changes and the transition to the UI-based "Share" button. |
* update kepler-jupyter to 0.4.0 * version to 0.4.0rc1 * add tests; update uv.lock * address comments * fix lint * Update test.yml * Update duckdb-table.ts Signed-off-by: bdjulbic <bdjulbic@foursquare.com>
|
Just noticing this PR now. I use kepler.gl to automatically generate map artifacts directly from python. It seems like that kind of workflow would no longer supported in this version of kepler for python? @lixun910, is that correct? In other words, to generate keplergl map artifacts in this new version requires user interaction in a jupyter widget? |
|
@karosc Yes, it is for now in the 0.4.0rc1. Is the |
|
@lixun910 : Yes, I use the python library to generate map reports automatically outside of jupyter entirely. If the official version of keplergl won't support that functionality anymore, I'll need to resort to maintaining my own fork of the older version. |
|
@karosc I see. Let me work on bring the save_to_html function to v0.4.0 |
|
@karosc Can you help to test if version 0.4.0rc2 works as you expected. Thanks! https://pypi.org/project/keplergl/0.4.0rc2/ |
|
@lixun910, I can take a look tomorrow and let you know if I have any issue. |
|
@lixun910 here is my feedback: Possible Bugs
Possible Improvements
I am happy to assist with a PR on any of these if you'd like. |
This pull request introduces a major refactor and modernization of the KeplerGL Jupyter Python package
uvfor Python,esbuildfor TypeScript)anywidgetNote: the function
save_to_html()will not be supported anymore: