Skip to content

Commit bbec6dd

Browse files
authored
feat: import ArcGIS Pro projects from CIM JSON (#1637)
* feat: import ArcGIS Pro projects from CIM JSON Reads .aprx and .mapx files directly, so ArcGIS Pro and ArcPy are not needed to bring an existing map into GeoLibre. The importer restores the first 2D map's extent, local vector layers, nested groups, visibility, simple symbols, and field labels, and reports per-layer why anything was skipped instead of failing the whole import. * feat: import ArcGIS Pro rasters and vector tile services Extends the importer beyond vector layers: local GeoTIFF raster layers and portal-hosted vector tile layers now come across, and the Tauri asset guard accepts .aprx/.mapx alongside .qgs/.qgz so an imported raster path can be read. Also fixes a pre-existing defect the raster loop inherited from the QGIS importer. The raster control creates its store layer before it awaits the GeoTIFF header, so a rejection left the layer on the map while the importer reported it as an unsupported format -- the layer list and the warning dialog disagreed. Failed rasters are now rolled back, and the striped "not tiled" rejection is no longer treated as a failure at all, since that layer stays on purpose while the non-tiled handler offers to convert it to a COG. * Address Copilot, Claude, and CodeRabbit review feedback - Store each layer's and group's own visibility instead of the cascaded value. applyGroupEffects folds ancestor visibility in at render time, so pre-cascading discarded a nested item's real state and left it stuck hidden after its parent group was re-enabled. - Measure the MAPX size in bytes rather than String.length, which counts UTF-16 code units, and check binary input before decoding it to a string. - Cap an APRX's combined decompressed size, not just each member's, so many members just under the per-entry limit cannot add up past it. - Read CIM colors according to their subclass. CMYK, gray, and HSV colors store different channels on different scales, so interpreting every values array as RGB produced the wrong color; unknown subclasses keep the default. - Warn when a saved extent is dropped because its coordinate system cannot be converted, instead of silently opening at the generic world view. - Join a folder workspace with its dataset generically so file formats other than shapefile and delimited text can resolve. - Omit `reason` from resolveDataSource on success rather than returning "format" for a supported layer, and rename the shadowed `bytes` binding. * Address Claude review feedback - Keep a layer group whose only members are a raster or a service. Both are attached after the project loads, so pruning the group made the later moveLayerToGroup a silent no-op and stranded them at the top level. - Report a raster on a UNC workspace as "network-path" rather than "format", matching what the vector path already reports for the same root cause. - Stop unbounded group recursion. A corrupt or crafted project can reference a group that already contains it, and children resolved from the archive share object identity, so the import recursed until the stack overflowed. Group nesting now carries an ancestor set plus a depth cap, and a project that trips either gets a warning instead of crashing the import. * Address Claude and CodeRabbit review feedback - Read layer opacity from CIM `transparency` (0-100, inverted) rather than a non-existent `opacity` field, so partially transparent ArcGIS layers no longer import as fully opaque. Applied to rasters, services, groups, and feature layers, and rounded so float noise does not reach the project file. - Give addArcGISLayer a `zoomTo` opt-out and use it when importing services. It fit the map to every layer's bounds unconditionally, throwing away the extent loadProject had just restored from the project file. - Place a raster in its imported group on the recoverable "not tiled" path. That layer stays on the map pending COG conversion, so returning early left it outside the group the project put it in. - Re-prune layer groups in the browser build after local-file layers are dropped, counting services (which still load) and not rasters (which never do), and keeping ancestors of surviving groups. - Rename the `allow_raster_asset` parameter and its caller option to `import_project_path`/`importProjectPath` now that it accepts ArcGIS Pro projects too, and update the Rust doc comment to match. - Assert an asymmetric gray level so the color test pins the scale direction.
1 parent 3517a6d commit bbec6dd

17 files changed

Lines changed: 1669 additions & 41 deletions

File tree

apps/geolibre-desktop/src-tauri/src/lib.rs

Lines changed: 11 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -460,29 +460,33 @@ fn read_local_file(path: String) -> Result<tauri::ipc::Response, String> {
460460
/// Add one GeoTIFF to the asset-protocol scope. The filesystem and asset scopes
461461
/// are separate in Tauri; dialogs and native drops grant the former, but
462462
/// maplibre-gl-raster fetches through the latter for range reads. A GeoTIFF
463-
/// referenced by an imported QGIS project is also accepted when that project
464-
/// file itself was explicitly selected by the user.
463+
/// referenced by an imported project -- QGIS (`.qgs`/`.qgz`) or ArcGIS Pro
464+
/// (`.aprx`/`.mapx`) -- is also accepted when that project file itself was
465+
/// explicitly selected by the user.
465466
#[tauri::command]
466467
fn allow_raster_asset(
467468
app: tauri::AppHandle,
468469
path: String,
469-
qgis_project_path: Option<String>,
470+
import_project_path: Option<String>,
470471
) -> Result<(), String> {
471472
let lower = path.to_ascii_lowercase();
472473
if !is_safe_absolute_path(&path) || !(lower.ends_with(".tif") || lower.ends_with(".tiff")) {
473474
return Err(format!(
474475
"Refusing to expose \"{path}\": not an absolute GeoTIFF path"
475476
));
476477
}
477-
let selected_qgis_project = qgis_project_path.is_some_and(|project_path| {
478+
let selected_import_project = import_project_path.is_some_and(|project_path| {
478479
let lower = project_path.to_ascii_lowercase();
479480
is_safe_absolute_path(&project_path)
480-
&& (lower.ends_with(".qgs") || lower.ends_with(".qgz"))
481+
&& (lower.ends_with(".qgs")
482+
|| lower.ends_with(".qgz")
483+
|| lower.ends_with(".aprx")
484+
|| lower.ends_with(".mapx"))
481485
&& app.fs_scope().is_allowed(&project_path)
482486
});
483-
if !app.fs_scope().is_allowed(&path) && !selected_qgis_project {
487+
if !app.fs_scope().is_allowed(&path) && !selected_import_project {
484488
return Err(format!(
485-
"Refusing to expose \"{path}\": neither the file nor its QGIS project was selected by the user"
489+
"Refusing to expose \"{path}\": neither the file nor its imported project was selected by the user"
486490
));
487491
}
488492
app.asset_protocol_scope()

apps/geolibre-desktop/src/components/layout/TopToolbar.tsx

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1551,6 +1551,7 @@ export function TopToolbar({
15511551
onOpenFromUrl={() => projectFiles.setProjectUrlDialogOpen(true)}
15521552
onOpenGallery={() => setGalleryDialogOpen(true)}
15531553
onImportQgisProject={() => void projectFiles.handleImportQgisProject()}
1554+
onImportArcgisProject={() => void projectFiles.handleImportArcgisProject()}
15541555
onOpenRecent={(path) => {
15551556
void projectFiles.handleOpenRecent(path).then((error) => {
15561557
if (error) projectFiles.setActionError(error);

apps/geolibre-desktop/src/components/layout/toolbar/ProjectFileDialogs.tsx

Lines changed: 32 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -76,6 +76,38 @@ export function ProjectFileDialogs({ projectFiles }: ProjectFileDialogsProps) {
7676
</form>
7777
</DialogContent>
7878
</Dialog>
79+
<Dialog
80+
open={projectFiles.arcgisImportWarnings !== null}
81+
onOpenChange={(open: boolean) => {
82+
if (!open) projectFiles.setArcgisImportWarnings(null);
83+
}}
84+
>
85+
<DialogContent className="max-w-lg">
86+
<DialogHeader>
87+
<DialogTitle>{t("toolbar.item.arcgisImportComplete")}</DialogTitle>
88+
<DialogDescription>
89+
{t("toolbar.item.arcgisImportWarnings", {
90+
count: projectFiles.arcgisImportWarnings?.length ?? 0,
91+
})}
92+
</DialogDescription>
93+
</DialogHeader>
94+
<ul className="max-h-64 space-y-2 overflow-y-auto text-sm">
95+
{projectFiles.arcgisImportWarnings?.map((warning, index) => (
96+
<li key={`${warning.layerName}-${index}`}>
97+
<strong>{warning.layerName}:</strong>{" "}
98+
{t(`toolbar.item.arcgisImportReason.${warning.reason}`, {
99+
layerType: warning.layerType || t("toolbar.item.arcgisUnknownLayerType"),
100+
})}
101+
</li>
102+
))}
103+
</ul>
104+
<div className="flex justify-end">
105+
<Button onClick={() => projectFiles.setArcgisImportWarnings(null)}>
106+
{t("common.ok")}
107+
</Button>
108+
</div>
109+
</DialogContent>
110+
</Dialog>
79111
<Dialog
80112
open={projectFiles.actionError !== null}
81113
onOpenChange={(open: boolean) => {

apps/geolibre-desktop/src/components/layout/toolbar/ProjectMenu.tsx

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -46,6 +46,7 @@ interface ProjectMenuProps {
4646
onOpenFromUrl: () => void;
4747
onOpenGallery: () => void;
4848
onImportQgisProject: () => void;
49+
onImportArcgisProject: () => void;
4950
onOpenRecent: (path: string) => void;
5051
onOpenHistory: () => void;
5152
onSave: () => void;
@@ -68,6 +69,7 @@ export function ProjectMenu({
6869
onOpenFromUrl,
6970
onOpenGallery,
7071
onImportQgisProject,
72+
onImportArcgisProject,
7173
onOpenRecent,
7274
onOpenHistory,
7375
onSave,
@@ -222,6 +224,10 @@ export function ProjectMenu({
222224
<FileInput className="me-2 h-3.5 w-3.5" />
223225
{t("toolbar.item.importQgisProjectEllipsis")}
224226
</DropdownMenuItem>
227+
<DropdownMenuItem onSelect={onImportArcgisProject}>
228+
<FileInput className="me-2 h-3.5 w-3.5" />
229+
{t("toolbar.item.importArcgisProjectEllipsis")}
230+
</DropdownMenuItem>
225231
</DropdownMenuSubContent>
226232
</DropdownMenuSub>
227233
)}

apps/geolibre-desktop/src/hooks/useProjectFileActions.ts

Lines changed: 198 additions & 20 deletions
Original file line numberDiff line numberDiff line change
@@ -6,7 +6,12 @@ import {
66
useAppStore,
77
type GeoLibreLayer,
88
} from "@geolibre/core";
9-
import { addRasterToMap, materializeEmbeddableVectorLayers } from "@geolibre/plugins";
9+
import {
10+
addArcGISLayer,
11+
addRasterToMap,
12+
isRecoverableNonTiledRasterError,
13+
materializeEmbeddableVectorLayers,
14+
} from "@geolibre/plugins";
1015
import type { FeatureCollection } from "geojson";
1116
import { type FormEvent, useRef, useState } from "react";
1217
import { useTranslation } from "react-i18next";
@@ -18,6 +23,7 @@ import {
1823
isHttpUrl,
1924
isTauri,
2025
loadDroppedRasterPaths,
26+
openArcgisProjectFile,
2127
openProjectFile,
2228
openQgisProjectFile,
2329
openRecentProjectFile,
@@ -41,6 +47,7 @@ import {
4147
materializeQgisRemoteLayers,
4248
type QgisProjectImportWarning,
4349
} from "../lib/qgis-project-import";
50+
import { importArcgisProject, type ArcgisProjectImportWarning } from "../lib/arcgis-project-import";
4451
import type { MapControllerRef } from "../components/layout/toolbar/constants";
4552

4653
/** A pending "strip env vars before saving?" prompt. */
@@ -134,6 +141,58 @@ function importedProjectMapReady(
134141
})();
135142
}
136143

144+
/**
145+
* Adds one raster from an imported QGIS/ArcGIS Pro project, cleaning up after
146+
* itself when the raster cannot be loaded.
147+
*
148+
* `addRasterToMap` resolves only once the GeoTIFF header has been read, but the
149+
* control creates the store layer earlier (its `rasteradd` fires before that
150+
* await). A rejection therefore leaves the layer behind, so importers that
151+
* simply caught the error listed a raster as unsupported while it was still
152+
* sitting in the layer list -- see the NLCD case in GeoLibre#1637. Rolling the
153+
* layer back keeps the warning dialog and the layer list telling the same story.
154+
*
155+
* The striped "not tiled" rejection is deliberately not a failure: that layer
156+
* stays on the map while the registered non-tiled handler offers to convert it
157+
* to a COG, so it is neither rolled back nor reported.
158+
*
159+
* @param app - The app API for the live map.
160+
* @param source - Raster source resolved from the project's layer path.
161+
* @param options - Passed through to {@link addRasterToMap}.
162+
* @param groupId - Imported layer group to move the raster into, if any.
163+
* @throws The original load error, after the partial layer has been removed.
164+
*/
165+
async function addImportedProjectRaster(
166+
app: ReturnType<typeof createAppAPI>,
167+
source: Parameters<typeof addRasterToMap>[1],
168+
options: Parameters<typeof addRasterToMap>[2],
169+
groupId: string | undefined,
170+
): Promise<void> {
171+
const before = new Set(useAppStore.getState().layers.map((layer) => layer.id));
172+
try {
173+
const layerId = await addRasterToMap(app, source, options);
174+
if (groupId) useAppStore.getState().moveLayerToGroup(layerId, groupId);
175+
} catch (error) {
176+
if (isRecoverableNonTiledRasterError(error)) {
177+
// The rejection carried no layer id, but the control already created the
178+
// store layer and is keeping it while the COG conversion is offered, so
179+
// it still has to be placed in its imported group -- otherwise a raster
180+
// that converts successfully ends up at the top level.
181+
if (groupId) {
182+
const { layers, moveLayerToGroup } = useAppStore.getState();
183+
const created = layers.find((layer) => !before.has(layer.id));
184+
if (created) moveLayerToGroup(created.id, groupId);
185+
}
186+
return;
187+
}
188+
const { layers, removeLayer } = useAppStore.getState();
189+
for (const layer of layers) {
190+
if (!before.has(layer.id)) removeLayer(layer.id);
191+
}
192+
throw error;
193+
}
194+
}
195+
137196
/**
138197
* Bundles every project file action (open from file/URL/recent, save, save as)
139198
* along with the related dialog state (Open-from-URL, env-var strip prompt, and
@@ -154,6 +213,9 @@ export function useProjectFileActions(mapControllerRef: MapControllerRef) {
154213
const [qgisImportWarnings, setQgisImportWarnings] = useState<QgisProjectImportWarning[] | null>(
155214
null,
156215
);
216+
const [arcgisImportWarnings, setArcgisImportWarnings] = useState<
217+
ArcgisProjectImportWarning[] | null
218+
>(null);
157219
const [projectUrlDialogOpen, setProjectUrlDialogOpen] = useState(false);
158220
const [projectUrl, setProjectUrl] = useState("");
159221
const [projectUrlError, setProjectUrlError] = useState<string | null>(null);
@@ -235,28 +297,30 @@ export function useProjectFileActions(mapControllerRef: MapControllerRef) {
235297
for (const raster of imported.rasters) {
236298
try {
237299
const [loaded] = await loadDroppedRasterPaths([raster.sourcePath], {
238-
qgisProjectPath: result.path,
300+
importProjectPath: result.path,
239301
});
240302
if (!loaded) throw new Error("Unsupported raster path");
241-
const rasterLayerId = await addRasterToMap(app, loaded.source, {
242-
name: raster.name,
243-
localPath: raster.sourcePath,
244-
// The Tauri/WebKitGTK WASM backend can stall when its first
245-
// source is created immediately after a project style load.
246-
// GPU renders this local COG directly and preserves the imported
247-
// QGIS ramp, so use the verified backend for project imports.
248-
defaults: { engine: "maplibre-gl-raster" },
249-
state: {
250-
...raster.state,
251-
visible: raster.visible,
252-
opacity: raster.opacity,
303+
await addImportedProjectRaster(
304+
app,
305+
loaded.source,
306+
{
307+
name: raster.name,
308+
localPath: raster.sourcePath,
309+
// The Tauri/WebKitGTK WASM backend can stall when its first
310+
// source is created immediately after a project style load.
311+
// GPU renders this local COG directly and preserves the imported
312+
// QGIS ramp, so use the verified backend for project imports.
313+
defaults: { engine: "maplibre-gl-raster" },
314+
state: {
315+
...raster.state,
316+
visible: raster.visible,
317+
opacity: raster.opacity,
318+
},
319+
beforeId: raster.beforeId,
320+
zoomTo: false,
253321
},
254-
beforeId: raster.beforeId,
255-
zoomTo: false,
256-
});
257-
if (raster.groupId) {
258-
useAppStore.getState().moveLayerToGroup(rasterLayerId, raster.groupId);
259-
}
322+
raster.groupId,
323+
);
260324
} catch (error) {
261325
console.error(`Failed to import QGIS raster "${raster.name}"`, error);
262326
imported.warnings.push({
@@ -276,6 +340,117 @@ export function useProjectFileActions(mapControllerRef: MapControllerRef) {
276340
}
277341
};
278342

343+
const handleImportArcgisProject = async () => {
344+
const result = await openArcgisProjectFile();
345+
if (!result) return;
346+
try {
347+
const imported = importArcgisProject(result.data, result.path);
348+
if (!isTauri()) {
349+
const unavailableLayerIds = new Set<string>();
350+
for (const layer of imported.project.layers) {
351+
if (layer.sourcePath && !isHttpUrl(layer.sourcePath)) {
352+
unavailableLayerIds.add(layer.id);
353+
imported.warnings.push({ layerName: layer.name, reason: "browser-local-file" });
354+
}
355+
}
356+
imported.project.layers = imported.project.layers.filter(
357+
(layer) => !unavailableLayerIds.has(layer.id),
358+
);
359+
for (const raster of imported.rasters) {
360+
imported.warnings.push({ layerName: raster.name, reason: "browser-local-file" });
361+
}
362+
// Rasters never load in the browser build, so drop them before the
363+
// group prune below rather than letting them keep a group alive that
364+
// will stay empty. Services still load, so they still count.
365+
imported.rasters = [];
366+
// Re-prune the groups: dropping the local-file layers can empty a group
367+
// that the importer kept, and an empty group left behind shows up as a
368+
// dangling entry in the layer panel.
369+
const usedGroupIds = new Set<string>([
370+
...imported.project.layers.flatMap((layer) => (layer.groupId ? [layer.groupId] : [])),
371+
...imported.services.flatMap((service) => (service.groupId ? [service.groupId] : [])),
372+
]);
373+
// A parent group stays as long as a surviving group still names it, so
374+
// walk up the chain before filtering.
375+
const groupById = new Map(
376+
(imported.project.layerGroups ?? []).map((group) => [group.id, group]),
377+
);
378+
for (const id of [...usedGroupIds]) {
379+
let parentId = groupById.get(id)?.parentId;
380+
while (parentId && !usedGroupIds.has(parentId)) {
381+
usedGroupIds.add(parentId);
382+
parentId = groupById.get(parentId)?.parentId;
383+
}
384+
}
385+
imported.project.layerGroups = imported.project.layerGroups?.filter((group) =>
386+
usedGroupIds.has(group.id),
387+
);
388+
}
389+
const mapReady = importedProjectMapReady(
390+
mapControllerRef,
391+
useAppStore.getState().basemapStyleUrl !== imported.project.basemapStyleUrl,
392+
);
393+
loadProject(imported.project, null);
394+
await mapReady;
395+
const app = createAppAPI(mapControllerRef);
396+
if (isTauri()) {
397+
for (const raster of imported.rasters) {
398+
try {
399+
const [loaded] = await loadDroppedRasterPaths([raster.sourcePath], {
400+
importProjectPath: result.path,
401+
});
402+
if (!loaded) throw new Error("Unsupported raster path");
403+
await addImportedProjectRaster(
404+
app,
405+
loaded.source,
406+
{
407+
name: raster.name,
408+
localPath: raster.sourcePath,
409+
defaults: { engine: "maplibre-gl-raster" },
410+
state: { visible: raster.visible, opacity: raster.opacity },
411+
zoomTo: false,
412+
},
413+
raster.groupId,
414+
);
415+
} catch (error) {
416+
console.error(`Failed to import ArcGIS raster "${raster.name}"`, error);
417+
imported.warnings.push({ layerName: raster.name, reason: "format" });
418+
}
419+
}
420+
}
421+
for (const service of imported.services) {
422+
try {
423+
const serviceLayerId = await addArcGISLayer(app, {
424+
itemId: service.itemId,
425+
layerType: "vector-tile",
426+
name: service.name,
427+
sourceType: "portal-item",
428+
// The project's saved extent was applied by loadProject above, and
429+
// this runs after it. Without the opt-out, each imported service
430+
// would fit the map to its own bounds and throw that extent away.
431+
zoomTo: false,
432+
});
433+
if (service.groupId) {
434+
useAppStore.getState().moveLayerToGroup(serviceLayerId, service.groupId);
435+
}
436+
if (!service.visible) {
437+
useAppStore.getState().setLayerVisibility(serviceLayerId, false);
438+
}
439+
} catch (error) {
440+
console.error(`Failed to import ArcGIS service "${service.name}"`, error);
441+
imported.warnings.push({ layerName: service.name, reason: "service" });
442+
}
443+
}
444+
useAppStore.setState({ isDirty: true });
445+
setArcgisImportWarnings(imported.warnings.length > 0 ? imported.warnings : null);
446+
} catch (error) {
447+
console.error("Failed to import ArcGIS project", error);
448+
setActionError(
449+
error instanceof Error ? error.message : t("toolbar.error.couldNotImportArcgisProject"),
450+
);
451+
}
452+
};
453+
279454
const handleOpenFromUrl = async (event: FormEvent<HTMLFormElement>) => {
280455
event.preventDefault();
281456
const normalizedUrl = normalizeProjectUrl(projectUrl);
@@ -830,6 +1005,8 @@ export function useProjectFileActions(mapControllerRef: MapControllerRef) {
8301005
setActionError,
8311006
qgisImportWarnings,
8321007
setQgisImportWarnings,
1008+
arcgisImportWarnings,
1009+
setArcgisImportWarnings,
8331010
projectUrlDialogOpen,
8341011
setProjectUrlDialogOpen,
8351012
handleProjectUrlDialogOpenChange,
@@ -853,6 +1030,7 @@ export function useProjectFileActions(mapControllerRef: MapControllerRef) {
8531030
cancelSaveNamePrompt,
8541031
handleOpenFromFile,
8551032
handleImportQgisProject,
1033+
handleImportArcgisProject,
8561034
handleOpenFromUrl,
8571035
openProjectFromShareUrl,
8581036
handleOpenRecent,

0 commit comments

Comments
 (0)