- python
- custom tile args parsing
- src
- use unstable formatting options
cargo +nightly fmt --all -- --unstable-features --config group_imports=StdExternalCrate,imports_granularity=Module,reorder_imports=true
- use unstable formatting options
- Use
typed-headerfeature ofaxum-extrafor tilejson(s) - Upgrade
rusqliteto v0.38.0metadata_duplicate_key_valuesnow returns(String, String, u32)count instead ofusize- use
i64where appropriate for sqlite integer values instead ofusize/u64
- use
prekinstead ofpre-commit - pin github actions
- format docs/markdown files
- update to pyo3-v0.29
- rust-2024 edition
- criterion-v6 though I am not sure I still need to be benchmarking that dumb experiment
- all
allowlints moved toexpect - docs website
- Updated several dependencies
- several minor versions
json-patchv4
- Clippy fixes
- useless conversions
- const functions
- pyo3 v0.24.x
- frozen py structs
- Moved
edgesfromutilestoutiles-core
- minor updates and re-jiggering
utiles webpifyandutiles-oxipngcommands now show correct size difference if/when the size of a tiles-db is larger than it was previously- fix axum 8 new routing paths
utiles-pyo3children now accept kwarg boolzorder:zorderreturns quadkey ordering:top-left,top-right,bottom-left,bottom-right- For some reason (I do not know why)
mercantileand@mapbox/tilebeltboth return children in the order oftop-left,top-right,bottom-right,bottom-leftwhich I call 'd-ordering'
cli- format string
{projwin}and{projwin_web}changed to return" "(space) separated coords
- format string
- fix axum 8 new routing paths
- NO UNWRAPPING! there is no unwrapping in utiles!
- updated
thiserrorto v2 - remove
gil-refspyo3 feature from python lib pyo3v0.23.x- supermercado compatible
edgesandburncli commands added as well a s rust lib functions (obviously) shapesandfeaturefunctions for tile produce polygons that now DO follow the right-hand-rule; exterior rings are clockwisesimplify/mergefunction(s) optimized for faster merging and WAY less memory usage- Parent methods now returns
Noneifz<=0 - New command(s):
agg-hashcommand that computes theagg-tiles-hashof a tiles-db as standardized by the martin/maplibre team (this supports more hash-types;xxh3appears to be the fastest and what utiles will likely default to if notxxh64)commandslist all available commands (including hidden/dev/unimplemented commands)enumeratelist xyz tiles in db(s) similar totippecanoe-enumerate; for tippecanoe compatibility use--tippecanoe/-tsqlite/dbsub-command group withvac/analyzecommands and will likely contain future pure sqlite util(e)s... these could totally be shell scripts, but they're nice to have on das-windowsheader/headcommand that prints the json of a sqlite db header (which has come in handy for weird dbs that use old code to write out sqlite dbs (yes I have seen this))vac/vacuumcommand that vacuums a sqlite db (optionally into a new db)analyzecommand that analyzes a sqlite db (basically the same as doingsqlite3 database.sqlite "PRAGMA analyze;")
copyandtouch- Now supports
flat/norm(normalized)/hash(flat-with-hash) formats as standardized by the martin/maplibre people. Should also work with non-martin-conforming mbtiles schemas (appears to for me)
- Now supports
- Dev/hidden commands:
webpifycommand that converts all non-webp raster-tiles to webp (lossless only due to image-crate not supporting lossy encoding...)oxipngcommand that optimizes png(s) in mbtiles db(s) usingoxipngcrate
- figured out how to make async tile-stream(s)
- Removed even more
unwrapusages - lager/logging/tracing is reloadable from python; using the new fancy
std::sync::LazyLock(yay) - lint/copy overhaul
- Added
--page-sizeto vacuum command - Using
json-patchfor metadata updates - Allow setting metadata value(s) from file if no value is provided (
-/--) for stdin - Figured out how to put the caller-site (eg
pyo3in the cli help so you (likely me) can tell which utiles you are calling) - python:
- Added
TileFmtsstring formatter object Lagersingleton that can toggle tracing format and level (WIP)
- Added
- Fix calling
utiles.ut_climultiple times causing tracing-subscriber crash
- Upgrade pyo3 to
v0.22.0-- had to add signatures to all fns with optional args/kwargs - Update python dev deps
- Added
{bbox},{projwin},{bbox_web}and{projwin_web}format tokens to tile-formatter (those projwins are handy for gdaling)
- Fixed backpressure issue when
unpyramidingdirectory to mbtiles; the loading of tiles was happening too fast and could cause memory issues on large tile-pyramids... (this was previously not an issue b/c I would run those jobs on my work machine which had 512gb or ram, but that machine died... RIP titus) - Write out
metadata.jsonwhenpyramid-ingmbtiles to directory if the metadata of the mbtiles does not contain duplicate keys (which it should not) - Limit jobs/concurrency when
pyramid-ingmbtiles to directory to 4 (if not specified by--jobs/-joption) to prevent nuking machines
- Moved metadata structs and tools from
utiles-coretoutiles utiles-python- Refactoring and reorganizing of code
- Fixed comparison of LngLat obj
- Using rust lib for
python -m utilesfile size formatting
clippy::pedantic- Many changes on the road to
clippy::pedantic utiles-coreis almost fully pedanticutilesis becoming pedanticutiles-pythonis not very pedantic yet
- Many changes on the road to
- Testing:
- More tests added (mostly testing w/ python)
- Added test mbtiles file
osm-standard.0z4.mbtiles
- fmt-str command and option
--fmtaddedtilescommand; allows string formatting of json-tiles:
Format json-tiles format-string
fmt-tokens:
`{json_arr}`/`{json}` -> [x, y, z]
`{json_obj}`/`{obj}` -> {x: x, y: y, z: z}
`{quadkey}`/`{qk}` -> quadkey string
`{pmtileid}`/`{pmid}` -> pmtile-id
`{x}` -> x tile coord
`{y}` -> y tile coord
`{z}` -> z/zoom level
`{-y}`/`{yup}` -> y tile coord flipped/tms
`{zxy}` -> z/x/y
Example:
> echo "[486, 332, 10]" | utiles fmtstr
[486, 332, 10]
> echo "[486, 332, 10]" | utiles fmtstr --fmt "{x},{y},{z}"
486,332,10
> echo "[486, 332, 10]" | utiles fmt --fmt "SELECT * FROM tiles WHERE zoom_level = {z} AND tile_column = {x} AND tile_row = {y};"
SELECT * FROM tiles WHERE zoom_level = 10 AND tile_column = 486 AND tile_row = 332;
- Fixed problem with python tile
__richcmp__not handling invalid tiles and non-tile-like objs
- Updated to pyo3
v0.21.0 - Cli help messages cleaned up
- General spring cleaning!
- Hid the
utiles tilejsoncli aliastrader-joes
- Minor bug fixes
- Expanded utiles cli with several more commands
- Converted cli to rust as an exercise in learning clap
- Moved old click cli to
utiles._legacy.cli - Added tilejson/tj command to rust cli to write out tilejson files for mbtiles
- Added meta command to rust cli to write out json of metadata table for mbtiles
- Drop python 3.7 (was good knowing you)
- Update pyo3 to 0.20.0
- Added rasterio/rio entry points ('utiles' and 'ut' alias bc why type
rio utilesoverrio ut)
- Added
__len__to TilesGenerator for pbars