Skip to content

Commit 364e715

Browse files
MrHDOLEKMrHDOLEK
authored
Merge branch '1.x' into #1390 
2 parents 6d769f2 + 48cb4bc commit 364e715

33 files changed

Lines changed: 774 additions & 435 deletions

File tree

‎CHANGELOG.md‎

Lines changed: 15 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,8 @@
1-
## [Unreleased] - 2026-07-11
1+
## [Unreleased] - 2026-07-12
22

33
### Added
4+
- [#2526](https://github.com/flow-php/flow/pull/2526) - **flow-php/etl - externalSortBucketSize config controlling rows per external sort bucket (default 10000)** - [@norberttech](https://github.com/norberttech)
5+
- [#2525](https://github.com/flow-php/flow/pull/2525) - **flow-php/symfony-telemetry-bundle - RouteNamePathMap optional cache warmer building a [route name => path] map at cache:warmup, rebuilt lazily when missing** - [@norberttech](https://github.com/norberttech)
46
- [#2523](https://github.com/flow-php/flow/pull/2523) - **flow-php/etl - Floe binary row serialization format with writer, reader, merger, extractor, loader and DSL.** - [@norberttech](https://github.com/norberttech)
57
- [#2523](https://github.com/flow-php/flow/pull/2523) - **flow-php/flow-php-ext - new Rust-based PHP extension for native Floe encoding/decoding.** - [@norberttech](https://github.com/norberttech)
68
- [#2523](https://github.com/flow-php/flow/pull/2523) - **flow-php/etl - Cache::read() for streaming cached Rows in batches.** - [@norberttech](https://github.com/norberttech)
@@ -28,6 +30,15 @@
2830
- [#2501](https://github.com/flow-php/flow/pull/2501) - **flow-php/phpunit-telemetry-bridge - test.suite.run.status attribute and telemetry.sdk.version resource attribute.** - [@norberttech](https://github.com/norberttech)
2931

3032
### Changed
33+
- [#2528](https://github.com/flow-php/flow/pull/2528) - **Reduce complexity of XML reading & writing** - [@stloyd](https://github.com/stloyd)
34+
- [#2527](https://github.com/flow-php/flow/pull/2527) - **Change `EntryFactory::createAs()` to use `match()`** - [@stloyd](https://github.com/stloyd)
35+
- [#2526](https://github.com/flow-php/flow/pull/2526) - **flow-php/etl - sorting algorithm is an explicit choice via sort config, EXTERNAL_SORT is the default** - [@norberttech](https://github.com/norberttech)
36+
- [#2526](https://github.com/flow-php/flow/pull/2526) - **flow-php/etl - external sort spills fewer, larger buckets and streams the final merge instead of writing it back to cache** - [@norberttech](https://github.com/norberttech)
37+
- [#2526](https://github.com/flow-php/flow/pull/2526) - **flow-php/etl - Rows::sortAscending/sortDescending resolve sort values once per row instead of twice per comparison** - [@norberttech](https://github.com/norberttech)
38+
- [#2526](https://github.com/flow-php/flow/pull/2526) - **flow-php/etl - RowsMinHeap compares sort values extracted once at insert** - [@norberttech](https://github.com/norberttech)
39+
- [#2526](https://github.com/flow-php/flow/pull/2526) - **flow-php/etl - MemorySort buffers rows in a plain array instead of per-batch Rows::merge** - [@norberttech](https://github.com/norberttech)
40+
- [#2526](https://github.com/flow-php/flow/pull/2526) - **flow-php/etl - Floe SchemaTracker caches container type fingerprints per Type instance** - [@norberttech](https://github.com/norberttech)
41+
- [#2525](https://github.com/flow-php/flow/pull/2525) - **flow-php/symfony-telemetry-bundle - HttpKernelSpanSubscriber constructor takes ?RouteNamePathMap instead of ?RouterInterface** - [@norberttech](https://github.com/norberttech)
3142
- [#2524](https://github.com/flow-php/flow/pull/2524) - **Replaced switch statements with match expressions across core ETL, parquet lib and http/parquet/seal adapters (no behavior change, outputs byte-identical)** - [@MrHDOLEK](https://github.com/MrHDOLEK)
3243
- [#2523](https://github.com/flow-php/flow/pull/2523) - **flow-php/etl - Cache stores only Rows; cache indexes are stored as Rows.** - [@norberttech](https://github.com/norberttech)
3344
- [#2523](https://github.com/flow-php/flow/pull/2523) - **flow-php/etl - cache implementations and external sort buckets serialize with Floe instead of PHP serializer.** - [@norberttech](https://github.com/norberttech)
@@ -58,6 +69,7 @@ instead of inside the per-row loop** - [@MrHDOLEK](https://github.com/MrHDOLEK)
5869
- [#2501](https://github.com/flow-php/flow/pull/2501) - **flow-php/psr18-telemetry-bridge - Client span name is the HTTP method only and server.port is always set.** - [@norberttech](https://github.com/norberttech)
5970

6071
### Fixed
72+
- [#2525](https://github.com/flow-php/flow/pull/2525) - **flow-php/symfony-telemetry-bundle - route_naming: path no longer calls Router::getRouteCollection() (full route collection rebuild) on every traced request** - [@norberttech](https://github.com/norberttech)
6173
- [#2523](https://github.com/flow-php/flow/pull/2523) - **flow-php/filesystem - MemoryStream appends at end of stream after ranged reads.** - [@norberttech](https://github.com/norberttech)
6274
- [#2523](https://github.com/flow-php/flow/pull/2523) - **flow-php/types - EnumType normalization accepts UnitEnum/BackedEnum interfaces.** - [@norberttech](https://github.com/norberttech)
6375
- [#2504](https://github.com/flow-php/flow/pull/2504) - **flow-php/etl - SchemaValidationException message no longer lists FROM_NULL columns accepted by the validator** - [@norberttech](https://github.com/norberttech)
@@ -73,6 +85,8 @@ instead of inside the per-row loop** - [@MrHDOLEK](https://github.com/MrHDOLEK)
7385
- [#2501](https://github.com/flow-php/flow/pull/2501) - **flow-php/symfony-telemetry-bundle - Suppress messenger worker poll/idle noise and tame orphan cache/DBAL spans.** - [@norberttech](https://github.com/norberttech)
7486

7587
### Removed
88+
- [#2526](https://github.com/flow-php/flow/pull/2526) - **flow-php/etl - SortAlgorithms::MEMORY_FALLBACK_EXTERNAL_SORT and SortAlgorithms::SQLITE_SORT** - [@norberttech](https://github.com/norberttech)
89+
- [#2526](https://github.com/flow-php/flow/pull/2526) - **flow-php/etl - sortMemoryLimit config, FLOW_SORT_MAX_MEMORY env var and OutOfMemoryException** - [@norberttech](https://github.com/norberttech)
7690
- [#2503](https://github.com/flow-php/flow/pull/2503) - **flow-php/symfony-telemetry-bundle - runtime_mode config and the WorkerModeDetector/RuntimeModeResolver runtime classes.** - [@norberttech](https://github.com/norberttech)
7791
- [#2501](https://github.com/flow-php/flow/pull/2501) - **flow-php/telemetry - Remove ResettableContextStorage and MemoryContextStorage::reset().** - [@norberttech](https://github.com/norberttech)
7892
- [#2501](https://github.com/flow-php/flow/pull/2501) - **flow-php/symfony-telemetry-bundle - Remove messenger link config, worker-cycle span, metrics_duration_unit and dbal.log_sql config.** - [@norberttech](https://github.com/norberttech)

‎documentation/components/bridges/symfony-telemetry-bundle.md‎

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1151,7 +1151,9 @@ The request (SERVER) span follows the OpenTelemetry HTTP semantic conventions fo
11511151
controlled by `route_naming`:
11521152

11531153
- `path` (default) — the route **path template**, e.g. `GET /orders/{id}` (low cardinality, semconv value
1154-
for `http.route`); resolved from the router.
1154+
for `http.route`); resolved from a `[route name => path]` map built once per deployment by an optional
1155+
cache warmer (`cache:warmup`) and rebuilt lazily when missing, so the router is never queried on the
1156+
request path.
11551157
- `name` — the Symfony **route name**, e.g. `GET order_show`.
11561158
- Sub-requests (`render(controller(...))`) have no route, so they are named after the **controller**
11571159
(`GET App\Controller\NavigationController::top`); a request that matches no route at all uses the method

‎documentation/upgrading.md‎

Lines changed: 36 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -294,6 +294,42 @@ implementation is `yield $this->get($key);`.
294294

295295
Convert `DateTime`/`DateTimeImmutable` subclasses to `DateTime`/`DateTimeImmutable` before caching or serializing.
296296

297+
### 21) `flow-php/symfony-telemetry-bundle` - `HttpKernelSpanSubscriber` takes a
298+
299+
`RouteNamePathMap` instead of the router
300+
301+
| Before | After |
302+
|-------------------------------------------------------|---------------------------------------------------------------------|
303+
| `new HttpKernelSpanSubscriber(…, router: $router, …)` | `new HttpKernelSpanSubscriber(…, routePaths: $routeNamePathMap, …)` |
304+
| `?RouterInterface $router = null` | `?RouteNamePathMap $routePaths = null` |
305+
306+
Applies only to direct construction; services wired by the bundle need no change.
307+
308+
### 22) `flow-php/etl` - sort algorithm is an explicit choice; external sort is the default
309+
310+
| Before | After |
311+
|-----------------------------------------------------------------|----------------------------------------------------------------|
312+
| `SortAlgorithms::MEMORY_FALLBACK_EXTERNAL_SORT` (default) | removed; default is `SortAlgorithms::EXTERNAL_SORT` |
313+
| `SortAlgorithms::SQLITE_SORT` | removed |
314+
| `SortAlgorithms::useMemory()` | removed |
315+
| `ConfigBuilder::sortMemoryLimit(Unit $unit)` | removed |
316+
| `SortConfigBuilder::sortMemoryLimit(Unit $unit)` | removed |
317+
| `SortConfig::$memoryLimit` | removed |
318+
| `SortConfig::SORT_MAX_MEMORY_ENV` / `FLOW_SORT_MAX_MEMORY` env | removed |
319+
| `new MemorySort(Unit $maximumMemory)` | `new MemorySort()` |
320+
| `MemorySort` throwing `Flow\ETL\Exception\OutOfMemoryException` | removed |
321+
| `new ExternalSort($cache, $bucketsCount)` | `new ExternalSort($cache, $bucketsCount, $bucketSize)` |
322+
| — | `ConfigBuilder::externalSortBucketSize(int)` (default `10000`) |
323+
324+
To sort in memory, opt in explicitly:
325+
326+
```php
327+
$config = config_builder();
328+
$config->sort->algorithm(SortAlgorithms::MEMORY_SORT);
329+
330+
data_frame($config)->read(...)->sortBy(ref('id'))->run();
331+
```
332+
297333
---
298334

299335
## Upgrading from 0.40.x to 0.41.x

‎src/adapter/etl-adapter-xml/src/Flow/ETL/Adapter/XML/Loader/XMLLoader.php‎

Lines changed: 5 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -22,11 +22,7 @@
2222
use Throwable;
2323

2424
use function array_key_exists;
25-
use function array_keys;
26-
use function array_map;
27-
use function array_values;
2825
use function count;
29-
use function implode;
3026

3127
final class XMLLoader implements Closure, FileLoader, Loader
3228
{
@@ -194,13 +190,12 @@ public function write(Rows $nextRows, array $partitions, FlowContext $context, R
194190
$this->writes[$stream->path()->path()] = 0;
195191
}
196192

197-
$xmlAttributes = implode(' ', array_map(
198-
static fn(string $key, string $value) => $key . '="' . $value . '"',
199-
array_keys($this->xmlAttributes),
200-
array_values($this->xmlAttributes),
201-
));
193+
$xmlAttributes = '';
194+
foreach ($this->xmlAttributes as $key => $value) {
195+
$xmlAttributes .= $key . '="' . $value . '" ';
196+
}
202197

203-
$stream->append('<?xml ' . $xmlAttributes . "?>\n<" . $this->rootElementName . ">\n");
198+
$stream->append('<?xml ' . trim($xmlAttributes) . "?>\n<" . $this->rootElementName . ">\n");
204199
} else {
205200
$stream = $streams->writeTo($this->path, $partitions);
206201
}

‎src/adapter/etl-adapter-xml/src/Flow/ETL/Adapter/XML/XMLParserExtractor.php‎

Lines changed: 12 additions & 42 deletions
Original file line numberDiff line numberDiff line change
@@ -60,7 +60,7 @@ final class XMLParserExtractor implements Extractor, FileExtractor, LimitableExt
6060
private string $xmlNodePath = '';
6161

6262
/**
63-
* In order to iterate only over <element> nodes use `$loader->withXMLNodePath('root/elements/element')`.
63+
* To iterate only over <element> nodes, use `$loader->withXMLNodePath('root/elements/element')`.
6464
*
6565
* <root>
6666
* <elements>
@@ -124,51 +124,18 @@ public function extract(FlowContext $context): Generator
124124
));
125125
}
126126

127-
if (count($this->elements)) {
128-
foreach ($this->elements as $element) {
129-
if ($shouldPutInputIntoRows) {
130-
$rowData = [
131-
'node' => $this->createDOMNode($element),
132-
'_input_file_uri' => $uri,
133-
];
134-
} else {
135-
$rowData = ['node' => $this->createDOMNode($element)];
136-
}
137-
138-
$signal = yield array_to_rows(
139-
$rowData,
140-
$context->entryFactory(),
141-
$stream->path()->partitions(),
142-
$this->schema,
143-
);
144-
145-
$this->incrementReturnedRows();
146-
147-
if ($signal === Signal::STOP || $this->reachedLimit()) {
148-
$context->streams()->closeStreams($this->path);
149-
$this->freeParser();
150-
151-
return;
152-
}
153-
}
154-
$this->elements = [];
155-
}
156-
}
157-
158-
xml_parse($this->parser(), '', true);
159-
160-
if (count($this->elements)) {
161127
foreach ($this->elements as $element) {
128+
$rowData = ['node' => $this->createDOMNode($element)];
162129
if ($shouldPutInputIntoRows) {
163-
$rowData = [
164-
'node' => $this->createDOMNode($element),
165-
'_input_file_uri' => $uri,
166-
];
167-
} else {
168-
$rowData = ['node' => $this->createDOMNode($element)];
130+
$rowData['_input_file_uri'] = $uri;
169131
}
170132

171-
$signal = yield array_to_rows([$rowData], $context->entryFactory(), $stream->path()->partitions());
133+
$signal = yield array_to_rows(
134+
$rowData,
135+
$context->entryFactory(),
136+
$stream->path()->partitions(),
137+
$this->schema,
138+
);
172139

173140
$this->incrementReturnedRows();
174141

@@ -179,9 +146,12 @@ public function extract(FlowContext $context): Generator
179146
return;
180147
}
181148
}
149+
182150
$this->elements = [];
183151
}
184152

153+
xml_parse($this->parser(), '', true);
154+
185155
$this->freeParser();
186156
}
187157
}

‎src/bridge/symfony/telemetry-bundle/src/Flow/Bridge/Symfony/TelemetryBundle/Instrumentation/HttpKernel/HttpKernelSpanSubscriber.php‎

Lines changed: 3 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -30,7 +30,6 @@
3030
use Symfony\Component\HttpKernel\Event\ResponseEvent;
3131
use Symfony\Component\HttpKernel\Event\TerminateEvent;
3232
use Symfony\Component\HttpKernel\KernelEvents;
33-
use Symfony\Component\Routing\RouterInterface;
3433

3534
use function array_key_exists;
3635
use function array_map;
@@ -59,7 +58,7 @@ public function __construct(
5958
private Propagator $propagator,
6059
private bool $contextPropagation = true,
6160
private bool $contextPropagationQuery = false,
62-
private ?RouterInterface $router = null,
61+
private ?RouteNamePathMap $routePaths = null,
6362
private RouteNaming $routeNaming = RouteNaming::Path,
6463
) {
6564
$this->excludePathRules = array_map(
@@ -98,11 +97,11 @@ public function onController(ControllerEvent $event): void
9897

9998
private function routeValue(string $routeName): string
10099
{
101-
if ($this->routeNaming !== RouteNaming::Path || $this->router === null) {
100+
if ($this->routeNaming !== RouteNaming::Path) {
102101
return $routeName;
103102
}
104103

105-
return $this->router->getRouteCollection()->get($routeName)?->getPath() ?? $routeName;
104+
return $this->routePaths?->pathFor($routeName) ?? $routeName;
106105
}
107106

108107
public function onException(ExceptionEvent $event): void

‎src/bridge/symfony/telemetry-bundle/src/Flow/Bridge/Symfony/TelemetryBundle/Instrumentation/HttpKernel/RouteNamePathMap.php‎

Lines changed: 114 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,114 @@
1+
<?php
2+
3+
declare(strict_types=1);
4+
5+
namespace Flow\Bridge\Symfony\TelemetryBundle\Instrumentation\HttpKernel;
6+
7+
use Symfony\Component\Config\ConfigCache;
8+
use Symfony\Component\HttpKernel\CacheWarmer\CacheWarmerInterface;
9+
use Symfony\Component\Routing\RouterInterface;
10+
use Throwable;
11+
12+
use function dirname;
13+
use function is_dir;
14+
use function is_writable;
15+
use function var_export;
16+
17+
/**
18+
* Deployment-static [route name => path template] map so request spans can carry the OTEL http.route
19+
* path template without touching the router at runtime — Router::getRouteCollection() bypasses the
20+
* compiled matcher and rebuilds the full route collection, which Symfony explicitly warns is too slow
21+
* for the request path.
22+
*/
23+
final class RouteNamePathMap implements CacheWarmerInterface
24+
{
25+
private ConfigCache $cache;
26+
27+
/** @var null|array<string, string> */
28+
private ?array $paths = null;
29+
30+
private bool $unavailable = false;
31+
32+
public function __construct(
33+
private readonly ?RouterInterface $router,
34+
string $directory,
35+
bool $debug,
36+
) {
37+
$this->cache = new ConfigCache($directory . '/flow_telemetry_route_paths.php', $debug);
38+
}
39+
40+
public function isOptional(): bool
41+
{
42+
return true;
43+
}
44+
45+
public function pathFor(string $routeName): ?string
46+
{
47+
if ($this->paths === null && !$this->load()) {
48+
return null;
49+
}
50+
51+
return $this->paths[$routeName] ?? null;
52+
}
53+
54+
public function warmUp(string $cacheDir, ?string $buildDir = null): array
55+
{
56+
if ($this->router === null) {
57+
return [];
58+
}
59+
60+
$collection = $this->router->getRouteCollection();
61+
$paths = [];
62+
63+
foreach ($collection->all() as $name => $route) {
64+
$paths[$name] = $route->getPath();
65+
}
66+
67+
$this->cache->write('<?php return ' . var_export($paths, true) . ';', $collection->getResources());
68+
$this->paths = $paths;
69+
70+
return [$this->cache->getPath()];
71+
}
72+
73+
private function load(): bool
74+
{
75+
if ($this->unavailable) {
76+
return false;
77+
}
78+
79+
try {
80+
if (!$this->cache->isFresh()) {
81+
$directory = dirname($this->cache->getPath());
82+
83+
// Guard against the unwritable branch before touching the router: under PHP-FPM the
84+
// failure memoization below lives one request only, so a throwing warmUp() would
85+
// rebuild the route collection on every request — the exact cost this map avoids.
86+
if (is_dir($directory) ? !is_writable($directory) : !is_writable(dirname($directory))) {
87+
$this->unavailable = true;
88+
89+
return false;
90+
}
91+
92+
$this->warmUp('');
93+
}
94+
} catch (Throwable) {
95+
$this->unavailable = true;
96+
97+
return false;
98+
}
99+
100+
if ($this->paths === null) {
101+
if (!$this->cache->isFresh()) {
102+
$this->unavailable = true;
103+
104+
return false;
105+
}
106+
107+
/** @var array<string, string> $paths */
108+
$paths = require $this->cache->getPath();
109+
$this->paths = $paths;
110+
}
111+
112+
return true;
113+
}
114+
}

0 commit comments

Comments
 (0)