Commit 9fee28c
Refactor field-selector filtering for PackageRevision list operation (kptdev#273)
* Issue kptdev#677 - refactor package revision filtering for list flow
- first cut at making it simpler to add new selectable fields
- reworked the 2 levels of PackageRevision filtering to use
Kubernetes SelectionPredicates, so we can more easily feed
in a set of field names and mappings to their counterparts
in the data structures
https://github.com/nephio-project/nephio/issues/677
* refactor ListPackageRevisionFilter parse method
* rework selectable-field refactoring to pull everything together in fieldselector.go
- results in a performance hit for list flow
- work in progress
https://github.com/nephio-project/nephio/issues/677
* rework selectable-field refactoring to pull everything together in fieldselector.go
- results in a performance hit for list flow
- work in progress
https://github.com/nephio-project/nephio/issues/677
* fix for manually filtering by namespace with no existing filter fields
* fix filtering on spec.repository
* Updating Github Action E2E Tests & fixing TestPodEvaluator & TestEditPackageRevision Flakyness (kptdev#271)
* updating github action porch e2e test to use latest versions & procedure
* changed kind context to 'porch-test'
* testing potential delete timing issue
* removed time.sleep with awaiting deletion with 20 sec timeout
* added generous timeout value for tests
* removed await pod deletion and just switching to different namespace to avoid clash in the first place
* added accurate comment on timeout
* Issue kptdev#677 - final refactor to package revision filtering
- selectable fields now all bound to elements of internal
repository.PackageRevision type
- this allows all filtering to be done at level of the
ListPackageRevisionFilter's matching
- upper-level packageRevisionFilter now only extends
ListPackageRevisionFilter upward into the registry/porch
package
- list of selectable field names added to API-level PackageRevision
type in v1alpha1 types.go
- new fieldselector.go file stores mappings of field names
to repository.PackageRevision object values
https://github.com/nephio-project/nephio/issues/677
* take now-unnecessary deletionPropoesdCacheMutex back out
- also create dedicated function type for PackageRevision
field mappings
* increase fieldselector.go coverage
* Implement a Database cache in Porch (kptdev#234)
* Implement a database cache in Porch.
* Add postgres and db cache config etc
* Add unit tests to SQL files
* Added unit tests
* More unit tests on SQL
* Package SQL unit tests done
Signed-off-by: liamfallon <liam.fallon@est.tech>
* Added unit tests for PackageRevision SQL
* Unit tests for resources sql table
Signed-off-by: liamfallon <liam.fallon@est.tech>
* Added sync and refactored around sync
* Update to fix external to cache repo key linking
Signed-off-by: liamfallon <liam.fallon@est.tech>
* Updated keying to fix some small bugs
* Sync and query filtering added
* Update tests and fix embedded postgres
Signed-off-by: liamfallon <liam.fallon@est.tech>
* Remove time check from unit tests
Signed-off-by: liamfallon <liam.fallon@est.tech>
* Added unit tests for DB Repository
Signed-off-by: liamfallon <liam.fallon@est.tech>
* More unit tests
Signed-off-by: liamfallon <liam.fallon@est.tech>
* More unit tests
Signed-off-by: liamfallon <liam.fallon@est.tech>
* Unit tests
Signed-off-by: liamfallon <liam.fallon@est.tech>
* Address review comments
* Remove reference to nonexistantant fake cache
Signed-off-by: liamfallon <liam.fallon@est.tech>
* Restore fake cache deleted in error
Signed-off-by: liamfallon <liam.fallon@est.tech>
* More unit test
* More unit test
Signed-off-by: liamfallon <liam.fallon@est.tech>
* Updated push of PR to external repo to be external repo independent
Signed-off-by: liamfallon <liam.fallon@est.tech>
* Added more unit test
Signed-off-by: liamfallon <liam.fallon@est.tech>
* Fixed copyright notices
Signed-off-by: liamfallon <liam.fallon@est.tech>
* update query and rebase
Signed-off-by: liamfallon <liam.fallon@est.tech>
* Added context to error messages
Signed-off-by: liamfallon <liam.fallon@est.tech>
* Add mock for repository Key() function
Signed-off-by: liamfallon <liam.fallon@est.tech>
* Add check on git version into sync
Signed-off-by: liamfallon <liam.fallon@est.tech>
* Add owner references to API metadata for PackageRevision and PackageRevisionResource
Signed-off-by: liamfallon <liam.fallon@est.tech>
* Update metadata on SetMeta() function
Signed-off-by: liamfallon <liam.fallon@est.tech>
* Tasks and deployment in status now working
Signed-off-by: liamfallon <liam.fallon@est.tech>
* Sort query results from DB and handle latest revision
Signed-off-by: liamfallon <liam.fallon@est.tech>
* Update latest PR on deletes
Signed-off-by: liamfallon <liam.fallon@est.tech>
* Updated code to cope with Latest revision calculated in DB
Signed-off-by: liamfallon <liam.fallon@est.tech>
* Updated to address review comments
Signed-off-by: liamfallon <liam.fallon@est.tech>
* Fix typo in query
Signed-off-by: liamfallon <liam.fallon@est.tech>
* Fix code smells fpound by sonar
Signed-off-by: liamfallon <liam.fallon@est.tech>
* Update tests for text canges
Signed-off-by: liamfallon <liam.fallon@est.tech>
* Increase cde coverage
Signed-off-by: liamfallon <liam.fallon@est.tech>
* Updated with responses to review comments
Signed-off-by: liamfallon <liam.fallon@est.tech>
* Updated with responses to review comments
Signed-off-by: liamfallon <liam.fallon@est.tech>
* Remove load balancer type from function runner deployment
Signed-off-by: liamfallon <liam.fallon@est.tech>
---------
Signed-off-by: liamfallon <liam.fallon@est.tech>
Co-authored-by: Fiachra Corcoran <fiachra.corcoran@est.tech>
* Issue kptdev#677 - refactor package revision filtering for list flow
- first cut at making it simpler to add new selectable fields
- reworked the 2 levels of PackageRevision filtering to use
Kubernetes SelectionPredicates, so we can more easily feed
in a set of field names and mappings to their counterparts
in the data structures
https://github.com/nephio-project/nephio/issues/677
* refactor ListPackageRevisionFilter parse method
* rework selectable-field refactoring to pull everything together in fieldselector.go
- results in a performance hit for list flow
- work in progress
https://github.com/nephio-project/nephio/issues/677
* Issue kptdev#677 - final refactor to package revision filtering
- selectable fields now all bound to elements of internal
repository.PackageRevision type
- this allows all filtering to be done at level of the
ListPackageRevisionFilter's matching
- upper-level packageRevisionFilter now only extends
ListPackageRevisionFilter upward into the registry/porch
package
- list of selectable field names added to API-level PackageRevision
type in v1alpha1 types.go
- new fieldselector.go file stores mappings of field names
to repository.PackageRevision object values
https://github.com/nephio-project/nephio/issues/677
* increase fieldselector.go coverage
* resolve rebase conflicts from DB cache introduction
- and fold in changes to field-selector mechanism
* test fix
- and attempt to keep 'make test' from messing up global Git config
* fix tests again and try doing Git config change on local level
* fix Git config addition in go-test Makefile
- lock the config addition behind a check so it doesn't interfere
with developers' local Git configuration
* bump coverage: fieldselector.go, repository.go
* bump coverage: repository/fieldselector.go
* fix test
* Issue kptdev#677 - fix incidental break in label-selector filtering
- this involves moving label selecting to ListPackageRevisionFilter's
new SelectionPredicate mechanism
- completing bundling of label/field selector handling into
ListPackageRevisionFilter
- and allows removal of the separate labels.Selector parameter from
the list-package-revisions and watch-package-revisions flows
- also add a new e2e test to cover label-selector filtering
https://github.com/nephio-project/nephio/issues/677
* formatting nitpicks in launch.json to retrigger E2E tests
* comment nitpick in Makefile to retrigger E2E tests
* Issue kptdev#677 - fix filtering on special Porch-managed latest-revision label
- with refactoring to SelectionPredicate filtering, list flow
only adds this label to a PackageRevision after the final check
for a match is performed
- added bit to filter handling (ParseAttrFunc method) to add the label
in specially if PackageRevision being parsed is the latest revision
- also special-case coverage in e2e test
* comment nitpick in launch.json to retrigger E2E tests
* comment nitpick to retrigger e2e tests
* minor e2e test refactor in attempt to de-flake TestProposeDeleteAndUndo against DB cache
* compile/test fixes after rebase
* Issue kptdev#677 - cut out SelectionPredicate-based filtering
- revert everything to PackageRevisionKey filtering
- removing redundant re-filter above DB Cache's SQL
filtering, for maximum efficiency
- results in no support for not-equals (!=) field
selectors
- so remove them from scope of testing
https://github.com/nephio-project/nephio/issues/677
* coverage improvements
- also move field-selector/filter-field mappings into porch package's
fieldselector.go
- since it's now associated purely with that level
* coverage improvements, minor relocation refactors
* fix merge detail that slipped through and broke compile
* lint fix
* review comments
- reworked PackageRevisionSelectableFields to an array of constants
* move PackageRevisionLifecycle util methods as per review comment
* delete unnecessary .bak file
---------
Signed-off-by: liamfallon <liam.fallon@est.tech>
Co-authored-by: Catalin Stratulat <159934629+Catalin-Stratulat-Ericsson@users.noreply.github.com>
Co-authored-by: Liam Fallon <35595825+liamfallon@users.noreply.github.com>
Co-authored-by: Fiachra Corcoran <fiachra.corcoran@est.tech>1 parent 9f5af4f commit 9fee28c
31 files changed
Lines changed: 1317 additions & 279 deletions
File tree
- .vscode
- api/porch/v1alpha1
- pkg
- cache
- crcache
- dbcache
- cli/commands/rpkg/upgrade
- externalrepo
- fake
- git
- registry/porch
- repository
- test
- e2e
- mockery/mocks/porch/pkg/engine
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
44 | 44 | | |
45 | 45 | | |
46 | 46 | | |
| 47 | + | |
This file was deleted.
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
5 | 5 | | |
6 | 6 | | |
7 | 7 | | |
8 | | - | |
| 8 | + | |
9 | 9 | | |
10 | | - | |
11 | | - | |
| 10 | + | |
| 11 | + | |
12 | 12 | | |
13 | 13 | | |
14 | 14 | | |
15 | | - | |
16 | | - | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
17 | 19 | | |
18 | 20 | | |
19 | 21 | | |
| |||
77 | 79 | | |
78 | 80 | | |
79 | 81 | | |
80 | | - | |
| 82 | + | |
81 | 83 | | |
82 | | - | |
| 84 | + | |
83 | 85 | | |
84 | 86 | | |
85 | 87 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
282 | 282 | | |
283 | 283 | | |
284 | 284 | | |
285 | | - | |
| 285 | + | |
286 | 286 | | |
287 | 287 | | |
288 | 288 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
31 | 31 | | |
32 | 32 | | |
33 | 33 | | |
34 | | - | |
35 | | - | |
36 | | - | |
37 | | - | |
38 | 34 | | |
39 | 35 | | |
40 | 36 | | |
41 | 37 | | |
42 | 38 | | |
43 | 39 | | |
44 | 40 | | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
| 54 | + | |
| 55 | + | |
| 56 | + | |
| 57 | + | |
| 58 | + | |
| 59 | + | |
| 60 | + | |
| 61 | + | |
| 62 | + | |
45 | 63 | | |
46 | 64 | | |
47 | 65 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
16 | 16 | | |
17 | 17 | | |
18 | 18 | | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
19 | 23 | | |
20 | 24 | | |
21 | 25 | | |
22 | 26 | | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
23 | 39 | | |
24 | 40 | | |
25 | 41 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
101 | 101 | | |
102 | 102 | | |
103 | 103 | | |
| 104 | + | |
| 105 | + | |
| 106 | + | |
| 107 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
84 | 84 | | |
85 | 85 | | |
86 | 86 | | |
| 87 | + | |
87 | 88 | | |
88 | 89 | | |
89 | 90 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
70 | 70 | | |
71 | 71 | | |
72 | 72 | | |
73 | | - | |
74 | | - | |
| 73 | + | |
| 74 | + | |
75 | 75 | | |
76 | 76 | | |
77 | 77 | | |
78 | 78 | | |
79 | 79 | | |
80 | 80 | | |
81 | 81 | | |
82 | | - | |
| 82 | + | |
83 | 83 | | |
84 | 84 | | |
85 | 85 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
305 | 305 | | |
306 | 306 | | |
307 | 307 | | |
| 308 | + | |
| 309 | + | |
| 310 | + | |
| 311 | + | |
308 | 312 | | |
309 | 313 | | |
310 | 314 | | |
| |||
0 commit comments