Commit 41aaf01
File attechment submodel (#193)
* Enhance Shell Retrieval with IdShort Filtering
* Refactor shell retrieval endpoints to support idShort filtering and enhance asset ID queries
* Added the implementation of get all submodels endpoint
* Added the level and extent as query param for specific submodel endpoint
* remove unnecessary cast to 'IEnumerable<ISubmodel>'.
* Potential fix for pull request finding
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
* Potential fix for pull request finding
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
* Potential fix for pull request finding
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
* Potential fix for pull request finding
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
* Potential fix for pull request finding
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
* Potential fix for pull request finding
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
* Potential fix for pull request finding
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
* Potential fix for pull request finding
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
* Potential fix for pull request finding
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
* Refactor and add unit tests for submodel service
Refactor the return statement in `SubmodelRepositoryService.cs` to use the C# 8.0 range operator for improved collection handling.
Add comprehensive unit tests in `GetAllSubmodelsControllerTests.cs` to cover various scenarios for the `GetAllSubmodelsAsync` method, including handling of query parameters, paging, and exception cases. Utilize `NSubstitute` for mocking and `Xunit` for test structuring.
* Remove unused ContactInformation template mapping from appsettings.development.json
* Enhance SubmodelRepositoryService to support concurrent operations with semaphore and add configuration for concurrent operations limit
* Add TemplateManagementConfig to SubmodelRepositoryService
Introduce `_templateManagementOptions` in `SubmodelRepositoryServiceTests` to configure `TemplateManagementConfig` with a `ServiceInstance` and `ConcurrentOperationsLimit`. Update the constructor of `SubmodelRepositoryService` to include the new parameter. Add necessary using directives for configuration and options handling.
* Add TemplateManagementConfig options to SubmodelRepositoryServiceTests
* Add ConcurrentOperationsLimit to service instances and validate configuration
* Sanitize URLs in logging for HTTP requests in TemplateProvider
* Refactor SubmodelRepositoryController and related tests to enhance GetAllSubmodelsAsync functionality and improve request handling
* Add GetAllSubmodelsControllerTests to validate GetAllSubmodelsAsync functionality
* Update SubmodelRepositoryControllerTests to use GetFilteredSubmodelTemplateAsync and adjust request paths
* Refactor SubmodelTemplateService and related tests to use GetFilteredSubmodelTemplateAsync for improved query handling
* Refactor various services to improve code clarity and consistency, including adjustments to semaphore release and data structure initialization.
* Add Get All Submodels - by idShort endpoint definition
* refactor the template provider tests
* Implement Get All Submodel Elements endpoint with associated request and response models, and add unit and integration tests
* Refactor query parameter handling in GetAllSubmodelElementsControllerTests for improved readability
* merge parent branch
* Implement GetFileAttachment functionality with associated tests and configuration
* Add file size validation and exception handling for file attachments
* Remove unused GetAllSubmodelElements method and SubmodelElementsDto mapping in SubmodelsMapperProfile
* Add file attachment handling and tracing to SubmodelRepositoryService
* Refactor file attachment handling in SubmodelRepositoryService to redirect URLs and remove unused code
* Add unit tests for GetFileAttachmentAsync in SubmodelRepositoryServiceTests
* Refactor file attachment handling to remove return types and simplify methods; delete unused configuration classes.
* Enhance file attachment handling by updating method signatures to return FileAttachmentResult; implement streaming timeout configuration; improve unit tests for file attachment retrieval.
* Refactor file attachment handling: update SubmodelRepositoryHandler and Service to support max file size; add unit tests for GetFileAttachment; adjust configuration settings.
* Implement file attachment streaming: add IFileAttachmentStreamProvider interface and FileAttachmentStreamProvider class; refactor SubmodelRepositoryService and tests to utilize new provider.
* Refactor file attachment handling: introduce TemplateServices and PluginServices for better dependency management; add FileSizeExceededException and InvalidFileUrlException for improved error handling; update appsettings.json to configure max file attachment size.
* Refactor file attachment handling: update exception types in GetFileAttachmentAsync method; improve null handling for submodel ID; enhance file size validation logic in SubmodelRepositoryService; adjust HTTP client configuration for plugins.
* Refactor file attachment handling: update documentation for file download endpoint; improve response descriptions; correct max file attachment size in configuration files.
* Updated API collections in both main and test plugin directories
* Add MaxLengthStream class for file size enforcement in streaming
* Add unit tests for FileAttachmentStreamProvider and MaxLengthStream
- Implemented tests for FileAttachmentStreamProvider to verify response headers and content stream reading.
- Added tests for MaxLengthStream to ensure it correctly enforces maximum byte limits and throws exceptions when exceeded.
- Refactored SubmodelRepositoryHandler to streamline file attachment retrieval logic.
- Updated FileSizeExceededException and InvalidFileUrlException to simplify constructors and messages.
- Introduced MaxLengthStream class to limit stream reading to a specified byte size.
- Modified SubmodelRepositoryService to enhance file URL validation and response handling.
- Updated JSON test data to remove Google Docs viewer URLs in favor of direct file URLs.
- Adjusted SQL load script to reflect changes in document URL handling.
* Enhance error logging in SubmodelRepositoryService: added logging for invalid file element types and empty file URLs.
* Refactor file attachment handling: introduce IFileContentProvider interface and FileContentProvider class; update SubmodelRepositoryService to use new provider; add unit tests for MaxLengthStream and FileContentProvider.
* Refactor SubmodelRepositoryControllerTests to use IFileContentProvider; update exception classes by removing unnecessary comments and whitespace.
* File attachment review findings and discussion
* Implement file attachment handling improvements: add LimitedFileStreamResult, FileAttachmentResult, and FileContentResponse classes; refactor SubmodelRepositoryService and related components; update exception handling for file size limits; remove obsolete exceptions.
* Refactor file attachment handling in tests: update SubmodelRepositoryControllerTests and SubmodelRepositoryHandlerTests to use new FileContentResponse and LimitedFileStreamResult; adjust assertions and exception handling for improved clarity and accuracy.
* Refactor file attachment handling: update GetFileAttachmentAsync method to return FileStreamResult instead of LimitedFileStreamResult; modify related unit tests for consistency and clarity.
* Refactor to use FileContentStreamResult for attachments
Renamed FileStreamResult to FileContentStreamResult and updated all references, including controller logic and tests, to ensure consistent handling of file attachments with the new result type.
* Refactor error handling in GetFileContentAsync method: replace EnsureSuccessStatusCode with explicit check for success status and throw InternalDataProcessingException on failure.
* Refactor file attachment handling: remove commented-out sections in tests and update exception handling to throw InvalidUserInputException with specific messages.
* updated module test
* refactor the exceptions handling in the submodel repository service
* Refactor GlobalExceptionHandler: streamline error handling by consolidating status code and message retrieval into GetErrorDetails method.
* Refactor GetErrorDetails method: streamline exception handling by consolidating return statements for clarity.
* Refactor FileContentResponse: simplify constructor by removing contentLength and contentType parameters; update related usages in FileContentProvider and tests.
* Removed unused variable
---------
Co-authored-by: Kevalkumar <kgi@mm-software.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Hardi Shah <hsh@mm-software.com>
Co-authored-by: Ankit Shah <ash@mm-software.com>1 parent c6db615 commit 41aaf01
44 files changed
Lines changed: 842 additions & 151 deletions
File tree
- example
- apiCollection/Submodel Repository/Submodel File Attachment
- postgres/seed
- source
- AAS.TwinEngine.DataEngine.ModuleTests/Api/Services/SubmodelRepository
- AAS.TwinEngine.DataEngine.UnitTests
- Api/SubmodelRepository
- Handler
- ApplicationLogic/Services/SubmodelRepository
- Infrastructure/Streaming
- AAS.TwinEngine.DataEngine
- Api
- SubmodelRepository
- Handler
- Requests
- Responses
- ApplicationLogic
- Exceptions
- Application
- Base
- Observability
- Services/SubmodelRepository
- Providers
- Infrastructure
- Http/Extensions
- Providers/FileContentProvider/Services
- ServiceConfiguration
- Config
- AAS.TwinEngine.Plugin.TestPlugin.PlaywrightTests/SubmodelRepository/TestData
- AAS.TwinEngine.Plugin.TestPlugin
- Data
- Example
- apiCollection/Submodel Repository/Submodel File Attachment
- plugin
- test/performance/databaseBulkDataLoader/sql
Some content is hidden
Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.
Lines changed: 21 additions & 0 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
Lines changed: 3 additions & 0 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
52 | 52 | | |
53 | 53 | | |
54 | 54 | | |
55 | | - | |
| 55 | + | |
56 | 56 | | |
57 | 57 | | |
58 | 58 | | |
| |||
98 | 98 | | |
99 | 99 | | |
100 | 100 | | |
101 | | - | |
| 101 | + | |
102 | 102 | | |
103 | 103 | | |
104 | 104 | | |
| |||
144 | 144 | | |
145 | 145 | | |
146 | 146 | | |
147 | | - | |
| 147 | + | |
148 | 148 | | |
149 | 149 | | |
150 | 150 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
87 | 87 | | |
88 | 88 | | |
89 | 89 | | |
90 | | - | |
| 90 | + | |
91 | 91 | | |
92 | 92 | | |
93 | 93 | | |
| |||
108 | 108 | | |
109 | 109 | | |
110 | 110 | | |
111 | | - | |
| 111 | + | |
112 | 112 | | |
113 | 113 | | |
114 | 114 | | |
| |||
129 | 129 | | |
130 | 130 | | |
131 | 131 | | |
132 | | - | |
| 132 | + | |
133 | 133 | | |
134 | 134 | | |
135 | 135 | | |
| |||
150 | 150 | | |
151 | 151 | | |
152 | 152 | | |
153 | | - | |
| 153 | + | |
154 | 154 | | |
155 | 155 | | |
156 | 156 | | |
| |||
171 | 171 | | |
172 | 172 | | |
173 | 173 | | |
174 | | - | |
| 174 | + | |
175 | 175 | | |
176 | 176 | | |
177 | 177 | | |
| |||
192 | 192 | | |
193 | 193 | | |
194 | 194 | | |
195 | | - | |
| 195 | + | |
196 | 196 | | |
197 | 197 | | |
198 | 198 | | |
| |||
213 | 213 | | |
214 | 214 | | |
215 | 215 | | |
216 | | - | |
| 216 | + | |
217 | 217 | | |
218 | 218 | | |
219 | 219 | | |
| |||
234 | 234 | | |
235 | 235 | | |
236 | 236 | | |
237 | | - | |
| 237 | + | |
238 | 238 | | |
239 | 239 | | |
240 | 240 | | |
| |||
255 | 255 | | |
256 | 256 | | |
257 | 257 | | |
258 | | - | |
| 258 | + | |
259 | 259 | | |
260 | 260 | | |
261 | 261 | | |
| |||
Lines changed: 72 additions & 0 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1 | 1 | | |
| 2 | + | |
2 | 3 | | |
3 | 4 | | |
4 | 5 | | |
| |||
7 | 8 | | |
8 | 9 | | |
9 | 10 | | |
| 11 | + | |
10 | 12 | | |
11 | 13 | | |
12 | 14 | | |
| |||
18 | 20 | | |
19 | 21 | | |
20 | 22 | | |
| 23 | + | |
21 | 24 | | |
22 | 25 | | |
23 | 26 | | |
| |||
27 | 30 | | |
28 | 31 | | |
29 | 32 | | |
| 33 | + | |
30 | 34 | | |
31 | 35 | | |
32 | 36 | | |
33 | 37 | | |
34 | 38 | | |
35 | 39 | | |
36 | 40 | | |
| 41 | + | |
37 | 42 | | |
38 | 43 | | |
39 | 44 | | |
40 | 45 | | |
41 | 46 | | |
42 | 47 | | |
43 | 48 | | |
| 49 | + | |
44 | 50 | | |
45 | 51 | | |
46 | 52 | | |
| |||
180 | 186 | | |
181 | 187 | | |
182 | 188 | | |
| 189 | + | |
| 190 | + | |
| 191 | + | |
| 192 | + | |
| 193 | + | |
| 194 | + | |
| 195 | + | |
| 196 | + | |
| 197 | + | |
| 198 | + | |
| 199 | + | |
| 200 | + | |
| 201 | + | |
| 202 | + | |
| 203 | + | |
| 204 | + | |
| 205 | + | |
| 206 | + | |
| 207 | + | |
| 208 | + | |
| 209 | + | |
| 210 | + | |
| 211 | + | |
| 212 | + | |
| 213 | + | |
| 214 | + | |
| 215 | + | |
| 216 | + | |
| 217 | + | |
| 218 | + | |
| 219 | + | |
| 220 | + | |
| 221 | + | |
| 222 | + | |
| 223 | + | |
| 224 | + | |
| 225 | + | |
| 226 | + | |
| 227 | + | |
| 228 | + | |
| 229 | + | |
| 230 | + | |
| 231 | + | |
| 232 | + | |
| 233 | + | |
| 234 | + | |
| 235 | + | |
| 236 | + | |
| 237 | + | |
| 238 | + | |
| 239 | + | |
| 240 | + | |
| 241 | + | |
| 242 | + | |
| 243 | + | |
| 244 | + | |
| 245 | + | |
| 246 | + | |
| 247 | + | |
| 248 | + | |
| 249 | + | |
| 250 | + | |
| 251 | + | |
| 252 | + | |
| 253 | + | |
| 254 | + | |
183 | 255 | | |
184 | 256 | | |
185 | 257 | | |
| |||
Lines changed: 8 additions & 0 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
144 | 144 | | |
145 | 145 | | |
146 | 146 | | |
| 147 | + | |
| 148 | + | |
| 149 | + | |
| 150 | + | |
| 151 | + | |
| 152 | + | |
| 153 | + | |
| 154 | + | |
147 | 155 | | |
148 | 156 | | |
149 | 157 | | |
| |||
Lines changed: 66 additions & 3 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1 | | - | |
| 1 | + | |
2 | 2 | | |
3 | 3 | | |
4 | 4 | | |
5 | 5 | | |
6 | 6 | | |
7 | 7 | | |
| 8 | + | |
8 | 9 | | |
9 | 10 | | |
10 | 11 | | |
| |||
15 | 16 | | |
16 | 17 | | |
17 | 18 | | |
| 19 | + | |
18 | 20 | | |
19 | 21 | | |
20 | 22 | | |
21 | 23 | | |
22 | 24 | | |
23 | 25 | | |
24 | | - | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
25 | 30 | | |
26 | 31 | | |
27 | 32 | | |
| |||
344 | 349 | | |
345 | 350 | | |
346 | 351 | | |
347 | | - | |
| 352 | + | |
348 | 353 | | |
349 | 354 | | |
350 | 355 | | |
| |||
452 | 457 | | |
453 | 458 | | |
454 | 459 | | |
| 460 | + | |
| 461 | + | |
| 462 | + | |
| 463 | + | |
| 464 | + | |
| 465 | + | |
| 466 | + | |
| 467 | + | |
| 468 | + | |
| 469 | + | |
| 470 | + | |
| 471 | + | |
| 472 | + | |
| 473 | + | |
| 474 | + | |
| 475 | + | |
| 476 | + | |
| 477 | + | |
| 478 | + | |
| 479 | + | |
| 480 | + | |
| 481 | + | |
| 482 | + | |
| 483 | + | |
| 484 | + | |
| 485 | + | |
| 486 | + | |
| 487 | + | |
| 488 | + | |
| 489 | + | |
| 490 | + | |
| 491 | + | |
| 492 | + | |
| 493 | + | |
| 494 | + | |
| 495 | + | |
| 496 | + | |
| 497 | + | |
| 498 | + | |
| 499 | + | |
| 500 | + | |
| 501 | + | |
| 502 | + | |
| 503 | + | |
| 504 | + | |
| 505 | + | |
| 506 | + | |
| 507 | + | |
| 508 | + | |
| 509 | + | |
| 510 | + | |
| 511 | + | |
| 512 | + | |
| 513 | + | |
| 514 | + | |
| 515 | + | |
| 516 | + | |
| 517 | + | |
455 | 518 | | |
0 commit comments