Skip to content

Commit ee58e72

Browse files
davishmcclurgdavishmcclurg
committed
Squashed 'JSON-Schema-Test-Suite/' changes from 82a07749..48461fc3
48461fc3 Merge pull request #775 from jviotti/annotations-content-id 9f7dce32 Merge pull request #766 from json-schema-org/ether/bare-email a1e077d4 ensure we do not parse email strings permissively 72c670b0 Delete identifiers in Content annotation tests 675186ba Take into account `$id` for annotation schema location assertions 9ad349be Merge pull request #773 from jdesrosiers/annotation-propname-desc f164982c Merge pull request #771 from bavulapati/add-blaze-as-consumer d2bd2ad2 Update annotation test description for propertyNames a7a64707 Add [Blaze](https://github.com/sourcemeta/blaze) to the README 9f256c88 Change "expected" to an object with schema locations 7f996868 Update content tests to only apply to string instances 5338ecd1 Remove tests that assert a keyword doesn't emit annotations 738653b5 Make order of assertion properties consistent 8b5de3b9 Updates based on feedback from Juan 6270e399 Updates based on feedback from Karen 341df3ec Add automation to check that annotation tests are valid 16988c67 Add annotation tests bc919bdb Merge pull request #755 from V02460/unevaluated-additional-properties 83e866b4 Merge pull request #763 from michaelmior/propertynames-const c5a9703f Merge pull request #760 from OptimumCode/rfc3490-label-separator b4c09b65 Add tests for propertyNames with const/enum 4fa572d8 Move tests for rfc3490#3.1 into a separate test case ce9f68ca Add link to rfc and quote ad94cacc Add test cases for other valid label separators in IDN hostnames 39002ae7 Merge pull request #762 from OptimumCode/rfc-html-link c8780535 Correct section anchor for rfc URL template 5f2ca7d6 Modify rfc url template to use html version 9c5d99b6 Merge pull request #761 from OptimumCode/annotation-script-rfc-support 9563ce7b Correct rfc URL template - incorrect path pattern was used 961bfad0 Correct spec kind extraction from defined key. Continue on unkown URL kind e524505b Merge pull request #759 from sirosen/hostname-format-reject-single-dot 4a3efd18 Add negative tests for "." for hostname formats 4ba013d5 Merge pull request #747 from santhosh-tekuri/duration aa500e80 Merge pull request #749 from json-schema-org/gregsdennis/json-everything-update eb8ce976 Merge pull request #757 from ajevans99/main dcdae5c0 Merge pull request #758 from sirosen/hostname-format-check-empty-string db21d21b Merge branch 'main' into hostname-format-check-empty-string 3fd78f04 Merge pull request #1 from ajevans99/swift-json-schema 3cada3a9 Update README.md 5273e0d6 Make test descriptions more specific 43828fee Simplify adjacent additionalProperties test 347d6099 unevaluatedProperties: Remove type keywords 7dfbb1e9 Add test for unevaluatedProperties b1ee90f6 json-everything moved to an org c00a3f94 test: duration format must start with P 202d5625 test: hostname format check fails on empty string git-subtree-dir: JSON-Schema-Test-Suite git-subtree-split: 48461fc3568972801b40eaccc428a31bce338f6e
1 parent 32182e7 commit ee58e72

43 files changed

Lines changed: 2347 additions & 42 deletions

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎.github/workflows/annotation-tests.yml‎

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,21 @@
1+
name: Validate annotation tests
2+
3+
on:
4+
pull_request:
5+
paths:
6+
- "annotations/**"
7+
8+
jobs:
9+
annotate:
10+
runs-on: ubuntu-latest
11+
12+
steps:
13+
- uses: actions/checkout@v4
14+
15+
- name: Set up Deno
16+
uses: denoland/setup-deno@v2
17+
with:
18+
deno-version: "2.x"
19+
20+
- name: Validate annotation tests
21+
run: deno --node-modules-dir=auto --allow-read --no-prompt bin/annotation-tests.ts

‎README.md‎

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -227,6 +227,7 @@ This suite is being used by:
227227

228228
### C++
229229

230+
* [Blaze](https://github.com/sourcemeta/blaze)
230231
* [Modern C++ JSON schema validator](https://github.com/pboettch/json-schema-validator)
231232
* [Valijson](https://github.com/tristanpenman/valijson)
232233

@@ -293,7 +294,7 @@ Node-specific support is maintained in a [separate repository](https://github.co
293294

294295
### .NET
295296

296-
* [JsonSchema.Net](https://github.com/gregsdennis/json-everything)
297+
* [JsonSchema.Net](https://github.com/json-everything/json-everything)
297298
* [Newtonsoft.Json.Schema](https://github.com/JamesNK/Newtonsoft.Json.Schema)
298299

299300
### Perl
@@ -339,6 +340,7 @@ Node-specific support is maintained in a [separate repository](https://github.co
339340
### Swift
340341

341342
* [JSONSchema](https://github.com/kylef/JSONSchema.swift)
343+
* [swift-json-schema](https://github.com/ajevans99/swift-json-schema)
342344

343345
If you use it as well, please fork and send a pull request adding yourself to
344346
the list :).

‎annotations/README.md‎

Lines changed: 116 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,116 @@
1+
# Annotations Tests Suite
2+
3+
The Annotations Test Suite tests which annotations should appear (or not appear)
4+
on which values of an instance. These tests are agnostic of any output format.
5+
6+
## Supported Dialects
7+
8+
Although the annotation terminology of didn't appear in the spec until 2019-09,
9+
the concept is compatible with every version of JSON Schema. Test Cases in this
10+
Test Suite are designed to be compatible with as many releases of JSON Schema as
11+
possible. They do not include `$schema` or `$id`/`id` keywords so
12+
implementations can run the same Test Suite for each dialect they support.
13+
14+
Since this Test Suite can be used for a variety of dialects, there are a couple
15+
of options that can be used by Test Runners to filter out Test Cases that don't
16+
apply to the dialect under test.
17+
18+
## Test Case Components
19+
20+
### description
21+
22+
A short description of what behavior the Test Case is covering.
23+
24+
### compatibility
25+
26+
The `compatibility` option allows you to set which dialects the Test Case is
27+
compatible with. Test Runners can use this value to filter out Test Cases that
28+
don't apply the to dialect currently under test. The terminology for annotations
29+
didn't appear in the spec until 2019-09, but the concept is compatible with
30+
older releases as well. When setting `compatibility`, test authors should take
31+
into account dialects before 2019-09 for implementations that chose to support
32+
annotations for older dialects.
33+
34+
Dialects are indicated by the number corresponding to their release. Date-based
35+
releases use just the year. If this option isn't present, it means the Test Case
36+
is compatible with any dialect.
37+
38+
If this option is present with a number, the number indicates the minimum
39+
release the Test Case is compatible with. This example indicates that the Test
40+
Case is compatible with draft-07 and up.
41+
42+
**Example**: `"compatibility": "7"`
43+
44+
You can use a `<=` operator to indicate that the Test Case is compatible with
45+
releases less then or equal to the given release. This example indicates that
46+
the Test Case is compatible with 2019-09 and under.
47+
48+
**Example**: `"compatibility": "<=2019"`
49+
50+
You can use comma-separated values to indicate multiple constraints if needed.
51+
This example indicates that the Test Case is compatible with releases between
52+
draft-06 and 2019-09.
53+
54+
**Example**: `"compatibility": "6,<=2019"`
55+
56+
For convenience, you can use the `=` operator to indicate a Test Case is only
57+
compatible with a single release. This example indicates that the Test Case is
58+
compatible only with 2020-12.
59+
60+
**Example**: `"compatibility": "=2020"`
61+
62+
### schema
63+
64+
The schema that will serve as the subject for the tests. Whenever possible, this
65+
schema shouldn't include `$schema` or `id`/`$id` because Test Cases should be
66+
designed to work with as many releases as possible.
67+
68+
### externalSchemas
69+
70+
This allows you to define additional schemas that `schema` makes references to.
71+
The value is an object where the keys are retrieval URIs and values are schemas.
72+
Most external schemas aren't self identifying (using `id`/`$id`) and rely on the
73+
retrieval URI for identification. This is done to increase the number of
74+
dialects that the test is compatible with. Because `id` changed to `$id` in
75+
draft-06, if you use `$id`, the test becomes incompatible with draft-03/4 and in
76+
most cases, that's not necessary.
77+
78+
### tests
79+
80+
A collection of Tests to run to verify the Test Case.
81+
82+
## Test Components
83+
84+
### instance
85+
86+
The JSON instance to be annotated.
87+
88+
### assertions
89+
90+
A collection of assertions that must be true for the test to pass.
91+
92+
## Assertions Components
93+
94+
### location
95+
96+
The instance location.
97+
98+
### keyword
99+
100+
The annotating keyword.
101+
102+
### expected
103+
104+
A collection of `keyword` annotations expected on the instance at `location`.
105+
`expected` is an object where the keys are schema locations and the values are
106+
the annotation that schema location contributed for the given `keyword`.
107+
108+
There can be more than one expected annotation because multiple schema locations
109+
could contribute annotations for a single keyword.
110+
111+
An empty object is an assertion that the annotation must not appear at the
112+
`location` for the `keyword`.
113+
114+
As a convention for this Test Suite, the `expected` array should be sorted such
115+
that the most recently encountered value for an annotation given top-down
116+
evaluation of the schema comes before previously encountered values.

‎annotations/assertion.schema.json‎

Lines changed: 24 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,24 @@
1+
{
2+
"$schema": "http://json-schema.org/draft-07/schema#",
3+
4+
"type": "object",
5+
"properties": {
6+
"location": {
7+
"markdownDescription": "The instance location.",
8+
"type": "string",
9+
"format": "json-pointer"
10+
},
11+
"keyword": {
12+
"markdownDescription": "The annotation keyword.",
13+
"type": "string"
14+
},
15+
"expected": {
16+
"markdownDescription": "An object of schemaLocation/annotations pairs for `keyword` annotations expected on the instance at `location`.",
17+
"type": "object",
18+
"propertyNames": {
19+
"format": "uri"
20+
}
21+
}
22+
},
23+
"required": ["location", "keyword", "expected"]
24+
}

‎annotations/test-case.schema.json‎

Lines changed: 38 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,38 @@
1+
{
2+
"$schema": "http://json-schema.org/draft-07/schema#",
3+
4+
"type": "object",
5+
"properties": {
6+
"description": {
7+
"markdownDescription": "A short description of what behavior the Test Case is covering.",
8+
"type": "string"
9+
},
10+
"compatibility": {
11+
"markdownDescription": "Set which dialects the Test Case is compatible with. Examples:\n- `\"7\"` -- draft-07 and above\n- `\"<=2019\"` -- 2019-09 and previous\n- `\"6,<=2019\"` -- Between draft-06 and 2019-09\n- `\"=2020\"` -- 2020-12 only",
12+
"type": "string",
13+
"pattern": "^(<=|=)?([123467]|2019|2020)(,(<=|=)?([123467]|2019|2020))*$"
14+
},
15+
"schema": {
16+
"markdownDescription": "This schema shouldn't include `$schema` or `id`/`$id` unless necesary for the test because Test Cases should be designed to work with as many releases as possible.",
17+
"type": ["boolean", "object"]
18+
},
19+
"externalSchemas": {
20+
"markdownDescription": "The keys are retrieval URIs and values are schemas.",
21+
"type": "object",
22+
"patternProperties": {
23+
"": {
24+
"type": ["boolean", "object"]
25+
}
26+
},
27+
"propertyNames": {
28+
"format": "uri"
29+
}
30+
},
31+
"tests": {
32+
"markdownDescription": "A collection of Tests to run to verify the Test Case.",
33+
"type": "array",
34+
"items": { "$ref": "./test.schema.json" }
35+
}
36+
},
37+
"required": ["description", "schema", "tests"]
38+
}

‎annotations/test-suite.schema.json‎

Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,15 @@
1+
{
2+
"$schema": "http://json-schema.org/draft-07/schema#",
3+
4+
"type": "object",
5+
"properties": {
6+
"description": {
7+
"type": "string"
8+
},
9+
"suite": {
10+
"type": "array",
11+
"items": { "$ref": "./test-case.schema.json" }
12+
}
13+
},
14+
"required": ["description", "suite"]
15+
}

‎annotations/test.schema.json‎

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,16 @@
1+
{
2+
"$schema": "http://json-schema.org/draft-07/schema#",
3+
4+
"type": "object",
5+
"properties": {
6+
"instance": {
7+
"markdownDescription": "The JSON instance to be annotated."
8+
},
9+
"assertions": {
10+
"markdownDescription": "A collection of assertions that must be true for the test to pass.",
11+
"type": "array",
12+
"items": { "$ref": "./assertion.schema.json" }
13+
}
14+
},
15+
"required": ["instance", "assertions"]
16+
}

0 commit comments

Comments
 (0)