Skip to content

Commit 0eadd69

Browse files
DavisVaughanDavisVaughan
authored
Update tree-sitter-r to reject .[digit] as an identifier (#496)
* Update tree-sitter-r to reject `.[digit]` as an `identifier` * CHANGELOG bullet
1 parent 5fdec74 commit 0eadd69

9 files changed

Lines changed: 181 additions & 146 deletions

File tree

‎CHANGELOG.md‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -4,6 +4,8 @@
44

55
- Hexadecimal constants with decimals are no longer parsed as two separate values (#495).
66

7+
- `.{digit}` (like `.1`) is now correctly parsed as a double rather than as an identifier. This means that things like `function(.1 = 1) {}`, where `.1` is expected to be an identifier but not a double, now correctly fails to parse (#496).
8+
79
# Development version
810

911
# 0.9.0

‎Cargo.lock‎

Lines changed: 1 addition & 1 deletion
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

‎Cargo.toml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -74,7 +74,7 @@ tower-lsp = { branch = "bugfix/patches", git = "https://github.com/lionel-/tower
7474
tracing = { version = "0.1.40", default-features = false, features = ["std"] }
7575
tracing-subscriber = "0.3.19"
7676
tree-sitter = "0.24.7"
77-
tree-sitter-r = { git = "https://github.com/r-lib/tree-sitter-r", rev = "ceee38d8260856c1ace9012348916a3d2965e5e5" }
77+
tree-sitter-r = { git = "https://github.com/r-lib/tree-sitter-r", rev = "37a6b1b5c8cc2426fd6ec16ca818cc98597d755b" }
7878
url = "2.5.3"
7979
uuid = { version = "1.11.0", features = ["v4"] }
8080
workspace = { path = "./crates/workspace" }

‎crates/air_r_parser/tests/snapshots/ok/value/complex_value.R‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2,3 +2,6 @@
22
2.5i
33
1e6i
44
0x123Fi
5+
6+
# https://github.com/r-lib/tree-sitter-r/pull/198
7+
.1i

‎crates/air_r_parser/tests/snapshots/ok/value/complex_value.R.snap‎

Lines changed: 12 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -10,6 +10,9 @@ expression: snapshot
1010
1e6i
1111
0x123Fi
1212
13+
# https://github.com/r-lib/tree-sitter-r/pull/198
14+
.1i
15+
1316
```
1417

1518

@@ -31,17 +34,20 @@ RRoot {
3134
RComplexValue {
3235
value_token: R_COMPLEX_LITERAL@12..20 "0x123Fi" [Newline("\n")] [],
3336
},
37+
RComplexValue {
38+
value_token: R_COMPLEX_LITERAL@20..75 ".1i" [Newline("\n"), Newline("\n"), Comments("# https://github.com/ ..."), Newline("\n")] [],
39+
},
3440
],
35-
eof_token: EOF@20..21 "" [Newline("\n")] [],
41+
eof_token: EOF@75..76 "" [Newline("\n")] [],
3642
}
3743
```
3844

3945
## CST
4046

4147
```
42-
0: R_ROOT@0..21
48+
0: R_ROOT@0..76
4349
0: (empty)
44-
1: R_EXPRESSION_LIST@0..20
50+
1: R_EXPRESSION_LIST@0..75
4551
0: R_COMPLEX_VALUE@0..2
4652
0: R_COMPLEX_LITERAL@0..2 "1i" [] []
4753
1: R_COMPLEX_VALUE@2..7
@@ -50,6 +56,8 @@ RRoot {
5056
0: R_COMPLEX_LITERAL@7..12 "1e6i" [Newline("\n")] []
5157
3: R_COMPLEX_VALUE@12..20
5258
0: R_COMPLEX_LITERAL@12..20 "0x123Fi" [Newline("\n")] []
53-
2: EOF@20..21 "" [Newline("\n")] []
59+
4: R_COMPLEX_VALUE@20..75
60+
0: R_COMPLEX_LITERAL@20..75 ".1i" [Newline("\n"), Newline("\n"), Comments("# https://github.com/ ..."), Newline("\n")] []
61+
2: EOF@75..76 "" [Newline("\n")] []
5462
5563
```

‎crates/air_r_parser/tests/snapshots/ok/value/double_value.R‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -3,6 +3,9 @@
33
1e6
44
0x123F
55

6+
# https://github.com/r-lib/tree-sitter-r/pull/198
7+
.1
8+
69
# Hexadecimal
710
# `x` vs `X`
811
0x123

‎crates/air_r_parser/tests/snapshots/ok/value/double_value.R.snap‎

Lines changed: 78 additions & 70 deletions
Original file line numberDiff line numberDiff line change
@@ -10,6 +10,9 @@ expression: snapshot
1010
1e6
1111
0x123F
1212
13+
# https://github.com/r-lib/tree-sitter-r/pull/198
14+
.1
15+
1316
# Hexadecimal
1417
# `x` vs `X`
1518
0x123
@@ -61,82 +64,85 @@ RRoot {
6164
value_token: R_DOUBLE_LITERAL@9..16 "0x123F" [Newline("\n")] [],
6265
},
6366
RDoubleValue {
64-
value_token: R_DOUBLE_LITERAL@16..50 "0x123" [Newline("\n"), Newline("\n"), Comments("# Hexadecimal"), Newline("\n"), Comments("# `x` vs `X`"), Newline("\n")] [],
67+
value_token: R_DOUBLE_LITERAL@16..70 ".1" [Newline("\n"), Newline("\n"), Comments("# https://github.com/ ..."), Newline("\n")] [],
68+
},
69+
RDoubleValue {
70+
value_token: R_DOUBLE_LITERAL@70..104 "0x123" [Newline("\n"), Newline("\n"), Comments("# Hexadecimal"), Newline("\n"), Comments("# `x` vs `X`"), Newline("\n")] [],
6571
},
6672
RDoubleValue {
67-
value_token: R_DOUBLE_LITERAL@50..56 "0X123" [Newline("\n")] [],
73+
value_token: R_DOUBLE_LITERAL@104..110 "0X123" [Newline("\n")] [],
6874
},
6975
RDoubleValue {
70-
value_token: R_DOUBLE_LITERAL@56..85 "0xDEAD" [Newline("\n"), Comments("# Numbers and letters"), Newline("\n")] [],
76+
value_token: R_DOUBLE_LITERAL@110..139 "0xDEAD" [Newline("\n"), Comments("# Numbers and letters"), Newline("\n")] [],
7177
},
7278
RDoubleValue {
73-
value_token: R_DOUBLE_LITERAL@85..92 "0XDEAD" [Newline("\n")] [],
79+
value_token: R_DOUBLE_LITERAL@139..146 "0XDEAD" [Newline("\n")] [],
7480
},
7581
RDoubleValue {
76-
value_token: R_DOUBLE_LITERAL@92..100 "0x1f2F3" [Newline("\n")] [],
82+
value_token: R_DOUBLE_LITERAL@146..154 "0x1f2F3" [Newline("\n")] [],
7783
},
7884
RDoubleValue {
79-
value_token: R_DOUBLE_LITERAL@100..108 "0X1f2F3" [Newline("\n")] [],
85+
value_token: R_DOUBLE_LITERAL@154..162 "0X1f2F3" [Newline("\n")] [],
8086
},
8187
RDoubleValue {
82-
value_token: R_DOUBLE_LITERAL@108..127 "0x0p0" [Newline("\n"), Comments("# `p` vs `P`"), Newline("\n")] [],
88+
value_token: R_DOUBLE_LITERAL@162..181 "0x0p0" [Newline("\n"), Comments("# `p` vs `P`"), Newline("\n")] [],
8389
},
8490
RDoubleValue {
85-
value_token: R_DOUBLE_LITERAL@127..133 "0x0P0" [Newline("\n")] [],
91+
value_token: R_DOUBLE_LITERAL@181..187 "0x0P0" [Newline("\n")] [],
8692
},
8793
RDoubleValue {
88-
value_token: R_DOUBLE_LITERAL@133..141 "0x0p123" [Newline("\n")] [],
94+
value_token: R_DOUBLE_LITERAL@187..195 "0x0p123" [Newline("\n")] [],
8995
},
9096
RDoubleValue {
91-
value_token: R_DOUBLE_LITERAL@141..149 "0x0P123" [Newline("\n")] [],
97+
value_token: R_DOUBLE_LITERAL@195..203 "0x0P123" [Newline("\n")] [],
9298
},
9399
RDoubleValue {
94-
value_token: R_DOUBLE_LITERAL@149..170 "0x0p+0" [Newline("\n"), Comments("# `+` and `-`"), Newline("\n")] [],
100+
value_token: R_DOUBLE_LITERAL@203..224 "0x0p+0" [Newline("\n"), Comments("# `+` and `-`"), Newline("\n")] [],
95101
},
96102
RDoubleValue {
97-
value_token: R_DOUBLE_LITERAL@170..177 "0x0p-0" [Newline("\n")] [],
103+
value_token: R_DOUBLE_LITERAL@224..231 "0x0p-0" [Newline("\n")] [],
98104
},
99105
RDoubleValue {
100-
value_token: R_DOUBLE_LITERAL@177..186 "0x0p+123" [Newline("\n")] [],
106+
value_token: R_DOUBLE_LITERAL@231..240 "0x0p+123" [Newline("\n")] [],
101107
},
102108
RDoubleValue {
103-
value_token: R_DOUBLE_LITERAL@186..195 "0x0p-123" [Newline("\n")] [],
109+
value_token: R_DOUBLE_LITERAL@240..249 "0x0p-123" [Newline("\n")] [],
104110
},
105111
RDoubleValue {
106-
value_token: R_DOUBLE_LITERAL@195..215 "0x." [Newline("\n"), Comments("# Decimal point"), Newline("\n")] [],
112+
value_token: R_DOUBLE_LITERAL@249..269 "0x." [Newline("\n"), Comments("# Decimal point"), Newline("\n")] [],
107113
},
108114
RDoubleValue {
109-
value_token: R_DOUBLE_LITERAL@215..220 "0x1." [Newline("\n")] [],
115+
value_token: R_DOUBLE_LITERAL@269..274 "0x1." [Newline("\n")] [],
110116
},
111117
RDoubleValue {
112-
value_token: R_DOUBLE_LITERAL@220..225 "0x.1" [Newline("\n")] [],
118+
value_token: R_DOUBLE_LITERAL@274..279 "0x.1" [Newline("\n")] [],
113119
},
114120
RDoubleValue {
115-
value_token: R_DOUBLE_LITERAL@225..231 "0x1.1" [Newline("\n")] [],
121+
value_token: R_DOUBLE_LITERAL@279..285 "0x1.1" [Newline("\n")] [],
116122
},
117123
RDoubleValue {
118-
value_token: R_DOUBLE_LITERAL@231..267 "0x.p1" [Newline("\n"), Comments("# Decimal point with ..."), Newline("\n")] [],
124+
value_token: R_DOUBLE_LITERAL@285..321 "0x.p1" [Newline("\n"), Comments("# Decimal point with ..."), Newline("\n")] [],
119125
},
120126
RDoubleValue {
121-
value_token: R_DOUBLE_LITERAL@267..274 "0x1.p1" [Newline("\n")] [],
127+
value_token: R_DOUBLE_LITERAL@321..328 "0x1.p1" [Newline("\n")] [],
122128
},
123129
RDoubleValue {
124-
value_token: R_DOUBLE_LITERAL@274..281 "0x.1p1" [Newline("\n")] [],
130+
value_token: R_DOUBLE_LITERAL@328..335 "0x.1p1" [Newline("\n")] [],
125131
},
126132
RDoubleValue {
127-
value_token: R_DOUBLE_LITERAL@281..289 "0x1.1p1" [Newline("\n")] [],
133+
value_token: R_DOUBLE_LITERAL@335..343 "0x1.1p1" [Newline("\n")] [],
128134
},
129135
],
130-
eof_token: EOF@289..289 "" [] [],
136+
eof_token: EOF@343..343 "" [] [],
131137
}
132138
```
133139

134140
## CST
135141

136142
```
137-
0: R_ROOT@0..289
143+
0: R_ROOT@0..343
138144
0: (empty)
139-
1: R_EXPRESSION_LIST@0..289
145+
1: R_EXPRESSION_LIST@0..343
140146
0: R_DOUBLE_VALUE@0..1
141147
0: R_DOUBLE_LITERAL@0..1 "1" [] []
142148
1: R_DOUBLE_VALUE@1..5
@@ -145,50 +151,52 @@ RRoot {
145151
0: R_DOUBLE_LITERAL@5..9 "1e6" [Newline("\n")] []
146152
3: R_DOUBLE_VALUE@9..16
147153
0: R_DOUBLE_LITERAL@9..16 "0x123F" [Newline("\n")] []
148-
4: R_DOUBLE_VALUE@16..50
149-
0: R_DOUBLE_LITERAL@16..50 "0x123" [Newline("\n"), Newline("\n"), Comments("# Hexadecimal"), Newline("\n"), Comments("# `x` vs `X`"), Newline("\n")] []
150-
5: R_DOUBLE_VALUE@50..56
151-
0: R_DOUBLE_LITERAL@50..56 "0X123" [Newline("\n")] []
152-
6: R_DOUBLE_VALUE@56..85
153-
0: R_DOUBLE_LITERAL@56..85 "0xDEAD" [Newline("\n"), Comments("# Numbers and letters"), Newline("\n")] []
154-
7: R_DOUBLE_VALUE@85..92
155-
0: R_DOUBLE_LITERAL@85..92 "0XDEAD" [Newline("\n")] []
156-
8: R_DOUBLE_VALUE@92..100
157-
0: R_DOUBLE_LITERAL@92..100 "0x1f2F3" [Newline("\n")] []
158-
9: R_DOUBLE_VALUE@100..108
159-
0: R_DOUBLE_LITERAL@100..108 "0X1f2F3" [Newline("\n")] []
160-
10: R_DOUBLE_VALUE@108..127
161-
0: R_DOUBLE_LITERAL@108..127 "0x0p0" [Newline("\n"), Comments("# `p` vs `P`"), Newline("\n")] []
162-
11: R_DOUBLE_VALUE@127..133
163-
0: R_DOUBLE_LITERAL@127..133 "0x0P0" [Newline("\n")] []
164-
12: R_DOUBLE_VALUE@133..141
165-
0: R_DOUBLE_LITERAL@133..141 "0x0p123" [Newline("\n")] []
166-
13: R_DOUBLE_VALUE@141..149
167-
0: R_DOUBLE_LITERAL@141..149 "0x0P123" [Newline("\n")] []
168-
14: R_DOUBLE_VALUE@149..170
169-
0: R_DOUBLE_LITERAL@149..170 "0x0p+0" [Newline("\n"), Comments("# `+` and `-`"), Newline("\n")] []
170-
15: R_DOUBLE_VALUE@170..177
171-
0: R_DOUBLE_LITERAL@170..177 "0x0p-0" [Newline("\n")] []
172-
16: R_DOUBLE_VALUE@177..186
173-
0: R_DOUBLE_LITERAL@177..186 "0x0p+123" [Newline("\n")] []
174-
17: R_DOUBLE_VALUE@186..195
175-
0: R_DOUBLE_LITERAL@186..195 "0x0p-123" [Newline("\n")] []
176-
18: R_DOUBLE_VALUE@195..215
177-
0: R_DOUBLE_LITERAL@195..215 "0x." [Newline("\n"), Comments("# Decimal point"), Newline("\n")] []
178-
19: R_DOUBLE_VALUE@215..220
179-
0: R_DOUBLE_LITERAL@215..220 "0x1." [Newline("\n")] []
180-
20: R_DOUBLE_VALUE@220..225
181-
0: R_DOUBLE_LITERAL@220..225 "0x.1" [Newline("\n")] []
182-
21: R_DOUBLE_VALUE@225..231
183-
0: R_DOUBLE_LITERAL@225..231 "0x1.1" [Newline("\n")] []
184-
22: R_DOUBLE_VALUE@231..267
185-
0: R_DOUBLE_LITERAL@231..267 "0x.p1" [Newline("\n"), Comments("# Decimal point with ..."), Newline("\n")] []
186-
23: R_DOUBLE_VALUE@267..274
187-
0: R_DOUBLE_LITERAL@267..274 "0x1.p1" [Newline("\n")] []
188-
24: R_DOUBLE_VALUE@274..281
189-
0: R_DOUBLE_LITERAL@274..281 "0x.1p1" [Newline("\n")] []
190-
25: R_DOUBLE_VALUE@281..289
191-
0: R_DOUBLE_LITERAL@281..289 "0x1.1p1" [Newline("\n")] []
192-
2: EOF@289..289 "" [] []
154+
4: R_DOUBLE_VALUE@16..70
155+
0: R_DOUBLE_LITERAL@16..70 ".1" [Newline("\n"), Newline("\n"), Comments("# https://github.com/ ..."), Newline("\n")] []
156+
5: R_DOUBLE_VALUE@70..104
157+
0: R_DOUBLE_LITERAL@70..104 "0x123" [Newline("\n"), Newline("\n"), Comments("# Hexadecimal"), Newline("\n"), Comments("# `x` vs `X`"), Newline("\n")] []
158+
6: R_DOUBLE_VALUE@104..110
159+
0: R_DOUBLE_LITERAL@104..110 "0X123" [Newline("\n")] []
160+
7: R_DOUBLE_VALUE@110..139
161+
0: R_DOUBLE_LITERAL@110..139 "0xDEAD" [Newline("\n"), Comments("# Numbers and letters"), Newline("\n")] []
162+
8: R_DOUBLE_VALUE@139..146
163+
0: R_DOUBLE_LITERAL@139..146 "0XDEAD" [Newline("\n")] []
164+
9: R_DOUBLE_VALUE@146..154
165+
0: R_DOUBLE_LITERAL@146..154 "0x1f2F3" [Newline("\n")] []
166+
10: R_DOUBLE_VALUE@154..162
167+
0: R_DOUBLE_LITERAL@154..162 "0X1f2F3" [Newline("\n")] []
168+
11: R_DOUBLE_VALUE@162..181
169+
0: R_DOUBLE_LITERAL@162..181 "0x0p0" [Newline("\n"), Comments("# `p` vs `P`"), Newline("\n")] []
170+
12: R_DOUBLE_VALUE@181..187
171+
0: R_DOUBLE_LITERAL@181..187 "0x0P0" [Newline("\n")] []
172+
13: R_DOUBLE_VALUE@187..195
173+
0: R_DOUBLE_LITERAL@187..195 "0x0p123" [Newline("\n")] []
174+
14: R_DOUBLE_VALUE@195..203
175+
0: R_DOUBLE_LITERAL@195..203 "0x0P123" [Newline("\n")] []
176+
15: R_DOUBLE_VALUE@203..224
177+
0: R_DOUBLE_LITERAL@203..224 "0x0p+0" [Newline("\n"), Comments("# `+` and `-`"), Newline("\n")] []
178+
16: R_DOUBLE_VALUE@224..231
179+
0: R_DOUBLE_LITERAL@224..231 "0x0p-0" [Newline("\n")] []
180+
17: R_DOUBLE_VALUE@231..240
181+
0: R_DOUBLE_LITERAL@231..240 "0x0p+123" [Newline("\n")] []
182+
18: R_DOUBLE_VALUE@240..249
183+
0: R_DOUBLE_LITERAL@240..249 "0x0p-123" [Newline("\n")] []
184+
19: R_DOUBLE_VALUE@249..269
185+
0: R_DOUBLE_LITERAL@249..269 "0x." [Newline("\n"), Comments("# Decimal point"), Newline("\n")] []
186+
20: R_DOUBLE_VALUE@269..274
187+
0: R_DOUBLE_LITERAL@269..274 "0x1." [Newline("\n")] []
188+
21: R_DOUBLE_VALUE@274..279
189+
0: R_DOUBLE_LITERAL@274..279 "0x.1" [Newline("\n")] []
190+
22: R_DOUBLE_VALUE@279..285
191+
0: R_DOUBLE_LITERAL@279..285 "0x1.1" [Newline("\n")] []
192+
23: R_DOUBLE_VALUE@285..321
193+
0: R_DOUBLE_LITERAL@285..321 "0x.p1" [Newline("\n"), Comments("# Decimal point with ..."), Newline("\n")] []
194+
24: R_DOUBLE_VALUE@321..328
195+
0: R_DOUBLE_LITERAL@321..328 "0x1.p1" [Newline("\n")] []
196+
25: R_DOUBLE_VALUE@328..335
197+
0: R_DOUBLE_LITERAL@328..335 "0x.1p1" [Newline("\n")] []
198+
26: R_DOUBLE_VALUE@335..343
199+
0: R_DOUBLE_LITERAL@335..343 "0x1.1p1" [Newline("\n")] []
200+
2: EOF@343..343 "" [] []
193201
194202
```

‎crates/air_r_parser/tests/snapshots/ok/value/integer_value.R‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,9 @@
11
1L
22
1e5L
33

4+
# https://github.com/r-lib/tree-sitter-r/pull/198
5+
.1L
6+
47
# Hexadecimal
58
# `x` vs `X`
69
0x123L

0 commit comments

Comments
 (0)