Skip to content

Commit c00d5ee

Browse files
lfoppianolfoppiano
authored
Merge branch 'master' into 0.7.2-fixes
2 parents 6390594 + 0412e2e commit c00d5ee

2 files changed

Lines changed: 79 additions & 73 deletions

File tree

‎README.md‎

Lines changed: 78 additions & 72 deletions
Original file line numberDiff line numberDiff line change
@@ -29,11 +29,21 @@ As the other GROBID models, the module relies only on machine learning and it us
2929
The normalisation of quantities is handled by the java
3030
library [Units of measurement](http://unitsofmeasurement.github.io/).
3131

32+
## Online demo
33+
34+
Grobid-quantities can be tested with the online demo kindly offered by Huggingface
35+
Spaces: https://lfoppiano-grobid-quantities.hf.space/
36+
3237
## Latest version
3338

3439
The latest released version of grobid-quantities
35-
is [0.7.1](https://github.com/kermitt2/grobid-quantities/releases/tag/0.7.1). The current development version is
36-
0.7.2-SNAPSHOT.
40+
is [0.7.2](https://github.com/kermitt2/grobid-quantities/releases/tag/0.7.2). The current development version is
41+
0.7.3-SNAPSHOT.
42+
43+
### Update from 0.7.1 to 0.7.2
44+
45+
In version 0.7.2 we have updated the DeLFT models.
46+
The DL models must be updated by running `./gradlew copyModels`.
3747

3848
### Update from 0.7.0 to 0.7.1
3949

@@ -51,51 +61,49 @@ You can find the latest documentation [here](http://grobid-quantities.readthedoc
5161

5262
## Evaluation
5363

54-
The results (Precision, Recall, F-score) for all the models have been obtained using an holdout set.
55-
For DL models we provide the average over 5 runs.
64+
The results (Precision, Recall, F-score) for all the models have been obtained using an holdout set.
65+
For DL models we provide the average over 5 runs.
5666
Update on the 27/10/2022
5767

58-
#### Quantities
59-
60-
| Labels | CRF | | | **BidLSTM_CRF** | | |
61-
|-----------------|----------------|-------------|---------------|-----------------|-------------|---------------|
62-
| Metrics | **Precision** | **Recall** | **F1-Score** | **Precision** | **Recall** | **F1-Score** |
63-
| `<unitLeft>` | 88.74 | 83.19 | 85.87 | 88.56 | 92.07 | 90.28 |
64-
| `<unitRight>` | 30.77 | 30.77 | 30.77 | 24.75 | 30.77 | 27.42 |
65-
| `<valueAtomic> | 76.29 | 78.66 | 77.46 | 78.14 | 86.06 | 81.90 |
66-
| `<valueBase>` | 84.62 | 62.86 | 72.13 | 83.51 | 94.86 | 88.61 |
67-
| `<valueLeast>` | 77.68 | 69.05 | 73.11 | 82.14 | 60.63 | 69.67 |
68-
| `<valueList>` | 45.45 | 18.87 | 26.67 | 62.15 | 10.19 | 17.34 |
69-
| `<valueMost>` | 71.62 | 54.64 | 61.99 | 77.64 | 68.25 | 72.61 |
70-
| `<valueRange>` | 100 | 97.14 | 98.55 | 96.72 | 100.00 | 98.32 |
71-
| -- | | | | | | |
72-
| All (micro avg) | 80.08 | 75 | 77.45 | 81.81 | 81.73 | 81.76 |
68+
#### Quantities
69+
70+
| Labels | CRF | | | **BidLSTM_CRF** | | |
71+
|-----------------|---------------|------------|--------------|-----------------|------------|--------------|
72+
| Metrics | **Precision** | **Recall** | **F1-Score** | **Precision** | **Recall** | **F1-Score** |
73+
| `<unitLeft>` | 88.74 | 83.19 | 85.87 | 88.56 | 92.07 | 90.28 |
74+
| `<unitRight>` | 30.77 | 30.77 | 30.77 | 24.75 | 30.77 | 27.42 |
75+
| `<valueAtomic> | 76.29 | 78.66 | 77.46 | 78.14 | 86.06 | 81.90 |
76+
| `<valueBase>` | 84.62 | 62.86 | 72.13 | 83.51 | 94.86 | 88.61 |
77+
| `<valueLeast>` | 77.68 | 69.05 | 73.11 | 82.14 | 60.63 | 69.67 |
78+
| `<valueList>` | 45.45 | 18.87 | 26.67 | 62.15 | 10.19 | 17.34 |
79+
| `<valueMost>` | 71.62 | 54.64 | 61.99 | 77.64 | 68.25 | 72.61 |
80+
| `<valueRange>` | 100 | 97.14 | 98.55 | 96.72 | 100.00 | 98.32 |
81+
| -- | | | | | | |
82+
| All (micro avg) | 80.08 | 75 | 77.45 | 81.81 | 81.73 | 81.76 |
7383

7484
#### Units
7585

76-
| | **CRF** | | | **BidLSTM_CRF** | | |
77-
|-----------------|---------------------|------------|--------------|------------------|------------|----------------|
78-
| Labels | **Precision** | **Recall** | **F1-Score** | **Precision** | **Recall** | **F1-Score** |
79-
| `<base>` | 80.57 | 82.34 | 81.45 | 56.01 | 50.34 | 53.02 |
80-
| `<pow>` | 72.65 | 74.45 | 73.54 | 93.70 | 62.38 | 74.88 |
81-
| `<prefix>` | 93.8 | 84.69 | 89.02 | 80.31 | 85.25 | 82.54 |
82-
| -- | | | | | | |
83-
| All (micro avg) | 80.73 | 80.6 | 80.66 | 70.19 | 60.88 | 65.20 |
84-
86+
| | **CRF** | | | **BidLSTM_CRF** | | |
87+
|-----------------|---------------|------------|--------------|-----------------|------------|--------------|
88+
| Labels | **Precision** | **Recall** | **F1-Score** | **Precision** | **Recall** | **F1-Score** |
89+
| `<base>` | 80.57 | 82.34 | 81.45 | 56.01 | 50.34 | 53.02 |
90+
| `<pow>` | 72.65 | 74.45 | 73.54 | 93.70 | 62.38 | 74.88 |
91+
| `<prefix>` | 93.8 | 84.69 | 89.02 | 80.31 | 85.25 | 82.54 |
92+
| -- | | | | | | |
93+
| All (micro avg) | 80.73 | 80.6 | 80.66 | 70.19 | 60.88 | 65.20 |
8594

8695
#### Values
8796

88-
| | **CRF** | | | **BidLSTM_CRF** | | |
89-
|-----------------|------------------|------------|--------------|-----------------|------------|----------|
90-
| Labels | **Precision** | **Recall** | **F1-Score** | **Precision** | **Recall** | F1-Score |
91-
| `<alpha>` | 97.66 | 99.21 | 98.43 | 99.53 | 100.00 | 99.76 |
92-
| `<base>` | 76.92 | 76.92 | 76.92 | 98.18 | 76.92 | 86.23 |
93-
| `<number>` | 97.8 | 98.52 | 98.16 | 98.93 | 97.73 | 98.32 |
94-
| `<pow>` | 83.33 | 76.92 | 80 | 100.00 | 76.92 | 86.96 |
95-
| `<time>` | 70.18 | 86.96 | 77.67 | 73.17 | 96.52 | 83.05 |
96-
| -- | | | | | | |
97-
| All (micro avg) | 95.81 | 97.52 | 96.66 | 97.41 | 97.42 | 97.41 |
98-
97+
| | **CRF** | | | **BidLSTM_CRF** | | |
98+
|-----------------|---------------|------------|--------------|-----------------|------------|----------|
99+
| Labels | **Precision** | **Recall** | **F1-Score** | **Precision** | **Recall** | F1-Score |
100+
| `<alpha>` | 97.66 | 99.21 | 98.43 | 99.53 | 100.00 | 99.76 |
101+
| `<base>` | 76.92 | 76.92 | 76.92 | 98.18 | 76.92 | 86.23 |
102+
| `<number>` | 97.8 | 98.52 | 98.16 | 98.93 | 97.73 | 98.32 |
103+
| `<pow>` | 83.33 | 76.92 | 80 | 100.00 | 76.92 | 86.96 |
104+
| `<time>` | 70.18 | 86.96 | 77.67 | 73.17 | 96.52 | 83.05 |
105+
| -- | | | | | | |
106+
| All (micro avg) | 95.81 | 97.52 | 96.66 | 97.41 | 97.42 | 97.41 |
99107

100108
<details>
101109
<summary>Previous evaluations</summary>
@@ -107,47 +115,45 @@ The `BidLSTM_CRF_FEATURES` model was evaluated on the 28/11/2021
107115

108116
#### Quantities
109117

110-
| | CRF | | | BidLSTM_CRF_FEATURES | | |
111-
|-----------------|-----------------|--------------|-----------------|----------------------------|--------------|----------|
112-
| Labels | **Precision** | **Recall** | **F1-Score** | **Precision** | **Recall** | F1-Score |
113-
| `<unitLeft>` | 96.45 | 95.06 | 95.74 | 95.17 | 96.67 | 95.91 |
114-
| `<unitRight>` | 88.96 | 68.65 | 75.43 | 92.52 | 83.64 | 87.69 |
115-
| `<valueAtomic> | 85.75 | 85.35 | 85.49 | 81.74 | 89.21 | 85.30 |
116-
| `<valueBase>` | 73.06 | 66.43 | 68.92 | 100.00 | 75.00 | 85.71 |
117-
| `<valueLeast>` | 85.68 | 79.03 | 82.07 | 89.24 | 82.25 | 85.55 |
118-
| `<valueList>` | 68.38 | 53.31 | 58.94 | 75.27 | 75.33 | 75.12 |
119-
| `<valueMost>` | 83.67 | 75.82 | 79.42 | 89.02 | 81.56 | 85.10 |
120-
| `<valueRange>` | 90.25 | 88.58 | 88.86 | 100.00 | 96.25 | 97.90 |
121-
| -- | | | | | | |
122-
| All (micro avg) | 88.96 | 85.4 | 87.14 | 87.23 | 89.00 | 88.10 |
123-
118+
| | CRF | | | BidLSTM_CRF_FEATURES | | |
119+
|-----------------|---------------|------------|--------------|----------------------|------------|----------|
120+
| Labels | **Precision** | **Recall** | **F1-Score** | **Precision** | **Recall** | F1-Score |
121+
| `<unitLeft>` | 96.45 | 95.06 | 95.74 | 95.17 | 96.67 | 95.91 |
122+
| `<unitRight>` | 88.96 | 68.65 | 75.43 | 92.52 | 83.64 | 87.69 |
123+
| `<valueAtomic> | 85.75 | 85.35 | 85.49 | 81.74 | 89.21 | 85.30 |
124+
| `<valueBase>` | 73.06 | 66.43 | 68.92 | 100.00 | 75.00 | 85.71 |
125+
| `<valueLeast>` | 85.68 | 79.03 | 82.07 | 89.24 | 82.25 | 85.55 |
126+
| `<valueList>` | 68.38 | 53.31 | 58.94 | 75.27 | 75.33 | 75.12 |
127+
| `<valueMost>` | 83.67 | 75.82 | 79.42 | 89.02 | 81.56 | 85.10 |
128+
| `<valueRange>` | 90.25 | 88.58 | 88.86 | 100.00 | 96.25 | 97.90 |
129+
| -- | | | | | | |
130+
| All (micro avg) | 88.96 | 85.4 | 87.14 | 87.23 | 89.00 | 88.10 |
124131

125132
#### Units
126133

127134
CRF was updated the 10/02/2021
128135

129-
| | CRF | | | BidLSTM_CRF_FEATURES | | |
130-
|-----------------|---------------|------------|--------------|----------------------|------------|-----------|
131-
| Labels | **Precision** | **Recall** | **F1-Score** | **Precision** | **Recall** | F1-Score |
132-
| `<base>` | 98.82 | 99.14 | 98.98 | 98.26 | 98.52 | 98.39 |
133-
| `<pow>` | 97.62 | 98.56 | 98.08 | 100.00 | 98.57 | 99.28 |
134-
| `<prefix>` | 99.5 | 98.76 | 99.13 | 98.89 | 97.75 | 98.30 |
135-
| -- | | | | | | |
136-
| All (micro avg) | 98.85 | 99.01 | 98.93 | 98.51 | 98.39 | 98.45 |
137-
136+
| | CRF | | | BidLSTM_CRF_FEATURES | | |
137+
|-----------------|---------------|------------|--------------|----------------------|------------|----------|
138+
| Labels | **Precision** | **Recall** | **F1-Score** | **Precision** | **Recall** | F1-Score |
139+
| `<base>` | 98.82 | 99.14 | 98.98 | 98.26 | 98.52 | 98.39 |
140+
| `<pow>` | 97.62 | 98.56 | 98.08 | 100.00 | 98.57 | 99.28 |
141+
| `<prefix>` | 99.5 | 98.76 | 99.13 | 98.89 | 97.75 | 98.30 |
142+
| -- | | | | | | |
143+
| All (micro avg) | 98.85 | 99.01 | 98.93 | 98.51 | 98.39 | 98.45 |
138144

139145
#### Values
140146

141-
| | CRF | | | BidLSTM_CRF_FEATURES | | |
142-
|-----------------|---------------|------------|--------------|----------------------|------------|-----------|
143-
| Labels | **Precision** | **Recall** | **F1-Score** | **Precision** | **Recall** | F1-Score |
144-
| `<alpha>` | 96.9 | 98.84 | 97.85 | 99.41 | 99.55 | 99.48 |
145-
| `<base>` | 85.14 | 74.48 | 79 | 96.67 | 100.00 | 98.00 |
146-
| `<number>` | 98.07 | 99.05 | 98.55 | 99.55 | 98.68 | 99.11 |
147-
| `<pow>` | 80.05 | 76.33 | 77.54 | 72.50 | 75.00 | 73.50 |
148-
| `<time>` | 73.07 | 86.82 | 79.26 | 80.84 | 100.00 | 89.28 |
149-
| -- | | | | | | |
150-
| All (micro avg) | 96.15 | 97.95 | 97.4 | 98.49 | 98.66 | 98.57 |
147+
| | CRF | | | BidLSTM_CRF_FEATURES | | |
148+
|-----------------|---------------|------------|--------------|----------------------|------------|----------|
149+
| Labels | **Precision** | **Recall** | **F1-Score** | **Precision** | **Recall** | F1-Score |
150+
| `<alpha>` | 96.9 | 98.84 | 97.85 | 99.41 | 99.55 | 99.48 |
151+
| `<base>` | 85.14 | 74.48 | 79 | 96.67 | 100.00 | 98.00 |
152+
| `<number>` | 98.07 | 99.05 | 98.55 | 99.55 | 98.68 | 99.11 |
153+
| `<pow>` | 80.05 | 76.33 | 77.54 | 72.50 | 75.00 | 73.50 |
154+
| `<time>` | 73.07 | 86.82 | 79.26 | 80.84 | 100.00 | 89.28 |
155+
| -- | | | | | | |
156+
| All (micro avg) | 96.15 | 97.95 | 97.4 | 98.49 | 98.66 | 98.57 |
151157

152158
</details>
153159

‎gradle.properties‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1 +1 @@
1-
version=0.7.2
1+
version=0.7.3-SNAPSHOT

0 commit comments

Comments
 (0)