Semantic Tags: Mass tagging project.

[andrewfg]

This is to inform developers about planned next steps in mass tagging of addons.

• There are 502 addons in OH
• There are 2236 thing- and bridge-types, of which 1440 (65%) are tagged
• There are 13'229 channel-types, of which 429 (3%) are tagged

In the last few days I started a mass tagging program (#18549, #18557, #18558, and #18563), in order to apply Equipment tags an all (almost) Thing-types in OH-Addons (which is why 1440 are now tagged)

It does NOT apply Point + Property tags on the Channel-types.

For this first mass tagging project, the PRs are split across bindings alphabetically by name.

Once the Thing-type Equipment tag PRs are merged, the next step is to apply Channel-type Point + Property tags. This will not be so easy.

Instead of making PRs split across bindings alphabetically by name, I intend to make PRs that are split across bindings functionally by Equipment tag. Some of the first mass tagging PR priorities are for subsets of Channel-types across the full set of addons, that are filtered according to the (newly applied) Equipment tags as follows:

1. LightSource, Lightbulb, LightStrip and children (ca. 78 thing-types)
2. Thermostat, Boiler, AirConditioner, HVAC and children (ca. 64 thing-types).
3. WindowCovering, Blinds, Drapes (ca. 32 thing-types)..
4. MediaPlayer, Speaker and children (ca. 59 thing-types)..
5. Maybe WeatherStation and WeatherService (ca. 45 thing-types).
6. Maybe WhiteGood and children (ca. 27 thing-types).
7. Inverter (ca. 28 thing-types).
8. Vehicle (ca. 17 thing-types).
9. Receiver (ca. 56 thing-types).
10. NetworkAppliance (ca. 87 thing-types).
11. WebService (ca. 75 thing-types).

=> Any thoughts about a relatively automated way to tag everything else?

[andrewfg]

Pinging @jlaur / @jimtng / @lsiepel / @ccutrer for your inputs.

[ccutrer]

Sounds reasonable to me.

[andrewfg]

For info: below are the statistics for the Thing-types currently in OH-Addons and having Equipment tags on them, sorted by popularity:

semantic-equipment-tag	thing-type-count	percent
ControlDevice	102	7.1%
NetworkAppliance	96	6.7%
PowerOutlet	86	6.0%
WebService	78	5.4%
AudioVisual	67	4.7%
ElectricMeter	66	4.6%
Sensor	58	4.0%
LightSource	58	4.0%
Receiver	56	3.9%
Thermostat	46	3.2%
Lightbulb	39	2.7%
Blinds	34	2.4%
Speaker	31	2.2%
Inverter	28	1.9%
MediaPlayer	28	1.9%
WeatherStation	27	1.9%
WeatherService	27	1.9%
Button	25	1.7%
AlarmSystem	20	1.4%
AlarmDevice	20	1.4%
ContactSensor	17	1.2%
Vehicle	17	1.2%
HeatPump	16	1.1%
WallSwitch	15	1.0%
Camera	14	1.0%
HVAC	13	0.9%
MotionDetector	13	0.9%
Boiler	12	0.8%
Television	12	0.8%
Keypad	11	0.8%
TemperatureSensor	11	0.8%
SmokeDetector	10	0.7%
WashingMachine	9	0.6%
Lock	9	0.6%
RemoteControl	8	0.6%
Refrigerator	8	0.6%
Doorbell	8	0.6%
LightStrip	8	0.6%
Projector	8	0.6%
WindowCovering	8	0.6%
AirConditioner	8	0.6%
GasMeter	7	0.5%
WaterMeter	7	0.5%
LeakSensor	7	0.5%
OccupancySensor	7	0.5%
EVSE	7	0.5%
Irrigation	6	0.4%
CleaningRobot	6	0.4%
WhiteGood	6	0.4%
Pump	6	0.4%
PowerSupply	6	0.4%
Fan	6	0.4%
WaterHeater	6	0.4%
Display	5	0.3%
RadiatorControl	5	0.3%
AirQualitySensor	5	0.3%
Dishwasher	4	0.3%
Router	4	0.3%
CeilingFan	4	0.3%
Wellness	4	0.3%
Dryer	4	0.3%
Battery	4	0.3%
Humidifier	4	0.3%
Drapes	4	0.3%
AirFilter	4	0.3%
Chlorinator	3	0.2%
Cooktop	3	0.2%
Furnace	3	0.2%
SwimmingPool	3	0.2%
WirelessAccessPoint	3	0.2%
CO2Sensor	3	0.2%
IlluminanceSensor	3	0.2%
LawnMower	3	0.2%
Tracker	3	0.2%
KitchenHood	3	0.2%
Siren	3	0.2%
Dial	3	0.2%
GarageDoor	3	0.2%
Oven	3	0.2%
HumiditySensor	3	0.2%
Computer	2	0.1%
COSensor	2	0.1%
VibrationSensor	2	0.1%
WaterSoftener	2	0.1%
SolarPanel	2	0.1%
Window	2	0.1%
PetCare	2	0.1%
HeatRecovery	2	0.1%
Valve	2	0.1%
SpotLight	2	0.1%
Dehumidifier	2	0.1%
PetFeeder	1	0.1%
WaterFilter	1	0.1%
UPS	1	0.1%
HotWaterFaucet	1	0.1%
DrinkingWater	1	0.1%
Generator	1	0.1%
FrontDoor	1	0.1%
Bed	1	0.1%
ExhaustFan	1	0.1%
WindGenerator	1	0.1%
Printer	1	0.1%
SmartVent	1	0.1%
Freezer	1	0.1%
HeatDetector	1	0.1%
TransferSwitch	1	0.1%
ControlDevicek	1	0.1%
Horticulture	1	0.1%
PetFlap	1	0.1%
TOTAL	1440

[andrewfg]

The point+property mass tagging project would be 10x larger than the Equipment tagging project now under review.

So I am thinking of writing scripts to do the tagging. But I am not sure if this would take more work than doing it by hand..

[mlobstein]

Typo?

[andrewfg]

Typo?

Good catch.

---

@mlobstein a great many thanks for pointing this one out. It made me realise that the Maven Build is currently NOT checking the validity of thing-description xml files against the XSD schema. :( So it prompted me to open #18579 to resolve that. :)

[andrewfg]

Hmm. I have been working on some code that could automatically apply point+property tags to the channel-types based on matching patterns of key words in the channel-type id, kind, item-type, label and description texts. And it looks as if this could achieve almost 98% coverage. I suspect it would include quite a few wrong tags, but it is perhaps easier to correct/eliminate the wrong ones rather than completely create them from scratch. => WDYT?

    private static String[] estimatePointPropertyTags(Node node) {
        if (hasTags(node)) {
            return new String[] {};
        }

        String itemType = getElementValue(node, "item-type");
        if (itemType == null) {
            return new String[] {};
        }

        String kind = getElementValue(node, "kind");
        if ("trigger".equalsIgnoreCase(kind)) {
            return new String[] {};
        }

        String id = getAttributeValue(node, "id");
        String label = getElementValue(node, "label");

        if (id == null || label == null) {
            return new String[] {};
        }

        String description = getElementValue(node, "description");
        String category = getElementValue(node, "category");

        boolean isWritable = "false".equalsIgnoreCase(getStateAttribute(node, "readOnly"));
        List<String> sourceTexts = new ArrayList<>(Arrays.asList(id, label));
        if (description != null) {
            sourceTexts.add(description);
        }

        String point = null;
        String property = null;

        switch (itemType) {
            case "Rollershutter":
                point = "Control";
                property = "OpenLevel";
                break;

            case "String":
                point = isWritable ? "Control" : "Status";
                break;

            case "Switch":
                if (keywordMatch(List.of("alarm"), sourceTexts)) {
                    point = "Alarm";
                } else if (keywordMatch(List.of("status"), sourceTexts)) {
                    point = "Status";
                } else if (keywordMatch(List.of("control", "command", "switches"), sourceTexts)) {
                    point = "Control";
                } else {
                    point = isWritable ? "Control" : "Status";
                }
                break;

            case "Dimmer":
                point = "Control";
                if (keywordMatch(List.of("thermostat", "target", "setpoint"), sourceTexts)) {
                    point = "Setpoint";
                    property = "Temperature";
                } else if (keywordMatch(List.of("brightness", "dimm"), sourceTexts)) {
                    property = "Brightness";
                } else if (keywordMatch(List.of("color"), sourceTexts)) {
                    property = "ColorTemperature";
                } else if (keywordMatch(List.of("tilt", "slat", "vane"), sourceTexts)) {
                    property = "Tilt";
                } else if (keywordMatch(List.of("window", "shade", "shutter"), sourceTexts)) {
                    property = "OpenLevel";
                } else if (keywordMatch(List.of("sound", "volume"), sourceTexts)) {
                    property = "SoundVolume";
                }
                break;

            case "DateTime":
                point = "Status";
                property = "TimeStamp";
                break;

            case "Color":
                point = "Control";
                property = "Color";
                break;

            case "Contact":
                if (keywordMatch(List.of("alarm", "warning", "error", "fault"), sourceTexts)) {
                    point = "Alarm";
                } else {
                    point = "Status";
                }
                break;

            case "Number":
                // fall through

            default:
                if (keywordMatch(List.of("forecast", "prediction"), sourceTexts)) {
                    point = "Forecast";
                } else if (keywordMatch(
                        List.of("thermostat", "target", "setpoint", "set-point", "set point", "set value"),
                        sourceTexts)) {
                    point = "Setpoint";
                } else if (keywordMatch(List.of("control", "command"), sourceTexts)) {
                    point = "Control";
                } else {
                    point = isWritable ? "Control" : "Measurement";
                }

                // Number:UNIT
                String[] itemParts = itemType.split(":");
                if (itemParts.length == 2) {
                    if ("Number".equals(itemParts[0])) {
                        switch (itemParts[1]) {
                            // case "Acceleration":
                            // case "AmountOfSubstance":
                            case "Angle":
                                if (keywordMatch(List.of("wind"), sourceTexts)) {
                                    property = "Wind";
                                }
                                break;
                            // case "Area":
                            // case "ArealDensity":
                            // case "CatalyticActivity":
                            // case "Currency":
                            // case "DataAmount":
                            // case "DataTransferRate":
                            // case "Density":
                            case "Dimensionless":
                                if (keywordMatch(List.of("humidity"), sourceTexts)) {
                                    property = "Humidity";
                                } else if (keywordMatch(List.of("volume", "sound"), sourceTexts)) {
                                    property = "SoundVolume";
                                }
                                break;
                            // case "ElectricCapacitance":
                            // case "ElectricCharge":
                            // case "ElectricConductance":
                            // case "ElectricConductivity":
                            case "ElectricCurrent":
                                property = "Current";
                                break;
                            // case "ElectricInductance":
                            case "ElectricPotential":
                                property = "Voltage";
                                break;
                            // case "ElectricResistance":
                            // case "EmissionIntensity":
                            case "Energy":
                                property = itemParts[1];
                                break;
                            // case "EnergyPrice":
                            // case "Force":
                            case "Frequency":
                                property = itemParts[1];
                                break;
                            case "Illuminance":
                                property = itemParts[1];
                                break;
                            // case "Intensity":
                            case "Length":
                                if (keywordMatch(List.of("rain"), sourceTexts)) {
                                    property = "Rain";
                                }
                                break;
                            // case "LuminousFlux":
                            // case "LuminousIntensity":
                            // case "MagneticFlux":
                            // case "MagneticFluxDensity":
                            // case "Mass":
                            case "Power":
                                if (keywordMatch(List.of("signal"), sourceTexts)) {
                                    property = "SignalStrength";
                                } // no else
                                if (keywordMatch(List.of("rssi", " db ", " dbw ", "decibel"), sourceTexts)) {
                                    property = "RSSI";
                                }
                                if (property == null) {
                                    property = itemParts[1];
                                }
                                break;
                            case "Pressure":
                                property = itemParts[1];
                                break;
                            // case "RadiationDoseAbsorbed":
                            // case "RadiationDoseEffective":
                            // case "RadiationSpecificActivity":
                            // case "RadioactiveActivity":
                            // case "SolidAngle":
                            case "Speed":
                                if (keywordMatch(List.of("wind"), sourceTexts)) {
                                    property = "Wind";
                                } else {
                                    property = itemParts[1];
                                }
                                break;
                            case "Temperature":
                                if (keywordMatch(List.of("color", "colour", "kelvin", "mirek", "mired"), sourceTexts)) {
                                    property = "ColorTemperature";
                                } else {
                                    property = itemParts[1];
                                }
                                break;
                            case "Time":
                                if (keywordMatch(List.of("last", "updated", "stamp"), sourceTexts)) {
                                    point = "Status";
                                    property = "Timestamp";
                                } else {
                                    property = "Duration";
                                }
                                break;
                            // case "Volume":
                            // case "VolumetricFlowRate":
                        }
                    }
                }
        }

        // estimate property if missing
        if (point != null && property == null) {
            if (keywordMatch(List.of("conditioning", "cooling"), sourceTexts)) {
                property = "Airconditioning";
            } else if (keywordMatch(List.of("air flow"), sourceTexts)) {
                property = "Airflow";
            } else if (keywordMatch(List.of("air quality"), sourceTexts)) {
                property = "AirQuality";
            } else if (keywordMatch(List.of(" app ", "application"), sourceTexts)) {
                property = "App";
            } else if (keywordMatch(List.of(" aqi "), sourceTexts)) {
                property = "AQI";
            } else if (keywordMatch(List.of("brightness", "dimm"), sourceTexts)) {
                property = "Brightness";
            } else if (keywordMatch(List.of("channel"), sourceTexts)) {
                property = "Channel";
            } else if (keywordMatch(List.of(" co ", "carbon monoxide"), sourceTexts)) {
                property = "CO";
            } else if (keywordMatch(List.of(" co2 ", "carbon dioxide"), sourceTexts)) {
                property = "CO2";
            } else if (keywordMatch(List.of("color"), sourceTexts)) {
                property = "Color";
            } else if (keywordMatch(List.of("color temperature", "colour temperature", "kelvin", "mirek", "mired"),
                    sourceTexts)) {
                property = "ColorTemperature";
            } else if (keywordMatch(List.of("current", "ampere"), sourceTexts)) {
                property = "Current";
            } else if (keywordMatch(List.of("duration"), sourceTexts)) {
                property = "Duration";
                // } else if (keywordMatch(List.of("enabled", "disabled"), sourceTexts)) {
                // property = "Enabled";
            } else if (keywordMatch(List.of("energy"), sourceTexts)) {
                property = "Energy";
            } else if (keywordMatch(List.of("frequency"), sourceTexts)) {
                property = "Frequency";
            } else if (keywordMatch(List.of("gas"), sourceTexts)) {
                property = "Gas";
            } else if (keywordMatch(List.of("location"), sourceTexts)) {
                property = "GeoLocation";
            } else if (keywordMatch(List.of("heating"), sourceTexts)) {
                property = "Heating";
            } else if (keywordMatch(List.of("humidity"), sourceTexts)) {
                property = "Humidity";
            } else if (keywordMatch(List.of("illuminance", " lux ", "light level"), sourceTexts)) {
                property = "Illuminance";
            } else if (keywordMatch(List.of("level"), sourceTexts)) {
                property = "Level";
            } else if (keywordMatch(List.of("light"), sourceTexts)) {
                property = "Light";
            } else if (keywordMatch(List.of("low battery", "battery alarm"), sourceTexts)) {
                property = "LowBattery";
            } else if (keywordMatch(List.of("media", "player", "television", "tv", "receiver"), sourceTexts)) {
                property = "MediaControl";
            } else if (keywordMatch(List.of("mode", "auto", "manual", "enabled", "disabled"), sourceTexts)) {
                property = "Mode";
            } else if (keywordMatch(List.of("moisture"), sourceTexts)) {
                property = "Moisture";
            } else if (keywordMatch(List.of("motion"), sourceTexts)) {
                property = "Motion";
            } else if (keywordMatch(List.of("noise"), sourceTexts)) {
                property = "Noise";
            } else if (keywordMatch(List.of("oil"), sourceTexts)) {
                property = "Oil";
            } else if (keywordMatch(List.of("open"), sourceTexts)) {
                property = "Opening";
            } else if (keywordMatch(List.of("ozone"), sourceTexts)) {
                property = "Ozone";
            } else if (keywordMatch(List.of("particulate", " PM"), sourceTexts)) {
                property = "ParticulateMatter";
            } else if (keywordMatch(List.of("pollen"), sourceTexts)) {
                property = "Pollen";
            } else if (keywordMatch(List.of("position"), sourceTexts)) {
                property = "Position";
            } else if (keywordMatch(List.of("power", " on", "off"), sourceTexts)) {
                property = "Power";
            } else if (keywordMatch(List.of("presence", "occup"), sourceTexts)) {
                property = "Presence";
            } else if (keywordMatch(List.of("pressure"), sourceTexts)) {
                property = "Pressure";
            } else if (keywordMatch(List.of("quality "), sourceTexts)) {
                property = "QualityOfService";
            } else if (keywordMatch(List.of("radon"), sourceTexts)) {
                property = "Radon";
            } else if (keywordMatch(List.of("rain"), sourceTexts)) {
                property = "Rain";
            } else if (keywordMatch(List.of(" rssi "), sourceTexts)) {
                property = "RSSI";
            } else if (keywordMatch(List.of("signal strength "), sourceTexts)) {
                property = "SignalStrength";
            } else if (keywordMatch(List.of("smoke"), sourceTexts)) {
                property = "Smoke";
            } else if (keywordMatch(List.of("sound", "audio", "decibel", " db ", " dba "), sourceTexts)) {
                property = "SoundVolume";
            } else if (keywordMatch(List.of("speed", "velocity"), sourceTexts)) {
                property = "Speed";
            } else if (keywordMatch(List.of("tamper"), sourceTexts)) {
                property = "Tampered";
            } else if (keywordMatch(List.of("temperature"), sourceTexts)) {
                property = "Temperature";
            } else if (keywordMatch(List.of("tilt", "vane", "slat"), sourceTexts)) {
                property = "Tilt";
            } else if (keywordMatch(List.of("stamp"), sourceTexts)) {
                property = "Timestamp";
            } else if (keywordMatch(List.of("ultra", "UV"), sourceTexts)) {
                property = "Ultraviolet";
            } else if (keywordMatch(List.of("ventilat", "fan"), sourceTexts)) {
                property = "Ventilation";
            } else if (keywordMatch(List.of("vibration"), sourceTexts)) {
                property = "Vibration";
            } else if (keywordMatch(List.of(" voc "), sourceTexts)) {
                property = "VOC";
            } else if (keywordMatch(List.of("volt"), sourceTexts)) {
                property = "Voltage";
            } else if (keywordMatch(List.of("water"), sourceTexts)) {
                property = "Water";
            } else if (keywordMatch(List.of("wind"), sourceTexts)) {
                property = "Wind";
            }
        }

        if (point != null) {
            if (property != null) {
                return new String[] { point, property };
            } else {
                return new String[] { point };
            }
        }
        return new String[] {};
    }

    private static boolean keywordMatch(List<String> keywords, List<String> args) {
        for (String keyword : keywords) {
            for (String arg : args) {
                if (arg != null && keyword != null && arg.toLowerCase().contains(keyword.toLowerCase())) {
                    return true;
                }
            }
        }
        return false;
    }

[andrewfg]

Status overview:

• Bindings A thru F: Add equipment tags #18549
• Bindings G thru L: Add equipment tags #18557
• Bindings M thru R: Add equipment tags #18558
• Bindings S thru Z: Add equipment tags #18563
• Add curated tags to addons (1/9) [cherry picked] #18664
• Add curated tags to addons (2/9) [cherry picked] #18816
• Add curated tags to addons (3/9) [A thru C] #18863
• Add curated tags to addons (4/9) [D thru F] #18864
• Add curated tags to addons (5/9) [G thru J] #18865
• Add curated tags to addons (6/9) [K thru M] #18867
• Add curated tags to addons (7/9) [N thru Q] #18868
• Add curated tags to addons (8/9) [R thru S] #18871
• Add curated tags to addons (9/9) [T thru Z] #18875

[andrewfg]

Closed via #18585