How to connect smart sensor when using pybricksdev?

[repkovsky]

How to connect smart sensor when using pybricksdev? I have tried several combinations of steps but I couldn't find any that matches the one available using online code editor.

[dlech]

Can you give an example of a "smart sensor"? (These days everything gets call "smart", so I'm not sure what exactly you are referring to. It could be anything. 😄)

[repkovsky]

I mean pybricksdev counterpart of your instruction how to connect phone to hub as smart sensor https://pybricks.com/learn/smart-sensors/getting-started/#your-phone-as-a-wireless-sensor :) (but you can still come up with some more distictive name for it :P )

[laurensvalk]

Yep, those are the ones. We'd love some good suggestions for better names.

• Anything with "vision/camera sensors"... But it won't just be for vision/camera.
• Anything with "phone"... But it's not just for phones.
• Anything with "machine learning"... But some of them just use OpenCV.
• Anything with "AI"... These days that's a synonym for LLMs, but we don't have those and don't plan to add them. Marketing would pick this one but we don't do marketing and we don't like deceptive names 😄

They're basically sensors that do some convenient preprocessing, hence "smart sensors" for now.

[repkovsky]

Well, tough choices 😄. I agree with all these arguments if you want to be strict, but in the end I would go for something like "AI sensors". From my perspective (I work with computer vision, signal processing and ML) AI is more widely understood as algorithms based on neural networks in contrast to machine learning, which becomes more and more synonym for all classical ML algorithms. Note that we use term "generative AI" a lot, so there must be some "non-generative AI" and LLMs are equal to AI only in layman view, while your audience are people with some programming abilities, so they can be treated more seriously 😄. Regarding OpenCV which is not ML nor AI - it is possible that at some point you change it into something more sophisticated. Some marketing also does not hurt, and regardless if we like it or not (as engineers probably rather not) marketing shapes naming conventions. But smart sensors are also ok, the most important thing is you introduced it and it works! 🥇

[laurensvalk]

Hey @repkovsky - nice to hear from you!

I'm assuming you want to use the existing smart sensors but code your robot with Pybricksdev?

It works the same as in the tutorial. The only thing is that Pybricksdev disconnects when it completes so you have to connect it again each time. Press the Bluetooth button on the hub to allow the second connection again. It will blink as usual.

You can make your life easier by using the beta firmware and use Pybricksdev with USB. It will auto connect even if the hub is already connected to your phone with Bluetooth.

---

And with minor adaptations your existing tools that use AppData should still work too. Much like you did before, you can build your own interfaces if you want. And with the second connection, you'll have even more options now.

[repkovsky]

I have analysed my problem a bit deeper.

Behavior of online editor + phone:

1. Turn on hub -> Hub Bluetooth button LED blinks
2. Connect online editor (no program upload and running) -> Hub Bluetooth button LED on
3. Push Hub Bluetooth button -> Hub Bluetooth button LED blinks
4. Connect phone -> Hub Bluetooth button LED on
5. Run program in editor -> program runs

Behavior of pybricksdev + phone:

1. Turn on hub -> Hub Bluetooth button LED blinks
2. pybricksdev run ble main.py -> program runs
3. Stop debugger in VSCode -> program still runs
4. Push Hub Center button -> program stops, hub Bluetooth button LED blinks
5. Connect phone -> Hub Bluetooth button LED on
6. Push Hub Bluetooth button -> Hub Bluetooth button LED blinks
7. pybricksdev run ble main.py -> program runs

Behavior of pybricksdev + browser on the same PC as pybricksdev:

1. Turn on hub -> Hub Bluetooth button LED blinks
2. pybricksdev run ble main.py -> program runs
3. Stop debugger in VSCode -> program still runs
4. Push Hub Center button -> program stops, hub Bluetooth button LED blinks
5. Connect Bluetooth in browser on PC -> Hub Bluetooth button LED on
6. Push Hub Bluetooth button -> Hub Bluetooth button LED blinks
7. pybricksdev run ble main.py -> Characteristic c5f50003-8280-46da-89f4-6d8051e4aeef was not found!

So it looks like I cannot run two Bluetooth connections from my PC, one from browser and other from Python. Am I doing something wrong or is this simply Bluetooth limitation?

---

I am very happy for this new interface, it really simplifies communication a lot. I would like to add a few more CV/ML extensions, as my free time permits, but first I wanted to start coding using VSCode with reliable saving and AI coding assistant.

[laurensvalk]

Right, the OS on the very same computer probably wouldn't want to connect to the same device twice. This was also not something we used to support before. Chrome does allow multiple "connections", but it is not always reliable.

I would just use pybricksdev run usb main.py so you can still keep everything on one computer.

Or use your phone for the camera. Note that it does not need any program running; just open the camera view and it will pick up instructions from the running program.

I am very happy for this new interface, it really simplifies communication a lot.

Thanks!