Welcome to rosettadrone Discussions!

[The1only]

✏️ Maybe it's better to have discussions outside the issues.

👋 Welcome!

I am The1Only, and ...

Just updated some of the old dependencies and moved to DJI SDK 4.16.4 (minor bug fix on DJI side)
There are still some modules that are NOT updated, as that breaks the code.

We at some point need to discuss whether to keep supporting Android 5.1 or to move to a minimum Android 7.1

Also, the Gradle itself is way outdated, but the last time I tried an upgrade it broke the video system
(that now is about to be rewritten as I understand it, it was a mess I admit).

To get started, comment below with an introduction of yourself and tell us about what you do with this community.

[The1only]

I do look forward to a plugin interface, where we can add external modules. My old AI module is kind of misplaced inside Rosettadrone.
And now I need a completely new one.

[kripper]

Just extend the Plugin class and add new required hooks in MainActivity.
Also take the chance to move the code temporality placed inside the PluginManager class into a specific Plugin class (AI and mailing stuff).

[kripper]

Sure. Actually I was thinking in moving the development issue to a discussion.

[kripper]

Hi, I'm Christopher (kripper), current maintainer of Rosetta.
I have been programming non stop since I was 8. I'm now 44 and I'm still sometimes dedicating 18 hours a day for several weeks programming my DJI Mini SE.

I decided to revive this project, merge forks and bring together all other repository owners with the goal to create an official solid MAVLink framework for DJI drones and connect with other big MAVLink speaking communities.

In the last months I have been rewriting and cleaning the code base, fixed bugs, implemented MAVLink v2 to make Rosetta compatible with MAVSDK and other existing MAVLink tools, added support for the DJI Mini series, implemented a MissionManager using virtual sticks, added a simulator to test Rosetta without connecting to a real drone, etc.

I'm currently working on AI scripts using computer vision and my focus is 100% drone automation.

I decided to work and contribute to Rosetta instead of writing my own stuff because I'm sure that on the long term my code will be useful to others and kept alive and up to date.

Why DJI? They are still the best hardware on the market considering features and price.

Why MAVLink? I don't want to be constrained to DJI in the future and also I want to benefit from the opensource ecosystem.

I'm based in Chile and Berlin.

[The1only]

Can we simulate without connecting a drone now? But with or without the hand controller?

[kripper]

Yes, without a drone.
The RC normally communicates directly with the aircraft, not with Rosetta, so the internal simulator will not be aware of RC commands.
The internal simulator is only aware of the commands sent by Rosetta (using the SDK).
Not all commands are implemented at the moment, but it is easy to extend them.

[The1only]

Which simulator is this based on? Arducopter or DJI or something you made?

[The1only]

This is really cool and something I was missing.

[The1only]

So I can takeoff using Rosettadrone software only? I am running Rosettadrone om my Mac M1 (Arm) and it seems OK but I can not takeoff ???
Sorry for all the questions, but this was something I was really missing.

[kripper]

Which simulator is this based on? Arducopter or DJI or something you made?

No, I just wrote a basic code to move the drones position and rotation. Can be extended and enhanced.

[kripper]

So I can takeoff using Rosettadrone software only? I am running Rosettadrone om my Mac M1 (Arm) and it seems OK but I can not takeoff ??? Sorry for all the questions, but this was something I was really missing.

It's fine.
I didn't implement taking off, but initialized it flying at 30 meters altitude.
Check the DummyProduct class.

[The1only]

Did you try to install Rosettadrone on a device that has newer had Rosettadrone or any other DJI software installed before?
Seems the DJI SDK login no longer shows up, and it fails and crashes?
hmm hold it, I'll get back, something is strange.
Did you test DJI CristalSky display/controller?

[kripper]

Did you try to install Rosettadrone on a device that has newer had Rosettadrone or any other DJI software installed before? Seems the DJI SDK login no longer shows up, and it fails and crashes? hmm hold it, I'll get back, something is strange. Did you test DJI CristalSky display/controller?

No, but I could test it.
I never saw this login screen you mention (only on the DJI video sample app).
Anyway, one Android app shouldn't depend on other.

I've never used any hardware apart from the DJI Mini SE.

[The1only]

OK got it, there was a bug as we still use EXTERNAL_STORAGE permissions. Fixed by testing another permission (ie. Internet):
Now it works on the latest Androids... will make a pull request.

    //Check the permissions before registering the application for android system 6.0 above.
    int permissionCheck = ContextCompat.checkSelfPermission(getApplicationContext(), android.Manifest.permission.INTERNET);
    int permissionCheck2 = ContextCompat.checkSelfPermission(getApplicationContext(), android.Manifest.permission.READ_PHONE_STATE);

    if (Build.VERSION.SDK_INT < Build.VERSION_CODES.M || (permissionCheck == 0 && permissionCheck2 == 0)) {

[The1only]

The dummy-product idea for a simulator is quite smart but will require a lot of work.
I'll see if I can get some students from a university to have a look at it.
It's a project by itself, if anyone wants to own it, let us know !!!

[kripper]

The dummy-product idea for a simulator is quite smart but will require a lot of work. I'll see if I can get some students from a university to have a look at it. It's a project by itself, if anyone wants to own it, let us know !!!

Yes, but IMO it's not important to create another full-fledged simulator (we have the DJI simulator, ArduCopter, etc).
I just wrote it to test the MAVLink 2 communication, new motion algorithms, PID controllers, waypoints mission implementation, etc. without having to connect the drone every time.
Anyway, Rosetta's simulator can be extended each time we need to test specific things.

IMO it's more important to improve the PID algorithms and to implement the gimbal motion together with the "lookAt ROI" feature to have the drone flying smoothly while looking at a region of interest at the same time.
Our friend @m4xw played a lot with this topics, but my impression is that he left most of his work incomplete.

Time ago I saw a video in youtube of someone comparing the waypoint smoothness of the 3 top alternative apps for DJI drones (Litchi, etc) and he was not satisfied with any of them.

The new virtual stick based MissionManager could be improved to use b-splines or some other kind of inter-waypoints interpolation model.

[The1only]

Ok, agree. I only need some lat,lon,alt,roll,pitch,yaw out to mavlink for a funtional test ON the AI side.

[m4xw]

Tbh it was complete for everything that I personally needed up to now :P
I will get more sorted once i am back into the drone game, summer is coming up, just need to renew my pilot license with my state.

lookAt etc i had implemented as well (worked real smooth) but i still had a bug when it was right on top of the lookAt target it would always look at, move slightly, then it sees it needs to fix lookat, that moves slightly again ending in a infinite loop.
I think i just enforced that theres a given amount of distance.

Regarding PID you HAVE to use real world testing. Sim is nice and works for most, but even if u tweak drag coefficients, wind speeds etc, its just not the real deal.

[The1only]

There are som lib files that is too large for the free github. Cleaning up will not help.

[kripper]

Hi @The1only,

I just wanted to let you know that I moved a lot of your custom code to a plugin called AI9Tek.java.
In case you need this code in the future, you will have to adapt and test it first.
I'm sorry, but it was time to move this code away from MainActivity. I know it may hurt a little, but it will be for good in the long term.
All code is still there (just moved to a plugin) and the changes can be tracked here:

• a75a635
• 48fe7df
• f3b016f

In case of a hurry, you could also checkout yesterdays version.

BTW, don't forget to cancel your payments to GitHub for the LFS service.

[The1only]

That is perfect, you are doing an amazing job. In my current research I do not use the AI interface but rather standard code. Could you tell me a bit about the synchronisation between video and tellemetry you did? This is an important taks.

[kripper]

Could you tell me a bit about the synchronisation between video and tellemetry you did?

Sure. Regarding synchronization:

1. I'm sending the timestamp and the yaw together with the uncompressed YUV frames here:

rosettadrone/app/src/main/java/sq/rogue/rosettadrone/plugins/RawVideoStreamer.java

Line 109 in d58a212

sendLong(System.nanoTime());

This is to make sure that the computer vision pose estimation uses the correct coordinate frame given by the yaw at the moment of the image.

The yaw can change quickly so forward and right coordinates could be pointing somewhere else.

2. I'm also synchronizing the time between the AI script and Rosetta to ensure that the vision frames are matched as closely as possible with the telemetry frames (if the images are too old, they shouldn't be used):

https://github.com/RosettaDrone/vision-landing-2/blob/20989d244e6a2c5b1da5d4f38ba03932ba565e98/vision_landing#L233

3. In the SmartLanding repo, I'm doing a simple linear regression to do predictions to estimate current position and velocity vectors based on past observations (images will always have some latency). This still requires revision and testing.

The idea of the SmartLanding algorithm is to implement very specific and transparent logic (simple and easy to validate). For example, I was thinking in moving the drone giving small steps initially to measure different parameters like video latency, the drone dynamics (transients, virtual stick output vs drone velocity, stability, etc) and the target behavior (is it still, moving with constant velocity, moving with constant acceleration, etc)...and then use all this parameters to move the drone as fast and certain as possible. The SmartLanding repo includes a framework for generating graphs and test algorithms (as stand alone or included inside the vision landing code working together with Rosetta). This is all work in progress. I will be working in other things now, but I released everything in case someone wants to jump in.

4. Yesterday I had a conversation with the current maintainer of the PX4's landing target estimator about using and sharing the same code base on Rosetta. They use a complex EKF that has been maintained by like 50 contributors. This could be useful on the long term as an alternative for my SmartLanding algorithm.

Related: #132