Don't push a change if there haven't been any updates

[asmeurer]

This is spamming with unnecessary updates. The result is that this repo is almost always at the top of https://github.com/sympy/, and I even once noticed it at https://github.com/timeline, which I think would normally be pretty rare. The bot should push only when there is something to push.

[certik]

I absolutely agree. However I don't have time currently to implement this. It shouldn't be that difficult --- pretty much this line:

https://github.com/sympy/planet.sympy.org/blob/master/server_update#L40

has to be modified. If anyone would like to give it a shot, that'd be great.

[certik]

For now, since the summer is over, I have decreased the bot frequency from every 20 minutes to every 2 hours. Let's fix it though, that'd be much better.

[asmeurer]

Do you know how to do it? It looks to me like it recreates the whole planet from scratch, then replaces the old one, so it's not obvious to me how to tell if something significant has changed or not.

I did notice that the rss10.xml and rss11.xml files are only updated when something changes (btw, I found 2f78760189bc0306059089bc2442bdc197aa9276 as a random example of a commit that actually changed something, since this is non-trivial to do). Do you know if this is always the case? If so, then we can make the rule to only commit if one of those files was updated (or maybe just if any file other than the three that are always updated is updated). Will that always work? I don't know too much about planet.

[certik]

Good catch. That could work. I don't know much myself, but at least we have things tracked by git, so that we can experiment and improve.

Btw, I made a mistake in the cronjob, and it was executing every minute. Oops. I fixed it finally, now it should run every two hours.

[certik]

Ok, I think I have fixed this in 87a03cf. I did a few tests and things seems to be working well. I am now running the bot every 20 minutes, and no change is pushed unless rss10.xml changes, see the commit. I am closing this one.

[certik]

I have to reopen it. Aaron, your blog is causing it to flip the RSS each time

[asmeurer]

I have no idea what's going on there. Obviously, my blog is just a vanilla WordPress blog, so I don't control any of that. Maybe it's a bug in planet.

By the way, why was gh-pages forced updated? Because of the 1 minute updates?

[certik]

I forced updated it, so that instead of 50+ MB, the git repo only has 2MB. For easier manipulation.

[certik]

So I think that we need to write a simple Python parser, that would parse the rss feed before and after, and tell the number and type of posts, somehow, and decide, whether something has changed or not.

[asmeurer]

Maybe we should contact planet and see if they've already solved this problem.

[certik]

I am currently a little busy, but it shouldn't be hard to fix this issue for good with a simple Python script, checking the changes with the help of git diff.