Broken links in Search engines

[omdxp]

Describe the bug
Merged or closed contributions results in broken links in Google searches.

To Reproduce
Steps to reproduce the behavior:

1. Go to google.com
2. Search for a keyword that could be part of a contribution
3. Scroll down to the dzcode link provided by Google's search engine
4. Click on the link and you get A broken link? error (.eg: https://www.dzcode.io/contribute/manage-memory-in-a-better-way-github-2922582687)

Expected behavior
Closed contributions links should be removed from search engines as well

Screenshots

Environment:

• OS: MacOS
• Browser : Safari
• Version 18.4

[ZibanPirate]

Expected behavior
Closed contributions links should be removed from search engines as well

They will eventually be removed because such pages return a 404 code, but the problem is that Google doesn’t remove links immediately.

I can’t think of an ideal solution, but perhaps as a workaround, we can still serve these pages so users can still see the content, but also keep returning a 404 (not a 200) so Google will know they need to be deleted. Eventually, they will be deleted.

However, when do we clean up these pages after? How do we know they’re removed from Google? Should we rely on time, like deleting them after two weeks of closure?

[omdxp]

We need to have a mechanism (probably another cron job per week) that do the search in famous search engines and see if they're still shown in the results; if they're not we can delete them