Easily checking for deadlinks on docs.gethue.com

Published on 17 October 2019 in Version 4 - 2 minutes read - Last modified on 26 May 2021

docs.gethue.com are getting some refreshed content continuously. In addition, a series of links not working (returning a 404) have been fixed. Here is how it was done.

First we used the muffet tool. muffet is a fast link checker crawler, very easy to use:

sudo snap install muffet

Then after booting the hugo documentation server, we point to its url. We also blacklist certain urls to avoid some noisy false positives:

muffet http://localhost:35741/ --exclude ".*releases.*" -f

And here is the output:

$ muffet http://localhost:35741/ --exclude ".*releases.*" -f
404 http://localhost:35741/administrator/configuration/editor/#connectors
404 http://localhost:35741/administrator/configuration/editor/#postgresql
dial tcp4 connect: connection refused http://localhost:5555/tasks
404 http://wiki.apache.org/hadoop/Hive/AdminManual/Configuration
dial tcp4 connect: connection refused http://localhost:16686
lookup prometheus on server misbehaving http://prometheus:9090/graph
lookup prometheus on server misbehaving http://prometheus:9090/targets
404 http://localhost:35741/administrator/configuration/dashboard
404 http://localhost:35741/administrator/configuration/editor/
404 http://localhost:35741/developer/editor/
404 http://localhost:35741/user/editor/
dialing to the given TCP address timed out https://dev.mysql.com/downloads/connector/j/
404 http://localhost:35741/administrator/configuration/external/
404 http://localhost:8000
404 https://github.com/cloudera/hue/blob/master/desktop/core/src/desktop/static/desktop/js/autocomplete/jison
404 https://github.com/cloudera/hue/tree/master/desktop/libs/metadata/catalog
404 http://localhost:35741/administrator/configuration/external/
404 http://localhost:35741/user/browsers#adls
404 http://localhost:35741/user/browsers#s3
404 http://localhost:35741/user/browsers/
404 http://docs.python.org/library/hotshot.html
404 https://en.wikipedia.org/wiki/Hue_(Software
404 https://twitter.com/gethue!
lookup developer on server misbehaving

Et voila! Then a few searches and replaces in the documentation content and we have a cleaner experience! Next action is to add the link checking to the Continuous Integration to fully automate the process and scale the developer productivity.

Any feedback or question? Feel free to comment here or on the Forum or @gethue and quick start SQL querying!


Romain from the Hue Team

comments powered by Disqus

More recent stories

03 May 2023
Discover the power of Apache Ozone using the Hue File Browser
Read More
23 January 2023
Hue 4.11 and its new dialects and features are out!
Read More