[Bug] Link checking does not work during CI

[yurishkuro]

I disabled it for now in #670. When link checker was enabled, it was failing, e.g.:

bin/htmltest
htmltest started at 11:25:26 on public
========================================================================
index.html
  src attribute empty --- index.html --> 
  src attribute empty --- index.html --> 
...
  src attribute empty --- index.html --> 
  src attribute empty --- index.html --> 
========================================================================
✘✘✘ failed in 1.710136092s
10 errors in 920 documents
make: *** [Makefile:55: run-link-checker] Error 1

1. unhelpful since it does not say where in the index.html it thinks the issue is, and
2. it's not actually checking the links correctly:
   * to check internal links only it needs to run with -s flag
   * for some reason we have make target for checking external links as well which passes a config file, why isn't the same config used for internal links?
   * when I did run it with -s, I saw a bunch of "skipping external check" for URLs that are actually to jaegertracing.io, probably because Hugo generates them this way and the checker treats them as external (perhaps there's a way to specify this in the configuration)
   * meanwhile checking external links doesn't work at all as many websites appear to throttle bots

Proposal

Need to investigate if this checker is the best tool we can use (may want to check OTEL website). If it is, then maybe there are configuration options for it to not skip our own domain (not sure how that would work as the checks are meant to be against local paths). Or perhaps we could tweak our Hugo config to not generate absolute URLs and always use relative ones.

[chibuikeIg]

@yurishkuro I would like to work on this

[chahatsagarmain]

@yurishkuro i was having a look at htmltest configuration options , there seem to be no option to include urls for internal link checking . maybe we should look for other link checking tool or link checking applied in OTEL website .