RIOT
RIOT copied to clipboard
Published
20 hours ago •
RIOT-OS
RIOT-OS
dist/tools: check for for broken links in documentation files
Contribution description
This PR provides a small shell script that uses the wget command to check for broken links in given files.
The documentation in boards, cpu, drivers, ... contains dozens of broken links. It should be checked for such broken links from time to time. This PR provides a very small tool that can detect such broken links in files specified as parameters or in files returned as changed files from dist/tools/ci/changed_files.sh.
As the check in some cases takes a considerable amount of time, it does not seem appropriate to integrate it into the static tests.
Broken links in documentation found:
Invalid URL in boards/remote-reva/README.md:108 https://zolertia.io/product/re-mote-suite/`
Invalid URL in boards/remote-revb/README.md:114 https://zolertia.io/product/re-mote-suite/
Invalid URL in boards/msb-430/include/board.h:14 http://www.mi.fu-berlin.de/inf/groups/ag-tech/projects/Z_Finished_Projects/ScatterWeb/modules/mod_MSB-430.html
Invalid URL in boards/airfy-beacon/doc.txt:117 https://raw.githubusercontent.com/wiki/RIOT-
Invalid URL in boards/airfy-beacon/doc.txt:14 https://www.indiegogo.com/projects/airfy-beacon-make-your-smart-
Invalid URL in boards/airfy-beacon/doc.txt:19 https://raw.githubusercontent.com/wiki/RIOT-
Invalid URL in boards/airfy-beacon/doc.txt:36 http://www.100y.com.tw/pdf_file/39-Nordic-NRF51822.pdf
Invalid URL in boards/airfy-beacon/doc.txt:96 https://github.com/texane/stlink/blob/master/README
Invalid URL in boards/arduino-mega2560/doc.txt:116 http://automation.binarysage.net/?p=1515
Invalid URL in boards/arduino-mega2560/doc.txt:198 http://eng.aten.eu/products/Mobility-&-USB/USB-Converters/USB-to-Serial-
Invalid URL in boards/arduino-mega2560/doc.txt:19 http://userpage.zedat.fu-
Invalid URL in boards/arduino-mega2560/doc.txt:81 https://raw.
Invalid URL in boards/arduino-mega2560/doc.txt:89 http://userpage.zedat.fu-
Invalid URL in boards/arduino-uno/doc.txt:26 http://www.atmel.com/images/atmel-8271-8-bit-avr-microcontroller-atmega48a-48pa-88a-88pa-168a-168pa-328-328p_datasheet_complete.pdf
Invalid URL in boards/arduino-zero/doc.txt:34 http://www.atmel.com/Images/Atmel-42181-SAM-D21_Datasheet.pdf
Invalid URL in boards/avr-rss2/doc.txt:61 https://formula.brew.sh/formula/avrdude#default
Invalid URL in boards/cc2538dk/doc.txt:112 https://cdn.sparkfun.com/a
Invalid URL in boards/cc2538dk/doc.txt:46 http://processors.wiki.ti.com/index.php/CC2538_Bootloader_Backdoor
Invalid URL in boards/cc2538dk/doc.txt:61 https://e2e.ti.com/support/microcontrollers/c2000/f/171/p/359074/18434
Invalid URL in boards/cc2650stk/doc.txt:108 https://www.bluetooth.org/DocMan/handlers/DownloadDoc.ashx?doc_id=286439
Invalid URL in boards/cc2650stk/doc.txt:109 https://www.bluetooth.org/DocMan/handlers/DownloadDoc.ashx?doc_id=302735
Invalid URL in boards/cc2650stk/doc.txt:110 http://www.ti.com.cn/cn/lit/an/swra475/swra475.pdf
Invalid URL in boards/cc2650stk/doc.txt:13 http://www.ti.com/ww/en/wireless_connectivity/sensortag2015
Invalid URL in boards/cc2650stk/doc.txt:149 http://www.ti.com/lit/ug/swcu117d/swcu117d.pdf
Invalid URL in boards/cc2650stk/doc.txt:201 https://www.bluetooth.org/DocMan/handlers/DownloadDoc.ashx?doc_id=302735
Invalid URL in boards/cc2650stk/doc.txt:36 http://www.ti.com/lit/ug/swcu117d/swcu117d.pdf
Invalid URL in boards/cc2650stk/doc.txt:58 http://processors.wiki.ti.com/index.php/CC13xx_CC26xx_Tools_Overview#Debuggers
Invalid URL in boards/chronos/doc.txt:8 http://riot-os.org/images/hardware-watch-
Invalid URL in boards/common/blxxxpill/doc.txt:145 http://www.st.com/content/st_com/en/products/microcontrollers/stm32-32-bit-
Invalid URL in boards/common/blxxxpill/doc.txt:148 http://www.st.com/content/ccc/resource/technical/document/datash
Invalid URL in boards/common/blxxxpill/doc.txt:153 https://www.ebay.com/sch/i.html?_nkw=stm32f103c8
Invalid URL in boards/common/blxxxpill/doc.txt:159 https://web.archive.org/web/20190428082446/http://wiki.stm32duino.com/images/thumb/1/19/STM32_Blue_Pill_top.jpg/800px-
Invalid URL in boards/common/blxxxpill/doc.txt:18 https://camo.githubusercontent.com/8df2fb54f87527bdd57fe007352d72c1f377d08f/687474703a2f2f77696b692e73746d33326475696e6f2e636f6d2f696d616765732f7468756d622f312f31392f53544d33325f426c75655f50696c6c5f746f702e6a70672f38303070782d53544d33325f426c75655f50696c6c5f746f702e6a7067
Invalid URL in boards/common/slwstk6000b/doc.txt:93 https://www.silabs.com/Support%20Documents/TechnicalDocs/AN0004.pdf
Invalid URL in boards/derfmega128/doc.txt:12 https://www.dresden-elektronik.de/produkt/24-ghz-avr-derfmega128-22m12.html
Invalid URL in boards/esp32-mh-et-live-minikit/doc.txt:27 https://wiki.wemos.cc/products:d1_mini_shields
Invalid URL in boards/esp32-mh-et-live-minikit/doc.txt:44 https://wiki.wemos.cc/products:d1_mini_shields:micro_sd_card_shield
Invalid URL in boards/esp32-wemos-lolin-d32-pro/doc.txt:132 https://wiki.wemos.cc/_media/products:d32:sch_d32_pro_v2.0.0.pdf
Invalid URL in boards/esp32-wemos-lolin-d32-pro/doc.txt:35 https://wiki.wemos.cc/_media/products:d32:d32_pro_v2.0.0_1_16x9.jpg
Invalid URL in boards/esp32-wemos-lolin-d32-pro/doc.txt:36 https://wiki.wemos.cc/_media/products:d32:d32_pro_v2.0.0_1_16x9.jpg
Invalid URL in boards/esp8266-esp-12x/doc.txt:55 https://wiki.wemos.cc/products:retired:d1_mini_v2.2.0
Invalid URL in boards/esp8266-esp-12x/doc.txt:55 https://wiki.wemos.cc/start#d1_mini_shields
Invalid URL in boards/esp8266-esp-12x/doc.txt:71 https://wiki.wemos.cc/start#d1_boards
Invalid URL in boards/esp8266-olimex-mod/doc.txt:45 https://github.com/OLIMEX/ESP8266/raw/master/HARDWARE/MOD-WIFI-ESP8266-DEV/MOD-WIFI-ESP8266-DEV_schematic.pdf
Invalid URL in boards/firefly/doc.txt:108 http://www.zolertia.io/products
Invalid URL in boards/frdm-k64f/doc.txt:47 https://github.com/jfischer-phytec-iot/openocd/tree/wip%40phytec
Invalid URL in boards/ikea-tradfri/doc.txt:97 https://www.silabs.com/Support%20Documents/TechnicalDocs/AN0004.pdf
Invalid URL in boards/iotlab-m3/doc.txt:10 http://www.intersil.com/en/products/optoelectronics/ambient-light-sensors/light-to-digital-sensors/ISL29020.html
Invalid URL in boards/iotlab-m3/doc.txt:14 http://www.datasheet4u.com/download.php?id=683085
Invalid URL in boards/iotlab-m3/doc.txt:26 https://github.com/iot-lab/iot-
Invalid URL in boards/limifrog-v1/doc.txt:14 http://www.limifrog.io/wordpress/wp-content/uploads/2015/07/LeadingPhoto-W500px.jpg
Invalid URL in boards/limifrog-v1/doc.txt:9 http://www.limifrog.io/home-en-
Invalid URL in boards/msb-430h/doc.txt:26 http://www.mi.fu-berlin.de/inf/groups/ag-tech/projects/Z_Finished_Projects/ScatterWeb/moduleComponents/msp430f1612.pdf?1346661398
Invalid URL in boards/msb-430h/doc.txt:27 http://www.mi.fu-berlin.de/inf/groups/ag-tech/projects/Z_Finished_Projects/ScatterWeb/moduleComponents/MSP430slau049f.pdf?1346661398
Invalid URL in boards/msb-430h/doc.txt:57 http://apple.clickandbuild.com/cnb/shop/ftdichip?productID=53&op=catalogue-product_info-null&prodCategoryID=105
Invalid URL in boards/msb-430h/doc.txt:61 http://www.mi.fu-berlin.de/inf/groups/ag-tech/projects/Z_Finished_Projects/ScatterWeb/modules/mod_MSB-430H.html
Invalid URL in boards/msba2/doc.txt:29 http://www.mi.fu-berlin.de/inf/groups/ag-tech/projects/Z_Finished_Projects/ScatterWeb/modules/mod_MSB-A2.html
Invalid URL in boards/msbiot/doc.txt:104 http://www.ti.com/lit/er/swrz020d/swrz020d.pdf
Invalid URL in boards/msbiot/doc.txt:139 http://www.ti.com/lit/ds/symlink/cc3000.pdf
Invalid URL in boards/msbiot/doc.txt:141 http://www.ti.com/product/CC3000/technicaldocuments
Invalid URL in boards/msbiot/doc.txt:142 http://processors.wiki.ti.com/index.php/CC3000
Invalid URL in boards/msbiot/doc.txt:172 http://www.invensense.com/mems/gyro/documents/PS-MPU-9150A-00v4_3.pdf
Invalid URL in boards/msbiot/doc.txt:173 http://www.invensense.com/mems/gyro/documents/RM-MPU-9150A-00v4_2.pdf
Invalid URL in boards/mulle/doc.txt:10 http://www.eistec.se
Invalid URL in boards/mulle/doc.txt:12 http://www.eistec.se/mulle
Invalid URL in boards/mulle/doc.txt:6 http://eistec.github.io/images/mulle-small.jpg
Invalid URL in boards/native/doc.txt:9 https://raw.githubusercontent.com/wiki/RIOT-
Invalid URL in boards/nrf51dk/doc.txt:19 https://armmbed.github.io/DAPLink//firmware/0251_sam3u2c_mkit_dk_dongle_nrf5x_0x5000.bin
Invalid URL in boards/nrf51dongle/doc.txt:21 https://www.nordicsemi.com/var/ezwebin_site/storage/images/media/images/products/nrf51822-dk/422047-1-eng-GB/nRF51822-DK.jpg
Invalid URL in boards/nrf51dongle/doc.txt:38 http://www.freqchina.com/cn/down.asp?ID=135
Invalid URL in boards/nrf51dongle/doc.txt:39 http://www.100y.com.tw/pdf_file/39-Nordic-NRF51822.pdf
Invalid URL in boards/nucleo-f030r8/doc.txt:13 http://www.open-electronics.org/wp-
Invalid URL in boards/nucleo-f070rb/doc.txt:13 http://www.open-electronics.org/wp-
Invalid URL in boards/nucleo-f401re/doc.txt:69 https://launchpad.net/gcc-arm-
Invalid URL in boards/nucleo-f411re/doc.txt:69 https://launchpad.net/gcc-arm-
Invalid URL in boards/nucleo-f446re/doc.txt:13 http://www.open-electronics.org/wp-
Invalid URL in boards/nucleo-f446re/doc.txt:71 https://launchpad.net/gcc-arm-
Invalid URL in boards/nucleo-f446ze/doc.txt:70 https://launchpad.net/gcc-arm-
Invalid URL in boards/openmote-cc2538/doc.txt:13 https://raw.githubusercontent.com/wiki/RIOT-
Invalid URL in boards/pba-d-01-kw2x/doc.txt:87 https://ams.com/jpn/content/download/291143/1065677/file/TCS3772_Datasheet_EN_v1.pdf
Invalid URL in boards/pba-d-01-kw2x/doc.txt:88 http://www.ti.com/product/TMP006
Invalid URL in boards/phynode-kw41z/doc.txt:25 https://www.phytec.eu/product-eu/internet-of-things/iot-enablement-kit-4/
Invalid URL in boards/samd21-xpro/doc.txt:36 http://ww1.microchip.com/downloads/en/DeviceDoc/40001882A.pdf
Invalid URL in boards/seeeduino_arch-pro/doc.txt:31 http://www.nxp.com/documents/data_sheet/LPC1768_66_65_64.pdf
Invalid URL in boards/seeeduino_arch-pro/doc.txt:34 http://www.seeedstudio.com/wiki/File:Arch_Pro_V1.0_Schematic.pdf
Invalid URL in boards/slstk3401a/doc.txt:141 https://www.silabs.com/Support%20Documents/TechnicalDocs/AN0004.pdf
Invalid URL in boards/slstk3402a/doc.txt:143 https://www.silabs.com/Support%20Documents/TechnicalDocs/AN0004.pdf
Invalid URL in boards/sltb001a/doc.txt:121 https://www.silabs.com/Support%20Documents/TechnicalDocs/AN0004.pdf
Invalid URL in boards/sodaq-autonomo/doc.txt:16 http://static.webshopapp.com/shops/104792/files/050759716/sodaq-sodaq-autonomo.jpg
Invalid URL in boards/sodaq-autonomo/doc.txt:34 http://www.atmel.com/Images/Atmel-42181-SAM-D21_Datasheet.pdf
Invalid URL in boards/sodaq-sara-aff/doc.txt:11 https://support.sodaq.com/wp-content/uploads/2018/07/sodaq-sara-pins-768x862.png
Invalid URL in boards/spark-core/doc.txt:115 http://docs.spark.io/assets/images/spark-
Invalid URL in boards/spark-core/doc.txt:15 https://raw.githubusercontent.com/wiki/RIOT-
Invalid URL in boards/spark-core/doc.txt:81 https://github.com/spark/core-firmware/tree/bootloader-patch-update1
Invalid URL in boards/stk3600/doc.txt:140 https://www.silabs.com/Support%20Documents/TechnicalDocs/AN0004.pdf
Invalid URL in boards/stk3700/doc.txt:140 https://www.silabs.com/Support%20Documents/TechnicalDocs/AN0004.pdf
Invalid URL in boards/stm32f3discovery/doc.txt:100 http://www.st.com/st-web-ui/static/active/en/fragment/legal/statements/disclaimer/disclaimer_errata.pdf
Invalid URL in boards/stm32f3discovery/doc.txt:23 http://www.st.com/st-web-ui/static/active/en/fragment/product_related/rpn_information/board_photo/stm32f3discovery.jpg
Invalid URL in boards/stm32f3discovery/doc.txt:79 http://www.st.com/st-web-ui/static/active/en/fragment/legal/statements/disclaimer/disclaimer_errata.pdf
Invalid URL in boards/stm32f3discovery/doc.txt:99 http://www.st.com/st-web-ui/static/active/en/resource/technical/document/datasheet/DM00036465.pdf
Invalid URL in boards/stm32f4discovery/doc.txt:32 http://www.st.com/web/en/resource/technical/document/programming_manual/DM00023388.pdf
Invalid URL in boards/stm32f4discovery/doc.txt:91 http://www.st.com/st-web-ui/static/active/en/resource/technical/document/datasheet/DM00025467.pdf
Invalid URL in boards/telosb/doc.txt:24 http://www.mi.fu-berlin.de/inf/groups/ag-tech/projects/Z_Finished_Projects/ScatterWeb/moduleComponents/MSP430slau049f.pdf?1346661398
Invalid URL in boards/telosb/doc.txt:56 http://www.advanticsys.com/shop/mtmcm5000msp-p-14.html
Invalid URL in boards/yunjia-nrf51822/doc.txt:14 http://www.ebay.com/sch/i.html?_from=R40&_trksid=p2050601.m570.l1313.TR0.TRC0.H0.Xnrf51822&_nkw=nrf51822&_sacat=0
Invalid URL in boards/yunjia-nrf51822/doc.txt:35 http://www.freqchina.com/cn/down.asp?ID=135
Invalid URL in boards/yunjia-nrf51822/doc.txt:36 http://www.100y.com.tw/pdf_file/39-Nordic-NRF51822.pdf
Invalid URL in boards/yunjia-nrf51822/doc.txt:55 https://raw.githubusercontent.com/texane/stlink/master/49-stlinkv2.rules
Invalid URL in boards/yunjia-nrf51822/doc.txt:63 https://github.com/texane/stlink/blob/master/README
Invalid URL in boards/z1/doc.txt:27 http://zolertia.com/products/Z1_Starter_Platform
Invalid URL in cpu/msp430_common/include/stdio.h:15 http://www.cplusplus.com/reference/cstdio/
Invalid URL in drivers/sht3x/doc.txt:92 https://www.sensirion.com/fileadmin/user_upload/customers/sensirion/Dokumente/0_Datasheets/Humidity/Sensirion_Humidity_Sensors_SHT3x_Datasheet_digital.pdf
Invalid URL in drivers/ccs811/doc.txt:105 https://ams.com/documents/20143/36005/CCS811_DS000459_6-00.pdf/c7091525-c7e5-37ac-eedb-b6c6828b0dcf
Invalid URL in drivers/include/sx127x.h:49 https://semtech.my.salesforce.com/sfc/p/E0000000JelG/a/440000001NCE/v_VBhk1IolDgxwwnOpcS_vTFxPfSEPQbuneK3mWsXlU
Invalid URL in drivers/include/sx127x.h:50 https://semtech.my.salesforce.com/sfc/p/E0000000JelG/a/2R0000001OKs/Bs97dmPXeatnbdoJNVMIDaKDlQz8q1N_gxDcgqi7g2o
Invalid URL in drivers/sdcard_spi/include/sdcard_spi_internal.h:19 https://www.sdcard.org/downloads/pls/pdf/part1_500.pdf.
Testing procedure
Check an board documentation, for example:
dist/tools/urlcheck/check.sh boards/arduino-mega2560/doc.txt
Checking boards/arduino-mega2560/doc.txt ...
Invalid URL in boards/arduino-mega2560/doc.txt:19 http://userpage.zedat.fu-
Invalid URL in boards/arduino-mega2560/doc.txt:81 https://raw.
Invalid URL in boards/arduino-mega2560/doc.txt:89 http://userpage.zedat.fu-
Invalid URL in boards/arduino-mega2560/doc.txt:116 http://automation.binarysage.net/?p=1515
Invalid URL in boards/arduino-mega2560/doc.txt:198 http://eng.aten.eu/products/Mobility-&-USB/USB-Converters/USB-to-Serial-
Issues/PRs references
@gschorcht Could you run your urlcheck.sh through the shellcheck tool? There are a few quoting issues with variables pointed out by shellcheck.
I think it would also be good to add a check in there if wget is available on the system.
@gschorcht Could you run your
urlcheck.shthrough the shellcheck tool? There are a few quoting issues with variables pointed out by shellcheck.
I will do. Thanks for the hint.
One problem I have already noticed is that the shell script requires that a URL is always defined in one line. To be able to handle line breaks in URLs, we would need a more sophisticated Python script.
On the other hand, also Doxygen is not always able to deal with line breaks in URLs. For example, see section Further reading in here. Therefore we should always define URLs without line breaks, even if they exceed the maximum line length.
I'm running the tool locally without any option. There are still broken links but that can be fixed in follow-up PRs.
I find the tool a little bit too verbose. Is it possible to only print lines with broken links and not all checked files ? Maybe the verbose mode could be enabled via an env variable ?
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. If you want me to ignore this issue, please mark it with the "State: don't stale" label. Thank you for your contributions.
![stale[bot] avatar](https://avatars.githubusercontent.com/in/1724?v=4 "Published by a pub.dev verified publisher"){kind=small}
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. If you want me to ignore this issue, please mark it with the "State: don't stale" label. Thank you for your contributions.