mkhtml: Get date from git log #2100

ninsbl · 2022-01-16T10:43:51Z

Trying to get the latest change and commit from gitlog.
Requires git and is not yet tested more broadly.
Please test. Feedback on HTML formating is very welcome!

utils/mkhtml.py

neteler · 2022-01-16T21:10:02Z

I have tested locally and get the same error as shown in the "GCC C/C++..." pipeline:

Traceback (most recent call last):
  File "/home/runner/work/grass/grass/dist.x86_64-pc-linux-gnu/utils/g.html2man.py", line 39, in main
    p.feed(line)
  File "/usr/lib/python3.8/html/parser.py", line 111, in feed
    self.goahead(0)
  File "/usr/lib/python3.8/html/parser.py", line 171, in goahead
    k = self.parse_starttag(i)
  File "/usr/lib/python3.8/html/parser.py", line 345, in parse_starttag
    self.handle_starttag(tag, attrs)
  File "/home/runner/work/grass/grass/dist.x86_64-pc-linux-gnu/utils/ghtml.py", line 281, in handle_starttag
    while not self.is_allowed(tag):
  File "/home/runner/work/grass/grass/dist.x86_64-pc-linux-gnu/utils/ghtml.py", line 277, in is_allowed
    return tag not in self.excluded and tag in allowed[self.top()]
KeyError: None

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/runner/work/grass/grass/dist.x86_64-pc-linux-gnu/utils/g.html2man.py", line 74, in <module>
    main()
  File "/home/runner/work/grass/grass/dist.x86_64-pc-linux-gnu/utils/g.html2man.py", line 43, in main
    % (infile, err.lineno, err.offset, err.msg)
AttributeError: 'KeyError' object has no attribute 'lineno'

…man page

tmszi · 2022-01-16T21:59:59Z

Trying to get the latest change and commit from gitlog. Requires git and is not yet tested more broadly. Please test. Feedback on HTML formating is very welcome!

Great idea, I looked at your code and it required a deeper analysis of the problem, I made changes in this commit 83c3d7d and I tested it, when compiling GRASS GIS the git local repo is used to get the commit and when installing the addon (via g.extension module) it is used to get the commit remote GitHub API server.

ninsbl · 2022-01-16T23:08:20Z

Seems we worked on this in parallel. Here is my approach for Addons and modules that are not part of the official addon repo:
https://github.com/ninsbl/grass/tree/mkhtml_git_addon
Your approach seems much more thorough, though, so i just pushed it to another branch, in case there is something wort transfering...

ninsbl · 2022-01-17T12:18:28Z

Seems addons that are not part of the official repo (e.g. local unversjoned addons, addons on gitlab, bitbucket, ...) need to be handled specificly:
https://github.com/OSGeo/grass/runs/4834443370?check_suite_focus=true#step:10:157
I would suggest to have a (one) fallback solution if no commit can be identified. So, for now it should be OK to not support all possible codesharing APIs (gitlab, bitbucket, gitea, ...))...
gitlab as another major platform may be added later (?)...

…rom official addons repo GitHub API

tmszi · 2022-01-17T13:13:57Z

Seems addons that are not part of the official repo (e.g. local unversjoned addons, addons on gitlab, bitbucket, ...) need to be handled specificly: https://github.com/OSGeo/grass/runs/4834443370?check_suite_focus=true#step:10:157 I would suggest to have a (one) fallback solution if no commit can be identified.

Yes you are right I fixed it.

So, for now it should be OK to not support all possible codesharing APIs (gitlab, bitbucket, gitea, ...))... gitlab as another major platform may be added later (?)...

Currently only official addons are supported, if addon is not official addon, commit and commit date in addon manual page will has unknown value.

utils/mkhtml.py

ninsbl · 2022-01-17T20:30:40Z

Now all tests pass. Nice work, @tmszi , as usual. From my point this can be merged...

tmszi · 2022-01-18T14:28:36Z

Now all tests pass. Nice work, @tmszi , as usual. From my point this can be merged...

I will have to test it as a simulation of compiling all add-ons running on our osgeo lxd container server.

neteler · 2022-01-18T19:49:26Z

I am +1 for merging, as patching our production system is always rather tricky...

tmszi · 2022-01-19T12:48:46Z

If user install addons via e.g. g.extension i.sentinel proxy="http=<value>" module and use proxy to connect to internet, mkhtml.py script have to use proxy settings too (exchange value between g.extension and mkhmtl.py script via global shell variable GRASS_PROXY). I added global shell variable GRASS_PROXY with explanation among other shell variables in manual page d1437bb.

tmszi · 2022-01-19T12:50:52Z

I successfully tested the simulation of the compilation of all add-ons on the osgeo lxd container server.

…text

neteler · 2022-01-19T16:18:59Z

I am +1 for merging, as patching our production system is always rather tricky...

FYI: I have enforced the compilation of binaries / addons on grass.osgeo.org. Hence, the improvements through this PR ~~should now be visible.~~ - nonsense as it hasn't been merged :D Sorry for the confusion.

…al page (OSGeo#2100) Git commit is getting from local repository if exists (during compilation GRASS GIS core modules, all addons compilation on our osgeo lxd container server) in others cases getting commit from remote GitHub API (official addons installed via g.extension module). In case it is not possible to obtain commit from local git repository or from remote GitHub API server, instead of commit and commit date, the time when the addon was installed is used (non official addons installed via g.extension module). Co-authored-by: Tomas Zigo <[email protected]>

tmszi · 2022-01-19T18:29:50Z

I am +1 for merging, as patching our production system is always rather tricky...

FYI: I have enforced the compilation of binaries / addons on grass.osgeo.org. Hence, the improvements through this PR ~~should now be visible.~~ - nonsense as it hasn't been merged :D Sorry for the confusion.

It's already merged.

wenzeslaus · 2022-01-19T18:44:56Z

Similarly to another change in the past, the straight backport of this breaks Travis CI because that one builds with old Python. See the old discussion here: #2070.

VERSION_NUMBER=7.8.7dev VERSION_DATE=2022 MODULE_TOPDIR=../.. \
        python3 /home/travis/build/OSGeo/grass/dist.x86_64-pc-linux-gnu/tools/mkhtml.py cairodriver > /home/travis/build/OSGeo/grass/dist.x86_64-pc-linux-gnu/docs/html/cairodriver.html
  File "/home/travis/build/OSGeo/grass/dist.x86_64-pc-linux-gnu/tools/mkhtml.py", line 74
    addons_url = f"https://github.com/OSGeo/grass-addons/tree/grass{major}/"
                                                                           ^
SyntaxError: invalid syntax

tmszi · 2022-01-19T18:59:56Z

Similarly to another change in the past, the straight backport of this breaks Travis CI because that one builds with old Python. See the old discussion here: #2070.

VERSION_NUMBER=7.8.7dev VERSION_DATE=2022 MODULE_TOPDIR=../.. \
        python3 /home/travis/build/OSGeo/grass/dist.x86_64-pc-linux-gnu/tools/mkhtml.py cairodriver > /home/travis/build/OSGeo/grass/dist.x86_64-pc-linux-gnu/docs/html/cairodriver.html
  File "/home/travis/build/OSGeo/grass/dist.x86_64-pc-linux-gnu/tools/mkhtml.py", line 74
    addons_url = f"https://github.com/OSGeo/grass-addons/tree/grass{major}/"
                                                                           ^
SyntaxError: invalid syntax

My mistake that originate during git cherry-picking.

tmszi · 2022-01-19T19:57:31Z

@neteler Before testing on the osgeo server, I still need to merge these two PRs #2110 #2111.

* Addon manual pages: remove unneeded SVN Date entry - remove already commented and unneeded SVN Date entries from all manual relate pages - selectively update section order to follow https://trac.osgeo.org/grass/wiki/Submitting/Docs#HTMLPages Note: This change brings in the disadvantage that *all* manual pages are touched in terms of editing date, which influences the recent PR OSGeo/grass#2100 and the `g.citation` efforts ongoing in #677. On the other hand, it needs to be done one day... See - discussion in #684 (comment) - in GRASS-core, see OSGeo/grass#2143 Co-authored-by: Veronica Andreo <[email protected]>

* Addon manual pages: remove unneeded SVN Date entry - remove already commented and unneeded SVN Date entries from all manual relate pages - selectively update section order to follow https://trac.osgeo.org/grass/wiki/Submitting/Docs#HTMLPages Note: This change brings in the disadvantage that *all* manual pages are touched in terms of editing date, which influences the recent PR OSGeo/grass#2100 and the `g.citation` efforts ongoing in OSGeo#677. On the other hand, it needs to be done one day... See - discussion in OSGeo#684 (comment) - in GRASS-core, see OSGeo/grass#2143 Co-authored-by: Veronica Andreo <[email protected]>

…al page (OSGeo#2100) Git commit is getting from local repository if exists (during compilation GRASS GIS core modules, all addons compilation on our osgeo lxd container server) in others cases getting commit from remote GitHub API (official addons installed via g.extension module). In case it is not possible to obtain commit from local git repository or from remote GitHub API server, instead of commit and commit date, the time when the addon was installed is used (non official addons installed via g.extension module). Co-authored-by: Tomas Zigo <[email protected]>

get date from gitlog

cef2a5e

ninsbl added enhancement New feature or request backport_needed manual Documentation related issues HTML Related code is in HTML labels Jan 16, 2022

ninsbl added this to the 8.0.1 milestone Jan 16, 2022

ninsbl requested a review from tmszi January 16, 2022 10:43

nilason reviewed Jan 16, 2022

View reviewed changes

utils/mkhtml.py Outdated Show resolved Hide resolved

utils/mkhtml.py: set core module/addon commit and commit date in the …

83c3d7d

…man page

tmszi added 2 commits January 16, 2022 23:08

Remove unnecessary var

0baa931

Set content type arg

f377223

ninsbl mentioned this pull request Jan 17, 2022

g.citation: get year from file attributes and enrich bibtex OSGeo/grass-addons#677

Closed

tmszi added 3 commits January 17, 2022 13:40

Fix condition if installed non-official addon doesn't return commit f…

51b00af

…rom official addons repo GitHub API

Set datetime_format var

1390b74

Fix extract commit date

ed144e8

ninsbl commented Jan 17, 2022

View reviewed changes

utils/mkhtml.py Outdated Show resolved Hide resolved

tmszi added 2 commits January 17, 2022 16:13

For unofficial add-ons, show the access datetime

d3f7e81

Substitute date_tag

b84d5fe

Add and explain GRASS_PROXY shell var

d1437bb

Fix output datetime format (day)

9f5ccdb

Change message level from fatal level to the warning and fix message …

78f9839

…text

tmszi approved these changes Jan 19, 2022

View reviewed changes

tmszi merged commit 432ceb7 into OSGeo:main Jan 19, 2022

tmszi removed the backport_needed label Jan 19, 2022

tmszi mentioned this pull request Jan 20, 2022

mkhtml: Fix get core modules commit from local Git repo and GitHub API #2113

Merged

echoix mentioned this pull request Jan 22, 2022

[Question] Keeping Last changed commented in manual pages OSGeo/grass-addons#684

Closed

nilason mentioned this pull request Jan 28, 2022

[Bug] Compiling version 8 fails with a no-git-repo #2137

Closed

tmszi mentioned this pull request Jan 29, 2022

mkhtml: Fix print warning/fatal message during compilation #2139

Merged

This was referenced Jan 30, 2022

manual pages: Remove unneeded SVN Date entry and selected cleanup #2143

Merged

Addon manual pages: remove unneeded SVN Date entry OSGeo/grass-addons#692

Merged

wenzeslaus changed the title ~~mkhtml.py: get date from gitlog~~ mkhtml: Get date from git log Apr 27, 2022

wenzeslaus mentioned this pull request Feb 28, 2025

docs: General todos for Markdown #5162

Open

15 tasks

Uh oh!

mkhtml: Get date from git log #2100

mkhtml: Get date from git log #2100

Uh oh!

Conversation

ninsbl commented Jan 16, 2022

Uh oh!

Uh oh!

neteler commented Jan 16, 2022

Uh oh!

tmszi commented Jan 16, 2022

Uh oh!

ninsbl commented Jan 16, 2022

Uh oh!

ninsbl commented Jan 17, 2022

Uh oh!

tmszi commented Jan 17, 2022

Uh oh!

Uh oh!

ninsbl commented Jan 17, 2022

Uh oh!

tmszi commented Jan 18, 2022

Uh oh!

neteler commented Jan 18, 2022

Uh oh!

tmszi commented Jan 19, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

tmszi commented Jan 19, 2022

Uh oh!

neteler commented Jan 19, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

tmszi commented Jan 19, 2022

Uh oh!

wenzeslaus commented Jan 19, 2022

Uh oh!

tmszi commented Jan 19, 2022

Uh oh!

tmszi commented Jan 19, 2022

Uh oh!

Uh oh!

tmszi commented Jan 19, 2022 •

edited

Loading

neteler commented Jan 19, 2022 •

edited

Loading