Add workflow for auto updating submodules and submitting a PR

[agorararmard]

This simple workflow will run whenever there is a push to master and with a corn job. It will update all submodules to the latest commit in their remote and submit a PR to master from the auto-created/auto-deleted branch.

• Tests pass
• Appropriate changes to README are included in PR

[umarcor]

@agorararmard, if I don't get it wrong, this is already supported by https://dependabot.com/. It used to be a (free) third-party service, but it was then bought by GitHub. It's now a built-in service and it's the tool used internally for tracking security/critical update warnings.

Therefore, I don't think a custom workflow is necessary. Maintainers just need to enable it in the settings.

[mithro]

@umarcor For various complicated reasons, we need to use a custom action here.

[umarcor]

actions/checkout has options for getting submodules too. See https://github.com/actions/checkout/#usage.

It might make sense to suggest option submodules: remote to be added upstream.

[agorararmard]

Wouldn't hurt I guess.

[mithro]

submodules: remote?

[mithro]

BTW We probably want a "Treeless clone" otherwise the worker will run out of disk space?

[umarcor]

@mithro, in this PR git submodule update --remote is executed after checkout, in order to update all submodules to their heads/master/main. That is required because actions/checkout does not support such feature. Hence, I was suggesting that maintainers of the action might want to add the feature.

[agorararmard]

@mithro: Yeah, with the current setup a Github-hosted runner will definitely run out of disk space. But would a treeless clone help here tho (I'm trying it out locally right now)?

[agorararmard]

@mithro, @umarcor: I'm afraid a treeless clone won't cut it (unless I'm doing it wrong). We will need a self-hosted runner to do this. However, since this workflow will only run on internal pushes, there are no security concerns regarding the protection of the self-hosted runner against pull-requests. In other words, this can be any machine.

After running the script to update submodules, after git clone --filter=tree:0 ....:

du -sh skywater-pdk/
53G	skywater-pdk/

[umarcor]

I would make the commit message and the title match. Maybe these params can be created in a previous step and then used here. That would allow to customize the message/body depending on which submodules were updated.

Moreover, hub is preinstalled in the virtual environments. Might be worth having a look at it, instead of dependending on a third-party Action.

[mithro]

Can we make the Commit message look similar to the ones in this pull request --> #233

[mithro]

I think this was the Python code which generated that status message,

        log = subprocess.check_output(['git', 'log', '--graph', '--pretty=short', '--decorate=short', '--decorate-refs=remotes/origin', '--shortstat', old_version+'..'+new_version], cwd=submod).decode('utf-8')

output = ['Updating submodules on {} UTC'.format(now), '']

for lib, ver, ov, nv in changes:
    if ov == nv:
        continue
    output.append('''\
 - Updating [`{lib}` {ver}](https://foss-eda-tools.googlesource.com/skywater-pdk/libs/{lib}/+/{ov}..{nv}) to {nv}'''.format(
     lib=lib, ver=ver, ov=ov, nv=nv))

output.append('')
output.append(get_submod_info())

print('\n'.join(output))

[umarcor]

In the workflow, the required change would be:

- id: generate
  run: echo 'BODY=any_text' >> $GITHUB_ENV
    
- uses: peter-evans/create-pull-request@v3
  with:
    body: ${{ env.body }}

See https://github.com/google/verible/blob/master/.github/workflows/verible-ci.yml.

Or, use hub, or pyGitHub.

[agorararmard]

@mithro, @umarcor: I changed the script that @mithro gave me a bit, and now it handles the commit message, then in the next step I'm exporting the PR body to be the same as the commit message. Can you take a look?

[google-cla]

All (the pull request submitter and all commit authors) CLAs are signed, but one or more commits were authored or co-authored by someone other than the pull request submitter.

We need to confirm that all authors are ok with their commits being contributed to this project. Please have them confirm that by leaving a comment that contains only @googlebot I consent. in this pull request.

Note to project maintainer: There may be cases where the author cannot leave a comment, or the comment is not properly detected as consent. In those cases, you can manually confirm consent of the commit author(s), and set the cla label to yes (if enabled on your project).

ℹ️ Googlers: Go here for more info.