Difference between revisions of "Dependency Updates"

From GnuCash
Jump to: navigation, search
(General Advise: (new))
(Pull Requests at Flathub: GNOME schedule)
 
(32 intermediate revisions by 3 users not shown)
Line 1: Line 1:
If updated versions of our dependencies get released, the Gnucash-on-... projects should be modified to pull the new versions.
+
GnuCash provides the following binary releases. When feasible and after testing, they can be updated to provide the new version. While the canonical repositories are on code, those are accessible only to the core team so the links are to the Github mirrors.
==General Advise==
 
If you have not set up the respective build environment to '''test your change before submitting''', send a pull request and ask someone to test it for you instead of pushing changes.
 
==Flatpak==
 
URLs and SHAs are stored in [https://github.com/Gnucash/gnucash-on-flatpak/blob/master/templates/org.gnucash.GnuCash.json.tpl  gnucash-on-flatpak/templates/org.gnucash.GnuCash.json.tpl].
 
  
==Macos==
+
'''Keep in mind that these binary distributions are for ordinary users.''' As a general rule only '''''stable release''''' dependencies should be used for these packages. Exceptions may be made when an unstable release fixes a serious bug, but that should be discussed at length by the core team.
  
==Windows==
+
;Developers with push privileges: unless you're able to
 +
:* test the new dependency version yourself,
 +
:* the upgrade is stable, and
 +
:* you're certain that there won't be a negative impact on users
 +
:''create a pull request'' on Github so that others in the core team can test the change.
 +
 
 +
[[Dependency Sources]] has a list of URLs to check for updates.
 +
[[Category:Packaging]]
 +
 
 +
==Linux Flatpak==
 +
[https://github.com/gnucash/gnucash-on-flatpak gnucash-on-flatpak]
 +
 
 +
Flatpak is configured via <tt>json</tt> module files mostly in the [https://github.com/Gnucash/gnucash-on-flatpak/tree/master/modules modules directory]. Some components like the <tt>Runtime</tt> platform are configured in other places. See their full list in [[Release Process#Package the Release on Flathub]].
 +
:Each dependency is defined with a download URL and a SHA checksum.
 +
:Updating a dependency means updating these URL's with accompanying SHA checskums.
 +
 
 +
There are two exceptions:
 +
* the '''gnucash''' and '''gnucash-docs''' version information itself is deduced by the build script. These should never be manually entered in module files. The build script will generate a gnucash.json file in the module's base directory. This file can also be used to update the ''download server'' [https://flathub.org/apps/details/org.gnucash.GnuCash Flathub GnuCash package] which is built automatically from the ''working'' [https://github.com/flathub/org.gnucash.GnuCash gnucash flathub repository].
 +
* the list of '''cpan modules''' in [https://github.com/Gnucash/gnucash-on-flatpak/blob/master/modules/finance-quote-sources.json finance-quote-sources.json] are generated using a script from [https://github.com/flatpak/flatpak-builder-tools flatpak-builder-tools]. Below is a short description on how this can be generated.
 +
 
 +
=== How to generate finance-quote-sources.json ===
 +
* Install the required perl modules to run the generator script. This is only needed if these modules are not on your system yet. As mentioned on the flatpak-builder-tools page on Fedora that would be<Syntaxhighlight lang="sh">
 +
sudo dnf install 'perl(App::cpanminus)' 'perl(Getopt::Long::Descriptive)' 'perl(JSON::MaybeXS)' 'perl(LWP::UserAgent)' 'perl(MetaCPAN::Client)' 'perl(Pod::Simple::SimpleTree)'</Syntaxhighlight>
 +
* Run the generator script<Syntaxhighlight lang="sh">
 +
FBTDIR=<local-path-to-flatpak-builder-tools-repo>
 +
GOFDIR=<local-path-to-gnucash-on-flatpak-repo>
 +
$FBTDIR/cpan/flatpak-cpan-generator.pl -o $GOFDIR/modules/finance-quote-sources.json Date::Manip Finance::Quote</Syntaxhighlight>
 +
 
 +
;Caveat: The generator script fails to keep consistent ordering in the finance-quote dependencies. So two subsequent runs of the generator script can result in plenty of changes even if no dependency has changed at all. This is unfortunately inherent to how cpan works so there's not much we can do about it. As we normally only update finance::quote dependencies whenever there's a new version of finance::quote I don't consider this a big issue.
 +
 
 +
=== Pull Requests at Flathub ===
 +
Sometimes there are issues reported and PRs offered like "Update Gnome runtime to <version>" by the flathub maintainers. They want to get rid of obsolete versions and reduce the download burden for the users of mulptiple flatpaks — [{{URL:Gnome|release}}calendar GNOME schedule]. This updates can require updates of dependencies. Because the histories of both repositories are different, fetch them into your local instance flathub/org.gnucash.GnuCash and cherry-pick the commits into gnucash/gnucash-on-flatpak.
 +
;Alternative: <tt>git format-patch</tt> in flatpak.org and <tt>git am</tt> in gnucash-on-flatpak.
 +
 
 +
==Apple MacOS==
 +
[https://github.com/gnucash/gnucash-on-osx gnucash-on-osx]
 +
* Repos, filenames and versions are stored in [https://github.com/Gnucash/gnucash-on-osx/blob/master/modulesets/gnucash.modules gnucash-on-osx/blob/master/modulesets/gnucash.modules]
 +
 
 +
==Microsoft Windows==
 +
[https://github.com/gnucash/gnucash-on-windows gnucash-on-windows]
 +
 
 +
* Repos, filenames and versions are stored in [https://github.com/Gnucash/gnucash-on-windows/blob/master/gnucash.modules gnucash-on-windows/blob/master/gnucash.modules],
 +
* on changes of major versions, adjust the include dirs in [https://github.com/Gnucash/gnucash-on-windows/blob/master/bundle-mingw64.ps1#L117 bundle-mingw64.ps1].
 +
 
 +
==Nightlies==
 +
Nightlies are currently built for Flatpak and Windows. Ideally you would update the changes for both.
 +
;Update workflow:
 +
:Since 2021-11-14 (aqbanking-6.4.0/gwenhywfar-5.7.4) '''aqbanking.de''' displays ''sha256'' checksums, but not the full path.
 +
:#;Older versions only: They show only ''MD5'' checksums, so you have to
 +
:#:# Download the .tar.gz file,
 +
:#:# Compare <syntaxhighlight lang="sh" inline>md5sum *.tar.gz</syntaxhighlight> with the website.
 +
:#:# Copy the output from <syntaxhighlight lang="sh" inline>sha256sum *.tar.gz</syntaxhighlight> into '''Flatpak''''s ''sha''.
 +
:# Copy the ''URL'' and the ''SHA256 checksum'' into your '''Flatpak''' repository —in Redmine each file version has a unique file number directory— with
 +
:#;URL: <tt>https://aquamaniac.de/rdm/attachments/'''download/'''<serialnumber>/<package></tt>
 +
:# Copy the relevant parts into <branch ''module'' and ''version''> of <tt>gnucash.modules</tt> in your '''Windows''' repository. We maintain no  '''OSX''' nightlies!
 +
:# In the '''commit message''' add an abstract of affected modules: <tt>aqhbci</tt>, <tt>ofxdc</tt>, ... from aqbankings git.
 +
:# Create '''PR'''s on ''gnucash-on-flatpak'' and ''gnucash-on-windows'' all with the ''same commit message''.
 +
:# After the change was committed update the <tt>* version nightly</tt> templates:
 +
{| class="wikitable"
 +
|-
 +
! scope="col"| Address
 +
! scope="col"| Content
 +
! scope="col"| Stable
 +
! scope="col"| Note
 +
 
 +
|- style="vertical-align:center;"
 +
|[[Template:AqB Version nightly]] Win
 +
|{{AqB Version nightly}}
 +
|rowspan="2"|{{AqB Version}}
 +
|- style="vertical-align:top;"
 +
|[[Template:AqB Version nightly FP]]
 +
|{{AqB Version nightly FP}}
 +
|
 +
 
 +
|- style="vertical-align:top;"
 +
|[[Template:FQ Version nightly]]
 +
|{{FQ Version nightly}}
 +
|{{FQ Version}}
 +
|
 +
|}
 +
 
 +
;Bundling unstable dependencies: Enable debug symbols (-g) also on this modules. More detailed stack traces will help their developers to fix critical bugs faster.
 +
 
 +
==Continuous Integration==
 +
Don't forget to add/update your dependency in the GitHub actions if it's not a standard package in Ubuntu, arch Linux, or macOS. Note that macOS CI uses a pre-built tarball of the dependencies because there's no suitable package manager that can install binaries. The files are in
 +
;Program: {{URL:GH}}Gnucash/gnucash/tree/maint/.github/workflows and
 +
:{{URL:GH}}Gnucash/gnucash/tree/maint/util/ci
 +
;Documentation:{{URL:GH}}Gnucash/gnucash-docs/tree/maint/.github/workflows

Latest revision as of 11:32, 2 July 2024

GnuCash provides the following binary releases. When feasible and after testing, they can be updated to provide the new version. While the canonical repositories are on code, those are accessible only to the core team so the links are to the Github mirrors.

Keep in mind that these binary distributions are for ordinary users. As a general rule only stable release dependencies should be used for these packages. Exceptions may be made when an unstable release fixes a serious bug, but that should be discussed at length by the core team.

Developers with push privileges
unless you're able to
  • test the new dependency version yourself,
  • the upgrade is stable, and
  • you're certain that there won't be a negative impact on users
create a pull request on Github so that others in the core team can test the change.

Dependency Sources has a list of URLs to check for updates.

Linux Flatpak

gnucash-on-flatpak

Flatpak is configured via json module files mostly in the modules directory. Some components like the Runtime platform are configured in other places. See their full list in Release Process#Package the Release on Flathub.

Each dependency is defined with a download URL and a SHA checksum.
Updating a dependency means updating these URL's with accompanying SHA checskums.

There are two exceptions:

  • the gnucash and gnucash-docs version information itself is deduced by the build script. These should never be manually entered in module files. The build script will generate a gnucash.json file in the module's base directory. This file can also be used to update the download server Flathub GnuCash package which is built automatically from the working gnucash flathub repository.
  • the list of cpan modules in finance-quote-sources.json are generated using a script from flatpak-builder-tools. Below is a short description on how this can be generated.

How to generate finance-quote-sources.json

  • Install the required perl modules to run the generator script. This is only needed if these modules are not on your system yet. As mentioned on the flatpak-builder-tools page on Fedora that would be
    sudo dnf install 'perl(App::cpanminus)' 'perl(Getopt::Long::Descriptive)' 'perl(JSON::MaybeXS)' 'perl(LWP::UserAgent)' 'perl(MetaCPAN::Client)' 'perl(Pod::Simple::SimpleTree)'
    
  • Run the generator script
    FBTDIR=<local-path-to-flatpak-builder-tools-repo>
    GOFDIR=<local-path-to-gnucash-on-flatpak-repo>
    $FBTDIR/cpan/flatpak-cpan-generator.pl -o $GOFDIR/modules/finance-quote-sources.json Date::Manip Finance::Quote
    
Caveat
The generator script fails to keep consistent ordering in the finance-quote dependencies. So two subsequent runs of the generator script can result in plenty of changes even if no dependency has changed at all. This is unfortunately inherent to how cpan works so there's not much we can do about it. As we normally only update finance::quote dependencies whenever there's a new version of finance::quote I don't consider this a big issue.

Pull Requests at Flathub

Sometimes there are issues reported and PRs offered like "Update Gnome runtime to <version>" by the flathub maintainers. They want to get rid of obsolete versions and reduce the download burden for the users of mulptiple flatpaks — GNOME schedule. This updates can require updates of dependencies. Because the histories of both repositories are different, fetch them into your local instance flathub/org.gnucash.GnuCash and cherry-pick the commits into gnucash/gnucash-on-flatpak.

Alternative
git format-patch in flatpak.org and git am in gnucash-on-flatpak.

Apple MacOS

gnucash-on-osx

Microsoft Windows

gnucash-on-windows

Nightlies

Nightlies are currently built for Flatpak and Windows. Ideally you would update the changes for both.

Update workflow
Since 2021-11-14 (aqbanking-6.4.0/gwenhywfar-5.7.4) aqbanking.de displays sha256 checksums, but not the full path.
  1. Older versions only
    They show only MD5 checksums, so you have to
    1. Download the .tar.gz file,
    2. Compare md5sum *.tar.gz with the website.
    3. Copy the output from sha256sum *.tar.gz into Flatpak's sha.
  2. Copy the URL and the SHA256 checksum into your Flatpak repository —in Redmine each file version has a unique file number directory— with
    URL
    https://aquamaniac.de/rdm/attachments/download/<serialnumber>/<package>
  3. Copy the relevant parts into <branch module and version> of gnucash.modules in your Windows repository. We maintain no OSX nightlies!
  4. In the commit message add an abstract of affected modules: aqhbci, ofxdc, ... from aqbankings git.
  5. Create PRs on gnucash-on-flatpak and gnucash-on-windows all with the same commit message.
  6. After the change was committed update the * version nightly templates:
Address Content Stable Note
Template:AqB Version nightly Win 6.5.12beta 6.5.4
Template:AqB Version nightly FP 6.5.12beta
Template:FQ Version nightly 1.63 1.63
Bundling unstable dependencies
Enable debug symbols (-g) also on this modules. More detailed stack traces will help their developers to fix critical bugs faster.

Continuous Integration

Don't forget to add/update your dependency in the GitHub actions if it's not a standard package in Ubuntu, arch Linux, or macOS. Note that macOS CI uses a pre-built tarball of the dependencies because there's no suitable package manager that can install binaries. The files are in

Program
https://github.com/Gnucash/gnucash/tree/maint/.github/workflows and
https://github.com/Gnucash/gnucash/tree/maint/util/ci
Documentation
https://github.com/Gnucash/gnucash-docs/tree/maint/.github/workflows