Language Administration

From GnuCash
Revision as of 17:23, 10 September 2021 by Fell (talk | contribs) (→‎Intro: ;Program only: intro)
Jump to: navigation, search

This page lists often used commands, sorted by components. In general there are 3 tasks, all done by admins in the repositories of the components:

  1. Updating translations (MsgStr) sent by translators over different channels;
    and, both requiring a fresh pot file,
  2. Add a new language on request of a potential translator
  3. Merging new messages (MsgId) from the source files.

Before this content was spread over several places in Translation.

Intro

About Checks
  • gettext has a basic set of checks, they should be run before each committing.
  • translate-toolkit, used by Weblate, has a very powerfull set, but it still needs configuration. It is recommended to install it locally and learn about it tests.
  • The only check, which User:Fell didn't find is msgfmt's --check-accelerators.
  • There are still cases, were our sources are of low quality and should be fixed before requiring the corresponding test in weblate.
  • There are other checks, which can be useful like i18nspector for the structure of po[t] files. Run it from time to time to verify that the headers are still state of the art.
As we get translations from several resources, we run basic checks before committing! Fix them or report them to the LAST TRANSLATOR, if possible.
Wrapping
Some editors — like Geany / PoHelper 1.36 — do not wrap the messages. To have them easier comparable, the wrapping can be done with: 
msgcat -o po/${LL}.po po/${LL}.po
When to Run Msgmerge
In Theory–All Components
Each time a programmer changes a user visible message, a new pot file should be created and merged into all po files.
Program only
As our programmers usually forget the msgmerge, there are two other opportunities:
New Po File Request
As it should be based on a fresh pot, that is a good reason to update also other po files, to have :them all on the same version.
On String Freeze
It is required.
Continue with #Msgmerge on Component Program.
After Msgmerge
To attract new translators consider to send an announcement to gnucash-devel or -user about "Translators: <Component> changed ..." with some clues, what changed like
"complete new download page" or
"fixed 42 typos in the english text".
Notes on command components
LANG=C
get the command output in english to forward it on demand to the translator.
$LOCALE
replace it by the desired locale.
$BUILDDIR
replace it with the right path.

Components

All

When
Weblate will send pull requests, but for #Program you need to check TP's latest gnucash files manually.
Pending changes on Weblate
Before merging Weblate's PRs, verify that there are no pending changes. If some exist, check in which component. You can then delay the component until weblates next force-push (~1h) of the PR or force-push it yourself on that page.
Else merge conflicts can happen, resulting in blocking weblate until resolution.

Glossary

The glossary defines the terminology used in all components.

Check
# 1 file:
LANG=C msgfmt -c --statistics po/glossary/$LOCALE.po
# OR all files:
for i in po/glossary/*.po; do echo -n "$i:"; LANG=C msgfmt -c --statistics $i ; done
Update
# 1. make pot:
po/glossary/txt-to-pot.sh po/glossary/gnc-glossary.txt > po/glossary/gnc-glossary.pot
# 2. update glossaries:
for i in po/glossary/*.po; do echo -n "$i:"; LANG=C msgmerge --previous -U $i po/glossary/gnc-glossary.pot ; done
New language
cd po/glossary/
# 1. make pot:
./txt-to-pot.sh gnc-glossary.txt > gnc-glossary.pot
# 2. add a new glossary:
msginit --no-translator -l $LOCALE
# DON'T FORGET: add it to CMakeLists.txt
cd ../..

Program

Assumption
Usage of ninja. If you are using make replace ninja by make.
TP specific
  1. check regularly TP's latest gnucash files and download them,
  2. run checks as usual.
  3. on commit set the file's Last-Translator as author.
Check
# 1 file:
LANG=C msgfmt -c --check-accelerators="_" --statistics po/$LOCALE.po
# OR all files:
for i in po/*.po; do echo -n "$i:"; LANG=C msgfmt -c --check-accelerators="_" --statistics $i ; done
Errors caused by --check-accelerators="_"
should be reported to the translator only,
Other errors
should be fixed. At least set the fuzzy flag to inform the translator.
Update
# 1. make pot:
ninja pot
# 2. update po files:
for i in po/*.po; do echo -n "$i:"; LANG=C msgmerge --previous -U $i $BUILDDIR/po/gnucash.pot ; done
New language
# 1. make pot: (tut's das?)
$BUILDDIR/ninja pot
# 2. create a new po file:
cd po/
msginit --no-translator -i $BUILDDIR/po/gnucash.pot -l $LOCALE
# DON'T FORGET: add it to CMakeLists.txt
cd ..
Language moves from TP to Weblate
  1. Ask the Last Translator, cc the language coordinator
    If there is no response after a week
  1. Ask Benno to mark the language as external.
  2. Verify it got marked external in Translation Project: The gnucash textual domain.
  3. In po/CMakeLists.txt move it from TP_LINGUAS to GC_LINGUAS. (ninja check;git commit)
  4. verify, the po content is recent, else msgmerge a fresh pot.
  5. In https://hosted.weblate.org/settings/gnucash/gnucash/#files hosted.weblate GnuCash Program Einstellungen Files] remove it from the language filter.

Website

As it is a small project, it uses a simple Makefile. Also check and statistics are integrated in the mo generation, which again is required to be committed each time a po file changes.

Check
# 1 file:
LANG=C make $LOCALE
# OR all files:
LANG=C make mos
Important
Commit the updated mo files to make the changes visible!
Update
after messages in source files changed 
# 1. make po/gnucash-htdocs.pot:
make pot
# 2. update po files:
make msgmerge
Create a new po file
# 1. make pot:
make pot
# 2. add a new po file:
cd po/
msginit --no-translator -l $LOCALE
cd ..
# DON'T FORGET: add it to Makefile
Integrate a new translation
into
  1. makefile
    to languages
  2. If a language is new or was bitrotten, add or enable in
    externals/header.phtml
    a line in <span id="language"> and
    lang.php
    a line in 
    # key: locale, value: lang_dir
$supported_languages = array(
        'ca_ES' => 'ca',
        :
        'C' => 'en'
        );
    Note
    A few languages have been disabled here, because they were bitrotten.

Msgmerge on Component Program

Who and When to Msgmerge

Some translators complain, it is so hard to update .po files. Who and when should it be done?

From https://translationproject.org/html/robot.html
The Translation Project robot is not allowed to update the registry with new information about maintainers, languages, or translators, nor is it allowed to process POT files. These things are still handled by hand. When a new POT file is being registered by a project coordinator, the registering script calls msgmerge when previous translations exist, so translators are notified of a PO file which is up-to-date. New POT files are announced automatically to a team's mailing list, if the team has asked for this feature.

So for $TP_LINGUAS it is done on each release - at least for the updated files.

OTOH Weblate has a different philosophy: it should be done just in time.
As TP ignores our po[t] files, it can be done often.

=Check for Patch Requirement

Sometimes there are changes, which can be easily applied on the translation without language knowledge like removing a hardcoded linefeed or format tag. 

# if it was not already saved:
# Checkout the last msgmerge commit
ninja pot
# To suppress noise in the diff: 
msgcat --no-location -o ${SRCDIR}/po/gnucash-${OLDDATE}.pot ${BUILDDIR}/po/gnucash.pot
# End if
# Checkout the current HEAD
ninja pot
msgcat --no-location -o ${SRCDIR}/po/gnucash-${NEWDATE}.pot ${BUILDDIR}/po/gnucash.pot

Now you can inspect the diff and if necessary create a preparing patch.

Note
with 
msgcat -s --no-location …

you can also create the file sorted by msgid to improve messages.

Merging with divergent source trees like 2.6 to 2.7

The source tree in GnuCash 2.7 was thoroughly restructured, so it is now very different from 2.6. One side effect of this is that the MsgIds in our 2.7 *.po files are in a completely different order compared to 2.6. This means git merge of maint in unstable will break the 2.7 *.po files. So when merging from 2.6 to 2.7 we have to ignore all changes in the maint po files at this point. To rescue at least parts of the work, try this. 

# Set your language in LL, e.g.: export LL=de
# while you are in the maint branch save your .po file outside of the repo e.g. as maint."$LL".po
git checkout unstable
git merge maint                                       # resolve the full LL.po merge by using the unstable version of LL.po (other files can be merged as normal)
cd ../build                                           # assuming ../build is your build directory
make pot                                              # update the template with changes from maint
cd ../src/po                                          # assuming src is your gnucash git repo
msgmerge "$LL".po ../build/po/gnucash.pot -o "$LL".po # update the po of your $LL
msgcat --use-first "$LL".po ../<path_to>/maint."$LL".po -o "$LL".new.po

For the best result you might test different msgcat parameters. Finally 

mv "$LL".new.po "$LL".po

Not PO Based Items

Account Templates

  1. Ask the author/maintainer mentioned in existing files for a review
  2. Account Hierarchy Template#Syntax Check: 
    for i in data/accounts/${LL}/*-xea; do echo $i; xmllint --noout $i; done
Retrieved from "https://wiki.gnucash.org/wiki/index.php?title=Language_Administration&oldid=20236"