Revision as of 02:29, 21 December 2021

This page lists often used commands, sorted by components. In general there are 3 tasks, all done by admins in the repositories of the components:

1. Updating translations (MsgStr) sent by translators over different channels;

   and—both requiring a fresh pot file:

2. Add a new language on request of a potential translator
3. Merging new messages (MsgId) from the source files.

Before this content was spread over several places in Translation.

Intro

About Checks

• gettext has a basic set of checks, they should be run before each committing.
• translate-toolkit, used by Weblate, has a very powerfull set, but it still needs configuration. It is recommended to install it locally and learn about it tests.

• The only check, which User:Fell didn't find is msgfmt's --check-accelerators.^[1]
• There are still cases, were our sources are of low quality and should be fixed before requiring the corresponding test in weblate.

• There are other checks, which can be useful like i18nspector for the structure of po[t] files. Run it from time to time to verify that the headers are still state of the art.

As we get translations from several resources, we run basic checks before committing! Fix them or report them to the LAST TRANSLATOR, if possible.

When to Run Msgmerge

In Theory–All Components

Each time a programmer changes a user visible message, a new pot file should be created and merged into all po files.

Program only

As our programmers usually forget the msgmerge, there are two other opportunities:

New Po File Request

As it should be based on a fresh pot, that is a good reason to update also other po files, to have :them all on the same version.

On String Freeze

It is required.

Continue with #Msgmerge on Component Program.

After Msgmerge

To attract new translators consider to send an announcement to gnucash-devel or -user about "Translators: <Component> changed ..." with some clues, what changed like

"complete new download page" or

"fixed 42 typos in the english text".

Other Considerations

Disable Backup on gettext commands

Because we have version control, we don't need backups of updated files. There are 2 ways:

1. Environment variable VERSION_CONTROL=off
2. Parameter like msgmerge --backup=off …

For now we use the parameter.

If something went wrong, just run git restore <file>…

Notes on command components

LANG=C

get the command output in english to forward it on demand to the translator.

$LOCALE

replace it by the desired locale.

$BUILDDIR

replace it with the right path.

Components

All

When to update

Weblate—perhaps also some translators—will send pull requests, but you need to check manually

• Bugzilla attachments for

• Program translation,
• Windows Installer,
• Documentation.

Pending changes on Weblate

Before merging Weblate's PRs, verify that there are no pending changes. If some exist, check in which component. You can then delay the component until weblates next force-push (~1h) of the PR or force-push it yourself on that page.

Else merge conflicts can happen, resulting in blocking weblate until resolution.

Garbage Collection

msgmerge saves obsolete entries with prefix #~ at the end of the po files. That is good, in case we revert a change.

When to clean that parts?

If the current changes do not contain reversions of previous changes, it is a good idea to do it before msgmerge.

How to clean that parts?

In theory

msgattrib --no-obsolete -o $FILE $FILE

is the command, but it can fail if it finds duplicate message definitions. Then you will have to use your editor,

search for #~ msgid " and remove it with the rest of the file.

Weblate

Error reporting

Translator has GitHub account

Comment on the PR @<translators GH name> msgfmt -c …

Else

send a mail

From: <you>
To: <translator's weblate email address from the commit> 
Subject: Your translation https://github.com/Gnucash/gnucash/pull/…
Hello …

I get a few errors
msgfmt -c …
Can you read https://wiki.gnucash.org/wiki/Translation#Special_characters_and_other_tips and fix them?

All other sources

Wrapping

Some editors—like Geany / PoHelper 1.36—do not wrap the messages. This format arrives often by TP, so compare the sent file with our previous version. To have them easier comparable—less noise—the wrapping can be done with:

msgcat -o po/${LL}.po po/${LL}.po

Glossary

The glossary defines the terminology used in all components.

Check

# 1 file:
LANG=C msgfmt -c --statistics po/glossary/$LOCALE.po
# OR all files:
for i in po/glossary/*.po; do echo -n "$i:"; LANG=C msgfmt -c --statistics $i ; done

Update

Attention

It seems weblate translators can add terms. That is the case, when anonymous updates all glossaries in one commit. Check it and—if useful—insert it together with an explanation into po/glossary/gnc-glossary.txt.

Tip

Run the check for all files as Weblte seems not to check very well for duplicates.

After editing po/glossary/gnc-glossary.txt run

# 1. make pot:
po/glossary/txt-to-pot.sh po/glossary/gnc-glossary.txt > po/glossary/gnc-glossary.pot
# 2. update glossaries:
for i in po/glossary/*.po; do\
 echo -n "$i:"; LANG=C msgmerge --backup=off --previous -U $i po/glossary/gnc-glossary.pot;\
 sed -i 's/Project-Id-Version: .*\\n/Project-Id-Version: GnuCash ${VERSION}-pot${POTVERSION}\\n/' $i;\
done

New language

cd po/glossary/
# 1. make pot:
./txt-to-pot.sh gnc-glossary.txt > gnc-glossary.pot
# 2. add a new glossary:
msginit --no-translator -l $LOCALE
# DON'T FORGET: add it to CMakeLists.txt
cd ../..

Program

Assumption

Usage of ninja. If you are using make replace ninja by make.

TP specific

1. check regularly TP's latest gnucash files and download them,
2. run checks as usual.
3. on commit set the file's Last-Translator as author.

Check

# 1 file:
LANG=C msgfmt -c --check-accelerators="_" --statistics po/$LOCALE.po
# OR all files:
for i in po/*.po; do echo -n "$i:"; LANG=C msgfmt -c --check-accelerators="_" --statistics $i ; done

Errors caused by --check-accelerators="_"

should be reported to the translator only,

Other errors

should be fixed. At least set the fuzzy flag to inform the translator.

Update

after preparation

# 1. make pot:
ninja pot
# 2. update po files:
## Set the proper $VERSION or skip that line!
for i in po/*.po; do echo -n "$i:";\
 LANG=C msgmerge --backup=off --previous -U $i $BUILDDIR/po/gnucash.pot;\
 sed -i 's/Project-Id-Version: .*\\n/Project-Id-Version: GnuCash ${VERSION}-pot${POTVERSION}\\n/' $i;\
done

^[2]

New language

# 1. make pot: (tut's das?)
$BUILDDIR/ninja pot
# 2. create a new po file:
cd po/
msginit --no-translator -i $BUILDDIR/po/gnucash.pot -l $LOCALE
# DON'T FORGET: add it to CMakeLists.txt
cd ..

Before Release

Update translator-credit: Copy new contributors—without email—from the header into the msgstring of msgid "translator-credits"

Website

As it is a small project, it uses a simple Makefile. Also check and statistics are integrated in the mo generation, which again is required to be committed each time a po file changes.

Check

# 1 file:
LANG=C make $LOCALE
LANGUAGE=$LOCALE make check
# OR all files:
LANG=C make mos
# make check in all languages …

Reason for 'LANGUAGE=$LOCALE make check'

Translators can break pages by inserting invalid HTML tags.

Important

Commit the updated mo files to make the changes visible!

Update

after messages in source files changed

# 0. Consider to remove obsolete messages:
for i in po/*.po; do echo "$i:"; LANG=C msgattrib --no-obsolete -o $i $i ; done
# 1. make po/gnucash-htdocs.pot:
make pot
# 2. update po files:
make msgmerge

Create a new po file

# 1. make pot:
make pot
# 2. add a new po file:
cd po/
msginit --no-translator -l $LOCALE
cd ..
# DON'T FORGET: add it to Makefile

Integrate a new translation

into

1. makefile

   to languages

2. If a language is new or was bitrotten, add or enable in

   externals/header.phtml

   a line in <span id="language"> and

   lang.php

   a line in

   # key: locale, value: lang_dir
   $supported_languages = array(
           'ca_ES' => 'ca',
           :
           'C' => 'en'
           );
   

   Note

   A few languages have been disabled here, because they were bitrotten.

Msgmerge on Component Program

Who and When to Msgmerge

Some translators complain, it is so hard to update .po files. Who and when should it be done?

From https://translationproject.org/html/robot.html

The Translation Project robot is not allowed to update the registry with new information about maintainers, languages, or translators, nor is it allowed to process POT files. These things are still handled by hand. When a new POT file is being registered by a project coordinator, the registering script calls msgmerge when previous translations exist, so translators are notified of a PO file which is up-to-date. New POT files are announced automatically to a team's mailing list, if the team has asked for this feature.

So for $TP_LINGUAS it is done on each release - at least for the updated files.

OTOH Weblate has a different philosophy: it should be done just in time.

As TP ignores our po[t] files, it can be done often.

Check for Patch Requirement

Sometimes there are changes, which can be easily applied on the translation without language knowledge like removing a hardcoded linefeed or format tag.

# if it was not already saved:
# Checkout the last msgmerge commit
ninja pot
# To suppress noise in the diff: 
msgcat --no-location -o ${SRCDIR}/po/gnucash-${OLDDATE}.pot ${BUILDDIR}/po/gnucash.pot
# End if
# Checkout the current HEAD
ninja pot
msgcat --no-location -o ${SRCDIR}/po/gnucash-${NEWDATE}.pot ${BUILDDIR}/po/gnucash.pot

Now you can inspect the diff and if necessary create a preparing patch.

Note

with

msgcat -s --no-location …

you can also create the file sorted by msgid to improve messages.

Merging with divergent source trees like 2.6 to 2.7

The source tree in GnuCash 2.7 was thoroughly restructured, so it is now very different from 2.6. One side effect of this is that the MsgIds in our 2.7 *.po files are in a completely different order compared to 2.6. This means git merge of maint in unstable will break the 2.7 *.po files. So when merging from 2.6 to 2.7 we have to ignore all changes in the maint po files at this point. To rescue at least parts of the work, try this.

# Set your language in LL, e.g.: export LL=de
# while you are in the maint branch save your .po file outside of the repo e.g. as maint."$LL".po
git checkout unstable
git merge maint                                       # resolve the full LL.po merge by using the unstable version of LL.po (other files can be merged as normal)
cd ../build                                           # assuming ../build is your build directory
make pot                                              # update the template with changes from maint
cd ../src/po                                          # assuming src is your gnucash git repo
msgmerge --backup=off "$LL".po ../build/po/gnucash.pot -o "$LL".po # update the po of your $LL
msgcat --use-first "$LL".po ../<path_to>/maint."$LL".po -o "$LL".new.po

For the best result you might test different msgcat parameters. Finally

mv "$LL".new.po "$LL".po

Not PO Based Items

Account Templates

1. Ask the author/maintainer mentioned in existing files for a review
2. Account Hierarchy Template#Syntax Check:

   for i in data/accounts/${LL}/*-xea; do echo $i; xmllint --noout $i; done
   

References