Language Administration

This page lists often used commands, sorted by components. In general there are 3 tasks, all done by admins in the repositories of the components:

1. Updating translations (MsgStr) sent by translators over different channels;

   and—both requiring a fresh pot file:

2. Add a new language on request of a potential translator
3. Merging new or modified messages (MsgId) from the source files.

Before this content was spread over several places in Translation.

Intro

About Checks

• gettext has a basic set of checks, they should be run before each committing.
• translate-toolkit, used by Weblate, has a very powerfull set, but it still needs configuration. It is recommended to install it locally and learn about it tests.

• The only check, which User:Fell didn't find is msgfmt's --check-accelerators.
• There are still cases, were our sources are of low quality and should be fixed before requiring the corresponding test in weblate.

• There are other checks, which can be useful like i18nspector for the structure of po[t] files. Run it from time to time to verify that the headers are still state of the art.

As we get translations from several resources, we run basic checks before committing! Fix them or report them to the LAST TRANSLATOR, if possible.

When to Run Msgmerge

In Theory–All Components

Each time a programmer changes a user visible message, a new pot file should be created and merged into all po files.

Program only

As our programmers usually forget the msgmerge, there are two other opportunities:

New Po File Request

As it should be based on a fresh pot, that is a good reason to update also other po files, to have :them all on the same version.

On String Freeze

It is required.

Continue with #Msgmerge on Component Program.

After Msgmerge

To attract new translators consider to send an announcement to gnucash-devel or -user about "Translators: <Component> changed ..." with some clues, what changed like

"complete new download page" or

"fixed 42 typos in the english text".

Other Considerations

Disable Backup on gettext commands

Because we have version control, we don't need backups of updated files. There are 2 ways:

1. Environment variable VERSION_CONTROL=off
2. Parameter like msgmerge --backup=off …

For now we use the parameter.

If something went wrong, just run git restore $FILE …

Notes on command components

LANG=C

get the command output in english to forward it on demand to the translator.

$LOCALE

replace it by the desired locale.

$BUILDDIR

replace it with the right path.

Components

All

When to update

Weblate—perhaps also some translators—will send pull requests, but you need to check manually

• Bugzilla attachments for

• Program translation,
• Windows Installer,
• Documentation.

Garbage Collection

msgmerge saves obsolete entries with prefix #~ at the end of the po files. That is good, in case we revert a change.

When to clean that parts?

If the current changes do not contain reversions of previous changes, it is a good idea to do it before msgmerge.

How to clean that parts?

In theory msgattrib --no-obsolete -o $FILE $FILE is the command, but it can fail if it finds duplicate message definitions. Then you will have to use your editor,

search for #~ msgid " and remove it with the rest of the file.

Before you start any work check for uncommitted changes on Weblate

Verify that there are no pending changes. If some exist, check in which component. You can then

• delay the component until weblates next force-push (~1h) of the PR or
• force-push it yourself on that page.

Else merge conflicts can happen, resulting in blocking weblate until resolution.

Separate git pushs for weblates changes and yours

It is pretty easy to confuse weblate if you

• revert one of its commits because of bad quality or
• update the po files by msgmerge.

Before msgmerge and other changes on the po files

merge all pending commits of the component from Weblate to avoid conflicts!

Merging from

Weblate

Additional checks

New Translators

Full Name

If they didn't set their full name in https://hosted.weblate.org/accounts/profile/#account they are recorded as account <account@provider>. That does not look nice e.g. in translator-credit. Send a mail

From: <you>
To: <LAST TRANSLATOR from the po file in this commit> 
Subject: Your GnuCash translation https://github.com/Gnucash/gnucash/{pull|commit}/…
Hello,

thanks for your contribution!

I would appreciate it if you would fill in "Full Name" on the User Account tab of https://hosted.weblate.org/accounts/profile/#account because that gets used at several places like e.g. translator_credits.

You can also save me time if you connect a GitHub account there. Then you will get informed if I comment on something there and I don't have to write a separate email like now.

Regards …

Error reporting

If the translator has a GitHub account

Comment on the PR @<translators GH name> msgfmt -c …

Else

send a mail

From: <you>
To: <LAST TRANSLATOR from the po file in this commit> 
Subject: Your GnuCash translation https://github.com/Gnucash/gnucash/{pull|commit}/…
Hello …

as it seems you have no Github account associated to your Weblate account I contact you this way.

At first thanks for your work.

Perhaps unrelated to your contribution the translation has a few issues:
msgfmt -c …
To  understand that log, 'po/mr.po:3584' means line 3584 in mr.po—you can download that format from weblate.
That section  reads:


Can you read https://wiki.gnucash.org/wiki/Translation#Special_characters_and_other_tips and try to fix them?

Regards …

Rejecting PRs

Weblate does not recognize closed (rejected) PRs and will create a new PR with the same stupid commit.

Current workaround

In the case all changes of a commit are wrong, commit and revert the commit.

If only some are wrong, search them in weblate, add a comment and perhaps a flag.

All other sources

Wrapping

Some editors—like Geany / PoHelper 1.36—do not wrap the messages. This format arrived often by TP, so compare the sent file with our previous version. To have them easier comparable—less noise—the wrapping can be done with:

msgcat -o po/${LL}.po po/${LL}.po

Still Untranslated?

List the last untranslated messages:

msgattrib --untranslated po/${LL}.po

Glossary

The glossary defines the terminology used in all components.

Check

# 1 file:
LANG=C msgfmt -c --statistics po/glossary/${LOCALE}.po
# OR all files:
for i in po/glossary/*.po; do echo -n "$i:"; LANG=C msgfmt -c --statistics $i ; done

Update

Attention

It seems weblate translators can add terms. That is the case, when anonymous updates all glossaries in one commit. Check it and—if useful—insert it together with an explanation into po/glossary/gnc-glossary.txt.

Tip

Run the check for all files as Weblate seems not to check very well for duplicates.

After editing po/glossary/gnc-glossary.txt run

# 1. make pot:
po/glossary/txt-to-pot.sh po/glossary/gnc-glossary.txt > po/glossary/gnc-glossary.pot
# optionally remove obsolete messages
for i in po/glossary/*.po; do\
 echo "$i:";\
 msgattrib --no-obsolete -o $i $i ;\
done
# 2. update glossaries: Set the used variables or replace them by their content!
# Variant A: On the first run use existing po files as compendium
cd po/glossary/
for i in *.po; do\
 echo -n "$i:";\
 LANG=C msgmerge --backup=off --previous -U --compendium ../$i $i gnc-glossary.pot;\
 sed -i 's/Project-Id-Version: .*\\n/Project-Id-Version: GnuCash '${VERSION}'\\n/' $i;\
done
cd ../..
# Variant B: without compendium:
for i in po/glossary/*.po; do\
 echo -n "$i:"; LANG=C msgmerge --backup=off --previous -U $i po/glossary/gnc-glossary.pot;\
 sed -i 's/Project-Id-Version: .*\\n/Project-Id-Version: GnuCash '${VERSION}'\\n/' $i;\
 # or in  rare cases:
 # sed -i 's/Project-Id-Version: .*\\n/Project-Id-Version: GnuCash '${VERSION}'-pot'${POTVERSION}'\\n/' $i;\
done

New language

cd po/glossary/
# 1. make pot:
./txt-to-pot.sh gnc-glossary.txt > gnc-glossary.pot
# 2. add a new glossary:
msginit --no-translator -l ${LOCALE}
# Set it's VERSION:
sed -i 's/Project-Id-Version: .*\\n/Project-Id-Version: GnuCash '${VERSION}'\\n/' ${LOCALE}.po
# add it to the repo:
git add ${LOCALE}.po
# DON'T FORGET: add it to CMakeLists.txt
cd ../..

Program

Assumption

Usage of ninja. If you are using make replace ninja by make.

Check

# 1 file:
LANG=C msgfmt -c --check-accelerators="_" --statistics po/${LOCALE}.po
# OR all files:
for i in po/*.po; do echo -n "$i:"; LANG=C msgfmt -c --check-accelerators="_" --statistics $i ; done

Errors caused by --check-accelerators="_"

should be reported to the translator only,

Other errors

should be fixed. At least set the fuzzy flag to inform the translator.

Update

Note

To get the c++-format flags at least gettext 0.22 is required.

After preparation replace the variables by their content in:

# 1. make pot:
ninja pot
# 2. update po files:
## Set the proper $VERSION or skip that line!
for i in po/*.po; do echo -n "$i:";\
 LANG=C msgmerge --backup=off --previous -U $i $BUILDDIR/po/gnucash.pot;\
 msgattrib --no-obsolete -o $i $i;\
 sed -i 's/Project-Id-Version: .*\\n/Project-Id-Version: GnuCash '${VERSION}'\\n/' $i;\
 # or in  rare cases:
 # sed -i 's/Project-Id-Version: .*\\n/Project-Id-Version: GnuCash '${VERSION}'-pot'${POTVERSION}'\\n/' $i;\
done

New language

# 1. make pot: (tut's das?)
$BUILDDIR/ninja pot
# 2. create a new po file:
cd po/
msginit --no-translator -i $BUILDDIR/po/gnucash.pot -l $LOCALE
# Set it's VERSION:
sed -i 's/Project-Id-Version: .*\\n/Project-Id-Version: GnuCash '${VERSION}'\\n/' ${LOCALE}.po
# add it to the repo:
git add ${LOCALE}.po
# DON'T FORGET: add it to CMakeLists.txt
cd ..

Important!

Don't forget to create also the corresponding #Glossary.

Before Release

Update translator-credit: Run

for i in po/*.po; do perl extract-translators.pl $i; done

and commit the resulting changes if any.

Website

As it is a small project, it uses a simple Makefile. Also check and statistics are integrated in the mo generation, which again is required to be committed each time a po file changes.

Check

# 1 file:
LANG=C make $LOCALE
LANGUAGE=$LOCALE make check
# OR all files:
LANG=C make mos
# make check in all languages …

Reason for 'LANGUAGE=$LOCALE make check'

Translators can break pages by inserting invalid HTML tags.

Important

1. Commit the updated mo files to make the changes visible!
2. Sometimes —misconfigured network— www.gnucash.org/ rejects updates from code.gnucash.org. Verify they were accepted:
   1. open one of the changing pages in your browser;
   2. commit the update;
   3. refresh your browser content. The page should now contain the changes.

Update

after messages in source files changed

# 0. Consider to remove obsolete messages:
for i in po/*.po; do echo "$i:"; LANG=C msgattrib --no-obsolete -o $i $i ; done
# 1. make po/gnucash-htdocs.pot:
make pot
# 2. update po files:
make msgmerge

Create a new po file

# 1. make pot:
make pot
# 2. add a new po file:
cd po/
msginit --no-translator -l $LOCALE
cd ..
# DON'T FORGET: add it to Makefile

Important!

Don't forget to check the existence of the corresponding #Glossary.

Integrate a new translation

into

1. makefile

   to languages

2. If a language is new or was bitrotten, add or enable in

   externals/header.phtml

   a line in <span id="language"> and

   lang.php

   a line in

   # key: locale, value: lang_dir
   $supported_languages = array(
           'ca_ES' => 'ca',
           :
           'C' => 'en'
           );
   

   Note

   A few languages have been disabled here, because they were bitrotten.

Msgmerge on Component Program

When to Msgmerge

Some translators complain, it is so hard to update .po files. Who and when should it be done?

Weblate thinks, it should be done just in time.

But it is at least required on

string freeze,

new public branches,

On Weblate they get their own repos like https://hosted.weblate.org/git/gnucash/program-beta/

but it doesn't hurt to do it more often.

Check for Patch Requirement

Sometimes there are changes, which can be easily applied on the translation without language knowledge like removing a hardcoded linefeed or format tag.

# if it was not already saved:
# Checkout the last msgmerge commit
ninja pot
# To suppress noise in the diff: 
msgcat --no-location -o ${SRCDIR}/po/gnucash-${OLDDATE}.pot ${BUILDDIR}/po/gnucash-prev.pot
# End if
# Checkout the current HEAD
ninja pot
msgcat --no-location -o ${SRCDIR}/po/gnucash-${NEWDATE}.pot ${BUILDDIR}/po/gnucash.pot

Now you can inspect the diff and if necessary create a preparing patch.

Note

With

msgcat -s --no-location -o po/gnucash[-$VERSION]-ns.pot po/gnucash[-$VERSION].pot

you can also create the file sorted by msgid to improve messages.

Apply it

Continue with Update in #Program.

Merging with divergent source trees like 2.6 to 2.7

The source tree in GnuCash 2.7 was thoroughly restructured, so it is now very different from 2.6. One side effect of this is that the MsgIds in our 2.7 *.po files are in a completely different order compared to 2.6. This means git merge of maint in unstable will break the 2.7 *.po files. So when merging from 2.6 to 2.7 we have to ignore all changes in the maint po files at this point. To rescue at least parts of the work, try this.

# Set your language in LL, e.g.: export LL=de
# while you are in the maint branch save your .po file outside of the repo e.g. as maint."$LL".po
git checkout unstable
git merge maint                                       # resolve the full LL.po merge by using the unstable version of LL.po (other files can be merged as normal)
cd ../build                                           # assuming ../build is your build directory
make pot                                              # update the template with changes from maint
cd ../src/po                                          # assuming src is your gnucash git repo
msgmerge --backup=off "$LL".po ../build/po/gnucash.pot -o "$LL".po # update the po of your $LL
msgcat --use-first "$LL".po ../<path_to>/maint."$LL".po -o "$LL".new.po

For the best result you might test different msgcat parameters. Finally

mv "$LL".new.po "$LL".po

Not PO Based Items

Account Templates

1. Ask the author/maintainer mentioned in existing files for a review
2. Account Hierarchy Template#Syntax Check:

   for i in data/accounts/${LL}/*-xea; do echo $i; xmllint --noout $i; done