Difference between revisions of "Language Administration"

From GnuCash
Jump to: navigation, search
(→‎Program: Language moves from TP to Weblate)
(→‎Program: New perl script for filling in translator credit msgstr.)
 
(77 intermediate revisions by one other user not shown)
Line 1: Line 1:
 
This page lists often used commands, sorted by components. In general there are 3 tasks, all done by admins in the repositories of the components:
 
This page lists often used commands, sorted by components. In general there are 3 tasks, all done by admins in the repositories of the components:
 
# '''Updating translations''' (<tt>MsgStr</tt>) sent by translators over different channels;  
 
# '''Updating translations''' (<tt>MsgStr</tt>) sent by translators over different channels;  
−
#: and, both requiring a fresh pot file,
+
#: and—both requiring a fresh pot file:
 
# '''Add a new language''' on request of a potential translator
 
# '''Add a new language''' on request of a potential translator
−
# '''Merging new messages''' (<tt>MsgId</tt>) from the source files.
+
# '''Merging new or modified messages''' (<tt>MsgId</tt>) from the source files.
 
Before this content was spread over several places in [[Translation]].
 
Before this content was spread over several places in [[Translation]].
 
[[Category:L10N|Language Administration]]
 
[[Category:L10N|Language Administration]]
Line 11: Line 11:
 
:* '''gettext''' has a basic set of checks, they should be run ''before each committing''.
 
:* '''gettext''' has a basic set of checks, they should be run ''before each committing''.
 
:* '''translate-toolkit''', used by [[Weblate]], has a very powerfull set, but it still needs configuration. It is ''recommended'' to install it locally and learn about it tests.
 
:* '''translate-toolkit''', used by [[Weblate]], has a very powerfull set, but it still needs configuration. It is ''recommended'' to install it locally and learn about it tests.
−
::*The only check, which [[User:Fell]] didn't find is msgfmt's --check-accelerators.
+
::*The only check, which [[User:Fell]] didn't find is msgfmt's --check-accelerators.<ref>https://care.weblate.org/#ticket/zoom/2990
 +
:Watch {{URL:GH}}WeblateOrg/weblate/pull/6451</ref>
 
::*There are still cases, were our sources are of low quality and should be fixed before requiring the corresponding test in weblate.
 
::*There are still cases, were our sources are of low quality and should be fixed before requiring the corresponding test in weblate.
 
:* There are other checks, which can be useful like '''i18nspector''' for the structure of po[t] files. Run it ''from time to time'' to verify that the headers are still state of the art.
 
:* There are other checks, which can be useful like '''i18nspector''' for the structure of po[t] files. Run it ''from time to time'' to verify that the headers are still state of the art.
 
:As we get translations from several resources, we run basic '''checks''' ''before committing''! Fix them or report them to the LAST TRANSLATOR, if possible.
 
:As we get translations from several resources, we run basic '''checks''' ''before committing''! Fix them or report them to the LAST TRANSLATOR, if possible.
−
:;Wrapping: Some editors — like Geany / PoHelper 1.36 — do not wrap the messages. To have them easier comparable, the wrapping can be done with: <syntaxhighlight lang="sh">
+
;When to Run Msgmerge:
−
msgcat -o po/${LL}.po po/${LL}.po
+
:;In Theory–All Components: Each time ''a programmer changes'' a user visible '''message''', a ''new pot'' file should be created and ''merged into all po'' files.
−
</syntaxhighlight>
+
:;Program only:
−
;Source String Changes: In theory each time ''a programmer changes'' a user visible '''message''', a ''new pot'' file should be created and ''merged into all po'' files. Sometimes '''new po''' files are requested.
+
::As our programmers usually forget the msgmerge, there are two other opportunities:
−
;After a msgmerge: Before we used weblate it was recommened to send a short mail to gnucash-devel about "Translators: <Component> changed ..." with some clues, what changed like  
+
::;New Po File Request: As it should be based on a fresh pot, that is a good reason to update also other po files, to have :them all on the same version.
−
:: "complete new download page" or  
+
::;On String Freeze: It is required.
−
:: "fixed 42 typos in the english text".
+
::Continue with [[#Msgmerge on Component Program]].
 +
:;After Msgmerge: To attract new translators consider to send an announcement to gnucash-devel or -user about "Translators: <Component> changed ..." with some clues, what changed like  
 +
::: "complete new download page" or  
 +
::: "fixed 42 typos in the english text".
 +
 
 +
;Other Considerations:
 +
:;Disable Backup on gettext commands: Because we have version control, we don't need backups of updated files. There are 2 ways:
 +
::# '''Environment variable''' <syntaxhighlight lang="sh" inline>VERSION_CONTROL=off</syntaxhighlight>
 +
::# '''Parameter''' like <syntaxhighlight lang="sh" inline>msgmerge --backup=off …</syntaxhighlight>
 +
::For now we use the parameter.
 +
::If something went wrong, just run <syntaxhighlight lang="sh" inline>git restore $FILE …</syntaxhighlight>
  
 
;Notes on command components:
 
;Notes on command components:
−
:;LANG=C: serves to get the messages in english to forward them to the translator.
+
:;LANG=C: get the command output in english to forward it on demand to the translator.
 
:;$LOCALE: replace it by the desired locale.
 
:;$LOCALE: replace it by the desired locale.
 
:;$BUILDDIR: replace it with the right path.
 
:;$BUILDDIR: replace it with the right path.
Line 30: Line 41:
 
==Components==
 
==Components==
 
===All===
 
===All===
−
In some cases weblate does not recognize the merge of its PRs: That happened actually in Program. So better verify, there are no [https://hosted.weblate.org/projects/gnucash/#repository pending] changes.
+
;When to update: Weblate—perhaps also some translators—will send '''pull requests''', but you need to check manually
 +
:* Bugzilla attachments for
 +
::* [{{BugURL}}/buglist.cgi?component=Translations&product=GnuCash&resolution=--- Program translation],
 +
::* [{{BugURL}}/buglist.cgi?component=Windows&list_id=14053&product=Packaging&resolution=--- Windows Installer],
 +
::* [{{BugURL}}/buglist.cgi?product=Documentation Documentation].
 +
:;Garbage Collection: <tt>msgmerge</tt> saves obsolete entries with prefix <q>#~ </q> at the end of the po files. That is good, in case we revert a change.
 +
::;When to clean that parts?: If the current changes do not contain reversions of previous changes, it is a good idea to do it before msgmerge.
 +
::;How to clean that parts?: In theory <syntaxhighlight lang="sh" inline>
 +
msgattrib --no-obsolete -o $FILE $FILE
 +
</syntaxhighlight> is the command, but it can fail if it finds duplicate message definitions. Then you will have to use your editor,
 +
::: search for <tt>#~ msgid "</tt> and remove it with the rest of the file.
 +
 
 +
;Before you start ''any work'' check for ''uncommitted'' changes on Weblate: Verify that there are no [https://hosted.weblate.org/projects/gnucash/#repository pending] changes. If some exist, check in which component. You can then
 +
:* delay the component until weblates next force-push (~1h) of the PR or
 +
:* force-push it yourself on that page.
 +
:Else ''merge conflicts'' can happen, resulting in '''blocking weblate''' until resolution.
 +
;Separate git pushs for weblates changes and yours: It is pretty easy to confuse weblate if you
 +
:* revert one of its commits because of bad quality or
 +
:* update the po files by msgmerge.
 +
;Before msgmerge and other changes on the po files: ''merge all pending commits'' of the component from Weblate to avoid conflicts!
 +
;Merging from:
 +
:;Weblate:
 +
::;Additional checks:
 +
:::;New Translators:
 +
::::;Full Name: If they didn't set their full name in https://hosted.weblate.org/accounts/profile/#account they are recorded as <tt>account <account@provider></tt>.  That does not look nice e.g. in translator-credit. Send a mail <Syntaxhighlight lang="text">
 +
From: <you>
 +
To: <LAST TRANSLATOR from the po file in this commit>
 +
Subject: Your GnuCash translation https://github.com/Gnucash/gnucash/{pull|commit}/…
 +
Hello,
 +
 
 +
thanks for your contribution!
 +
 
 +
I would appreciate it if you would fill in "Full Name" on the User Account tab of https://hosted.weblate.org/accounts/profile/#account because that gets used at several places like e.g. translator_credits.
 +
 
 +
You can also save me time if you connect a GitHub account there. Then you will get informed if I comment on something there and I don't have to write a separate email like now.
 +
 
 +
Regards …
 +
</Syntaxhighlight>
 +
::;Error reporting:
 +
:::;If the translator has a GitHub account: Comment on the PR <q>@<translators GH name>  <tt>msgfmt -c …</tt>
 +
:::;Else: send a mail <Syntaxhighlight lang="text">
 +
From: <you>
 +
To: <LAST TRANSLATOR from the po file in this commit>
 +
Subject: Your GnuCash translation https://github.com/Gnucash/gnucash/{pull|commit}/…
 +
Hello …
 +
 
 +
as it seems you have no Github account associated to your Weblate account I contact you this way.
 +
 
 +
At first thanks for your work.
 +
 
 +
Perhaps unrelated to your contribution the translation has a few issues:
 +
msgfmt -c …
 +
To  understand that log, 'po/mr.po:3584' means line 3584 in mr.po—you can download that format from weblate.
 +
That section  reads:
 +
 
 +
 
 +
Can you read https://wiki.gnucash.org/wiki/Translation#Special_characters_and_other_tips and try to fix them?
 +
 
 +
Regards …
 +
</Syntaxhighlight>
 +
::;Rejecting PRs: Weblate does not recognize closed (rejected) PRs and will create a new PR with the same stupid commit.
 +
:::;Current workaround: In the case all changes of a commit are wrong, commit and revert the commit.
 +
::::If only some are wrong, search them in weblate, add a comment and perhaps a flag.
 +
 
 +
:;All other sources:
 +
::;Wrapping: Some editors—like Geany / PoHelper 1.36—do not wrap the messages. This format arrived often by TP, so compare the sent file with our previous version. To have them easier comparable—less noise—the wrapping can be done with: <syntaxhighlight lang="sh">
 +
msgcat -o po/${LL}.po po/${LL}.po
 +
</syntaxhighlight>
 +
::;Still Untranslated?: List the last untranslated messages: <syntaxhighlight lang="sh">
 +
msgattrib --untranslated po/${LL}.po
 +
</syntaxhighlight>
  
 
===Glossary===
 
===Glossary===
Line 36: Line 117:
 
;Check: <Syntaxhighlight lang="sh">
 
;Check: <Syntaxhighlight lang="sh">
 
# 1 file:
 
# 1 file:
−
msgfmt -c --statistics po/glossary/$LOCALE.po
+
LANG=C msgfmt -c --statistics po/glossary/${LOCALE}.po
 
# OR all files:
 
# OR all files:
 
for i in po/glossary/*.po; do echo -n "$i:"; LANG=C msgfmt -c --statistics $i ; done
 
for i in po/glossary/*.po; do echo -n "$i:"; LANG=C msgfmt -c --statistics $i ; done
 
</syntaxhighlight>
 
</syntaxhighlight>
  
−
;Update: <Syntaxhighlight lang="sh">
+
;Update:  
 +
:;Attention: It seems weblate translators can add terms. That is the case, when ''<tt>anonymous</tt> updates all glossaries'' in one commit. Check it and—if useful—insert it together with an explanation into <tt>po/glossary/gnc-glossary.txt</tt>.
 +
::;Tip: Run the ''check for all files'' as Weblate seems not to check very well for duplicates.
 +
:After editing <tt>po/glossary/gnc-glossary.txt</tt> run <Syntaxhighlight lang="sh">
 
# 1. make pot:
 
# 1. make pot:
 
po/glossary/txt-to-pot.sh po/glossary/gnc-glossary.txt > po/glossary/gnc-glossary.pot
 
po/glossary/txt-to-pot.sh po/glossary/gnc-glossary.txt > po/glossary/gnc-glossary.pot
−
# 2. update glossaries:
+
# optionally remove obsolete messages
−
for i in po/glossary/*.po; do echo -n "$i:"; LANG=C msgmerge --previous -U $i po/glossary/gnc-glossary.pot ; done
+
for i in po/glossary/*.po; do\
 +
echo "$i:";\
 +
msgattrib --no-obsolete -o $i $i ;\
 +
done
 +
# 2. update glossaries: Set the used variables or replace them by their content!
 +
# Variant A: On the first run use existing po files as compendium
 +
cd po/glossary/
 +
for i in *.po; do\
 +
echo -n "$i:";\
 +
LANG=C msgmerge --backup=off --previous -U --compendium ../$i $i gnc-glossary.pot;\
 +
sed -i 's/Project-Id-Version: .*\\n/Project-Id-Version: GnuCash '${VERSION}'\\n/' $i;\
 +
done
 +
cd ../..
 +
# Variant B: without compendium:
 +
for i in po/glossary/*.po; do\
 +
echo -n "$i:"; LANG=C msgmerge --backup=off --previous -U $i po/glossary/gnc-glossary.pot;\
 +
sed -i 's/Project-Id-Version: .*\\n/Project-Id-Version: GnuCash '${VERSION}'\\n/' $i;\
 +
# or in  rare cases:
 +
# sed -i 's/Project-Id-Version: .*\\n/Project-Id-Version: GnuCash '${VERSION}'-pot'${POTVERSION}'\\n/' $i;\
 +
done
 
</syntaxhighlight>
 
</syntaxhighlight>
 
;New language: <Syntaxhighlight lang="sh">
 
;New language: <Syntaxhighlight lang="sh">
Line 52: Line 155:
 
./txt-to-pot.sh gnc-glossary.txt > gnc-glossary.pot
 
./txt-to-pot.sh gnc-glossary.txt > gnc-glossary.pot
 
# 2. add a new glossary:
 
# 2. add a new glossary:
−
msginit --no-translator -l $LOCALE
+
msginit --no-translator -l ${LOCALE}
 +
# Set it's VERSION:
 +
sed -i 's/Project-Id-Version: .*\\n/Project-Id-Version: GnuCash '${VERSION}'\\n/' ${LOCALE}.po
 +
# add it to the repo:
 +
git add ${LOCALE}.po
 
# DON'T FORGET: add it to CMakeLists.txt
 
# DON'T FORGET: add it to CMakeLists.txt
 
cd ../..
 
cd ../..
Line 59: Line 166:
 
===Program===
 
===Program===
 
;Assumption: Usage of '''ninja'''. If you are using make replace ''ninja'' by ''make''.
 
;Assumption: Usage of '''ninja'''. If you are using make replace ''ninja'' by ''make''.
−
;TP specific:
 
−
:# download from [{{URL:TP}}latest/gnucash/?C=M;O=D TP's latest gnucash files],
 
−
:# checks as usual.
 
−
:# on commit set the file's <tt>Last-Translator</tt> as author.
 
 
;Check: <Syntaxhighlight lang="sh">
 
;Check: <Syntaxhighlight lang="sh">
 
# 1 file:
 
# 1 file:
−
LANG=C msgfmt -c --check-accelerators="_" --statistics po/$LOCALE.po
+
LANG=C msgfmt -c --check-accelerators="_" --statistics po/${LOCALE}.po
 
# OR all files:
 
# OR all files:
 
for i in po/*.po; do echo -n "$i:"; LANG=C msgfmt -c --check-accelerators="_" --statistics $i ; done
 
for i in po/*.po; do echo -n "$i:"; LANG=C msgfmt -c --check-accelerators="_" --statistics $i ; done
Line 71: Line 174:
 
:;Errors caused by <tt>--check-accelerators="_"</tt>: should be reported to the translator only,
 
:;Errors caused by <tt>--check-accelerators="_"</tt>: should be reported to the translator only,
 
:;Other errors: should be fixed. At least set the <tt>fuzzy</tt> flag to inform the translator.
 
:;Other errors: should be fixed. At least set the <tt>fuzzy</tt> flag to inform the translator.
−
;Update: <Syntaxhighlight lang="sh">
+
;Update:
 +
:;Note: To get the <tt>c++-format</tt> flags at least '''gettext 0.22''' is required.
 +
:After [[#Check for Patch Requirement|preparation]] replace the variables by their content in: <Syntaxhighlight lang="sh">
 
# 1. make pot:
 
# 1. make pot:
 
ninja pot
 
ninja pot
 
# 2. update po files:
 
# 2. update po files:
−
for i in po/*.po; do echo -n "$i:"; LANG=C msgmerge --previous -U $i $BUILDDIR/po/gnucash.pot ; done
+
## Set the proper $VERSION or skip that line!
−
</syntaxhighlight>
+
for i in po/*.po; do echo -n "$i:";\
 +
LANG=C msgmerge --backup=off --previous -U $i $BUILDDIR/po/gnucash.pot;\
 +
msgattrib --no-obsolete -o $i $i;\
 +
sed -i 's/Project-Id-Version: .*\\n/Project-Id-Version: GnuCash '${VERSION}'\\n/' $i;\
 +
# or in  rare cases:
 +
# sed -i 's/Project-Id-Version: .*\\n/Project-Id-Version: GnuCash '${VERSION}'-pot'${POTVERSION}'\\n/' $i;\
 +
done
 +
</syntaxhighlight><ref>Project-Id-Version update from {{URL:GH}}WeblateOrg/hello/blob/main/Makefile</ref>
 
;New language: <Syntaxhighlight lang="sh">
 
;New language: <Syntaxhighlight lang="sh">
 
# 1. make pot: (tut's das?)
 
# 1. make pot: (tut's das?)
Line 83: Line 195:
 
cd po/
 
cd po/
 
msginit --no-translator -i $BUILDDIR/po/gnucash.pot -l $LOCALE
 
msginit --no-translator -i $BUILDDIR/po/gnucash.pot -l $LOCALE
 +
# Set it's VERSION:
 +
sed -i 's/Project-Id-Version: .*\\n/Project-Id-Version: GnuCash '${VERSION}'\\n/' ${LOCALE}.po
 +
# add it to the repo:
 +
git add ${LOCALE}.po
 
# DON'T FORGET: add it to CMakeLists.txt
 
# DON'T FORGET: add it to CMakeLists.txt
 
cd ..
 
cd ..
 
</syntaxhighlight>
 
</syntaxhighlight>
−
;Language moves from TP to Weblate:
+
:;Important!: Don't forget to create also the corresponding [[#Glossary]].
−
:# Verify it got marked external in [{{URL:TP}}domain/gnucash.html Translation Project: The  gnucash  textual domain].
+
;Before Release: Update translator-credit: Run
−
:# In [{{URL:GH}}Gnucash/gnucash/blob/maint/po/CMakeLists.txt#L5-L7 po/CMakeLists.txt] move it from TP_LINGUAS to GC_LINGUAS. (<Syntaxhighlight lang="sh" inline>ninja check;git commit</syntaxhighlight>)
+
<syntaxhighlight lang="sh">
−
:# In {{URL:wl set}}gnucash/#files hosted.weblate GnuCash Program Einstellungen Files] remove it from the <tt>language filter</tt>.
+
for i in po/*.po; do perl extract-translators.pl $i; done
 +
</syntaxhighlight>  
 +
and commit the resulting changes if any.
  
 
===Website===
 
===Website===
−
As it is asmall project, it uses a simple <tt>Makefile</tt>. Also check and statistics are integrated in the '''mo generation''', which again is ''required'' to be committed each time a po file changes.
+
As it is a small project, it uses a simple <tt>Makefile</tt>. Also check and statistics are integrated in the '''mo generation''', which again is ''required'' to be committed each time a po file changes.
 
;Check: <Syntaxhighlight lang="sh">
 
;Check: <Syntaxhighlight lang="sh">
 
# 1 file:
 
# 1 file:
−
make $LOCALE
+
LANG=C make $LOCALE
 +
LANGUAGE=$LOCALE make check
 
# OR all files:
 
# OR all files:
−
make mos
+
LANG=C make mos
 +
# make check in all languages …
 
</syntaxhighlight>
 
</syntaxhighlight>
−
:;Important: Commit the updated mo files to make the changes visible!
+
:;Reason for 'LANGUAGE=$LOCALE make check': Translators can break pages by inserting invalid HTML tags.
 +
:;Important:
 +
::# Commit the updated mo files to make the changes visible!
 +
::# Sometimes —misconfigured network— {{WebServer}} rejects updates from {{BuildServer}}. Verify they were accepted:
 +
::##open one of the changing pages in your browser;
 +
::## commit the update;
 +
::## refresh your browser content. The page should now contain the changes.
  
 
;Update: after messages in source files changed <Syntaxhighlight lang="sh">
 
;Update: after messages in source files changed <Syntaxhighlight lang="sh">
 +
# 0. Consider to remove obsolete messages:
 +
for i in po/*.po; do echo "$i:"; LANG=C msgattrib --no-obsolete -o $i $i ; done
 
# 1. make po/gnucash-htdocs.pot:
 
# 1. make po/gnucash-htdocs.pot:
 
make pot
 
make pot
Line 116: Line 244:
 
# DON'T FORGET: add it to Makefile
 
# DON'T FORGET: add it to Makefile
 
</syntaxhighlight>
 
</syntaxhighlight>
 +
:;Important!: Don't forget to check the existence of the corresponding [[#Glossary]].
 
;Integrate a new translation: into
 
;Integrate a new translation: into
 
:#;makefile: to <tt>languages</tt>
 
:#;makefile: to <tt>languages</tt>
Line 128: Line 257:
 
         );
 
         );
 
</syntaxhighlight>
 
</syntaxhighlight>
−
:#;Note: A few languages are disabled here, because they are '''bitrotten'''.
+
:#;Note: A few languages have been disabled here, because they were '''bitrotten'''.
  
−
== Moving a Language Between Translation Projects ==
+
==Msgmerge on Component Program==
−
;From TP to weblate:
+
===When to Msgmerge===
−
:# Ask the Last Translator, cc the language coordinator
+
Some translators complain, it is so hard to update .po files. Who and when should it be done?
−
:;If there is no response after a week:
+
:Weblate thinks, it should be done '''just in time'''.
−
::#Ask Benno to mark the language as external. The result can be seen in [https://translationproject.org/domain/gnucash.html Translation Project: The  gnucash  textual domain];
+
:But it is at least required on
−
::# in [https://github.com/Gnucash/gnucash/blob/maint/po/CMakeLists.txt#L5-L7 po/CMakeLists.txt] move it from <tt>TP_LINGUAS</tt> to <tt>GC_LINGUAS</tt>
+
::'''string freeze''',
−
::# verify, the po content is recent, else [[#Program|msgmerge]] a fresh pot.
+
::;new public branches,: On Weblate they get their own repos like <tt>https://hosted.weblate.org/git/gnucash/program-beta/</tt>
−
::# in [https://hosted.weblate.org/settings/gnucash/gnucash/#files weblate:GnuCash/Program/Einstellungen] remove it from the filter.
+
: but it doesn't hurt to do it more often.
  
−
=When to msgmerge=
+
===Check for Patch Requirement===
−
Some translators complain, it is so hard to update .po files. When should we do it?
+
Sometimes there are changes, which can be easily applied on the translation without language knowledge like removing a hardcoded linefeed or format tag.
−
;From https://translationproject.org/html/robot.html: The Translation Project robot is not allowed to update the registry with new information about maintainers, languages, or translators, nor is it allowed to process POT files. These things are still handled by hand. ''When a new POT file is being registered by a project coordinator, the registering script calls <tt>msgmerge</tt>'' when previous translations exist, so translators are notified of a PO file which is up-to-date. New POT files are announced automatically to a team's mailing list, if the team has asked for this feature.
+
<syntaxhighlight lang="sh">
−
So for $TP_LINGUAS it is done on each release - at least for the updated files.
+
# if it was not already saved:
−
:OTOH Weblate has a different philosophy: it should be done just in time.
+
# Checkout the last msgmerge commit
−
:As TP ignores our po[t] files, it can be done often.
+
ninja pot
 +
# To suppress noise in the diff:  
 +
msgcat --no-location -o ${SRCDIR}/po/gnucash-${OLDDATE}.pot ${BUILDDIR}/po/gnucash-prev.pot
 +
# End if
 +
# Checkout the current HEAD
 +
ninja pot
 +
msgcat --no-location -o ${SRCDIR}/po/gnucash-${NEWDATE}.pot ${BUILDDIR}/po/gnucash.pot
 +
</syntaxhighlight>
 +
Now you can inspect the diff and if necessary create a '''preparing patch'''.
 +
:;Note: With <syntaxhighlight lang="sh">
 +
msgcat -s --no-location -o po/gnucash[-$VERSION]-ns.pot po/gnucash[-$VERSION].pot
 +
</syntaxhighlight>
 +
you can also create the file sorted by msgid to '''improve messages'''.
 +
 
 +
===Apply it===
 +
Continue with Update in [[#Program]].
  
 
=Merging with divergent source trees like 2.6 to 2.7=
 
=Merging with divergent source trees like 2.6 to 2.7=
Line 157: Line 301:
 
make pot                                              # update the template with changes from maint
 
make pot                                              # update the template with changes from maint
 
cd ../src/po                                          # assuming src is your gnucash git repo
 
cd ../src/po                                          # assuming src is your gnucash git repo
−
msgmerge "$LL".po ../build/po/gnucash.pot -o "$LL".po # update the po of your $LL
+
msgmerge --backup=off "$LL".po ../build/po/gnucash.pot -o "$LL".po # update the po of your $LL
 
msgcat --use-first "$LL".po ../<path_to>/maint."$LL".po -o "$LL".new.po
 
msgcat --use-first "$LL".po ../<path_to>/maint."$LL".po -o "$LL".new.po
 
</syntaxhighlight>
 
</syntaxhighlight>
 
For the best result you might test different msgcat parameters. Finally
 
For the best result you might test different msgcat parameters. Finally
 
<syntaxhighlight lang="sh">mv "$LL".new.po "$LL".po</syntaxhighlight>
 
<syntaxhighlight lang="sh">mv "$LL".new.po "$LL".po</syntaxhighlight>
 +
 +
=Not PO Based Items=
 +
==Account Templates==
 +
# Ask the author/maintainer mentioned in existing files for a review
 +
# [[Account Hierarchy Template#Syntax Check]]: <SyntaxHighlight lang="sh">
 +
for i in data/accounts/${LL}/*-xea; do echo $i; xmllint --noout $i; done
 +
</SyntaxHighlight>
 +
=References=
 +
<references />

Latest revision as of 21:30, 9 December 2024

This page lists often used commands, sorted by components. In general there are 3 tasks, all done by admins in the repositories of the components:

  1. Updating translations (MsgStr) sent by translators over different channels;
    and—both requiring a fresh pot file:
  2. Add a new language on request of a potential translator
  3. Merging new or modified messages (MsgId) from the source files.

Before this content was spread over several places in Translation.

Intro

About Checks
  • gettext has a basic set of checks, they should be run before each committing.
  • translate-toolkit, used by Weblate, has a very powerfull set, but it still needs configuration. It is recommended to install it locally and learn about it tests.
  • The only check, which User:Fell didn't find is msgfmt's --check-accelerators.[1]
  • There are still cases, were our sources are of low quality and should be fixed before requiring the corresponding test in weblate.
  • There are other checks, which can be useful like i18nspector for the structure of po[t] files. Run it from time to time to verify that the headers are still state of the art.
As we get translations from several resources, we run basic checks before committing! Fix them or report them to the LAST TRANSLATOR, if possible.
When to Run Msgmerge
In Theory–All Components
Each time a programmer changes a user visible message, a new pot file should be created and merged into all po files.
Program only
As our programmers usually forget the msgmerge, there are two other opportunities:
New Po File Request
As it should be based on a fresh pot, that is a good reason to update also other po files, to have :them all on the same version.
On String Freeze
It is required.
Continue with #Msgmerge on Component Program.
After Msgmerge
To attract new translators consider to send an announcement to gnucash-devel or -user about "Translators: <Component> changed ..." with some clues, what changed like
"complete new download page" or
"fixed 42 typos in the english text".
Other Considerations
Disable Backup on gettext commands
Because we have version control, we don't need backups of updated files. There are 2 ways:
  1. Environment variable VERSION_CONTROL=off
  2. Parameter like msgmerge --backup=off …
For now we use the parameter.
If something went wrong, just run git restore $FILE …
Notes on command components
LANG=C
get the command output in english to forward it on demand to the translator.
$LOCALE
replace it by the desired locale.
$BUILDDIR
replace it with the right path.

Components

All

When to update
Weblate—perhaps also some translators—will send pull requests, but you need to check manually
  • Bugzilla attachments for
Garbage Collection
msgmerge saves obsolete entries with prefix #~ at the end of the po files. That is good, in case we revert a change.
When to clean that parts?
If the current changes do not contain reversions of previous changes, it is a good idea to do it before msgmerge.
How to clean that parts?
In theory msgattrib --no-obsolete -o $FILE $FILE is the command, but it can fail if it finds duplicate message definitions. Then you will have to use your editor,
search for #~ msgid " and remove it with the rest of the file.
Before you start any work check for uncommitted changes on Weblate
Verify that there are no pending changes. If some exist, check in which component. You can then
  • delay the component until weblates next force-push (~1h) of the PR or
  • force-push it yourself on that page.
Else merge conflicts can happen, resulting in blocking weblate until resolution.
Separate git pushs for weblates changes and yours
It is pretty easy to confuse weblate if you
  • revert one of its commits because of bad quality or
  • update the po files by msgmerge.
Before msgmerge and other changes on the po files
merge all pending commits of the component from Weblate to avoid conflicts!
Merging from
Weblate
Additional checks
New Translators
Full Name
If they didn't set their full name in https://hosted.weblate.org/accounts/profile/#account they are recorded as account <account@provider>. That does not look nice e.g. in translator-credit. Send a mail 
From: <you>
To: <LAST TRANSLATOR from the po file in this commit> 
Subject: Your GnuCash translation https://github.com/Gnucash/gnucash/{pull|commit}/…
Hello,

thanks for your contribution!

I would appreciate it if you would fill in "Full Name" on the User Account tab of https://hosted.weblate.org/accounts/profile/#account because that gets used at several places like e.g. translator_credits.

You can also save me time if you connect a GitHub account there. Then you will get informed if I comment on something there and I don't have to write a separate email like now.

Regards …
Error reporting
If the translator has a GitHub account
Comment on the PR @<translators GH name> msgfmt -c …
Else
send a mail 
From: <you>
To: <LAST TRANSLATOR from the po file in this commit> 
Subject: Your GnuCash translation https://github.com/Gnucash/gnucash/{pull|commit}/…
Hello …

as it seems you have no Github account associated to your Weblate account I contact you this way.

At first thanks for your work.

Perhaps unrelated to your contribution the translation has a few issues:
msgfmt -c …
To  understand that log, 'po/mr.po:3584' means line 3584 in mr.po—you can download that format from weblate.
That section  reads:


Can you read https://wiki.gnucash.org/wiki/Translation#Special_characters_and_other_tips and try to fix them?

Regards …
Rejecting PRs
Weblate does not recognize closed (rejected) PRs and will create a new PR with the same stupid commit.
Current workaround
In the case all changes of a commit are wrong, commit and revert the commit.
If only some are wrong, search them in weblate, add a comment and perhaps a flag.
All other sources
Wrapping
Some editors—like Geany / PoHelper 1.36—do not wrap the messages. This format arrived often by TP, so compare the sent file with our previous version. To have them easier comparable—less noise—the wrapping can be done with: 
msgcat -o po/${LL}.po po/${LL}.po
Still Untranslated?
List the last untranslated messages: 
msgattrib --untranslated po/${LL}.po

Glossary

The glossary defines the terminology used in all components.

Check
# 1 file:
LANG=C msgfmt -c --statistics po/glossary/${LOCALE}.po
# OR all files:
for i in po/glossary/*.po; do echo -n "$i:"; LANG=C msgfmt -c --statistics $i ; done
Update
Attention
It seems weblate translators can add terms. That is the case, when anonymous updates all glossaries in one commit. Check it and—if useful—insert it together with an explanation into po/glossary/gnc-glossary.txt.
Tip
Run the check for all files as Weblate seems not to check very well for duplicates.
After editing po/glossary/gnc-glossary.txt run 
# 1. make pot:
po/glossary/txt-to-pot.sh po/glossary/gnc-glossary.txt > po/glossary/gnc-glossary.pot
# optionally remove obsolete messages
for i in po/glossary/*.po; do\
 echo "$i:";\
 msgattrib --no-obsolete -o $i $i ;\
done
# 2. update glossaries: Set the used variables or replace them by their content!
# Variant A: On the first run use existing po files as compendium
cd po/glossary/
for i in *.po; do\
 echo -n "$i:";\
 LANG=C msgmerge --backup=off --previous -U --compendium ../$i $i gnc-glossary.pot;\
 sed -i 's/Project-Id-Version: .*\\n/Project-Id-Version: GnuCash '${VERSION}'\\n/' $i;\
done
cd ../..
# Variant B: without compendium:
for i in po/glossary/*.po; do\
 echo -n "$i:"; LANG=C msgmerge --backup=off --previous -U $i po/glossary/gnc-glossary.pot;\
 sed -i 's/Project-Id-Version: .*\\n/Project-Id-Version: GnuCash '${VERSION}'\\n/' $i;\
 # or in  rare cases:
 # sed -i 's/Project-Id-Version: .*\\n/Project-Id-Version: GnuCash '${VERSION}'-pot'${POTVERSION}'\\n/' $i;\
done
New language
cd po/glossary/
# 1. make pot:
./txt-to-pot.sh gnc-glossary.txt > gnc-glossary.pot
# 2. add a new glossary:
msginit --no-translator -l ${LOCALE}
# Set it's VERSION:
sed -i 's/Project-Id-Version: .*\\n/Project-Id-Version: GnuCash '${VERSION}'\\n/' ${LOCALE}.po
# add it to the repo:
git add ${LOCALE}.po
# DON'T FORGET: add it to CMakeLists.txt
cd ../..

Program

Assumption
Usage of ninja. If you are using make replace ninja by make.
Check
# 1 file:
LANG=C msgfmt -c --check-accelerators="_" --statistics po/${LOCALE}.po
# OR all files:
for i in po/*.po; do echo -n "$i:"; LANG=C msgfmt -c --check-accelerators="_" --statistics $i ; done
Errors caused by --check-accelerators="_"
should be reported to the translator only,
Other errors
should be fixed. At least set the fuzzy flag to inform the translator.
Update
Note
To get the c++-format flags at least gettext 0.22 is required.
After preparation replace the variables by their content in: 
# 1. make pot:
ninja pot
# 2. update po files:
## Set the proper $VERSION or skip that line!
for i in po/*.po; do echo -n "$i:";\
 LANG=C msgmerge --backup=off --previous -U $i $BUILDDIR/po/gnucash.pot;\
 msgattrib --no-obsolete -o $i $i;\
 sed -i 's/Project-Id-Version: .*\\n/Project-Id-Version: GnuCash '${VERSION}'\\n/' $i;\
 # or in  rare cases:
 # sed -i 's/Project-Id-Version: .*\\n/Project-Id-Version: GnuCash '${VERSION}'-pot'${POTVERSION}'\\n/' $i;\
done
[2]
New language
# 1. make pot: (tut's das?)
$BUILDDIR/ninja pot
# 2. create a new po file:
cd po/
msginit --no-translator -i $BUILDDIR/po/gnucash.pot -l $LOCALE
# Set it's VERSION:
sed -i 's/Project-Id-Version: .*\\n/Project-Id-Version: GnuCash '${VERSION}'\\n/' ${LOCALE}.po
# add it to the repo:
git add ${LOCALE}.po
# DON'T FORGET: add it to CMakeLists.txt
cd ..
Important!
Don't forget to create also the corresponding #Glossary.
Before Release
Update translator-credit: Run
for i in po/*.po; do perl extract-translators.pl $i; done

and commit the resulting changes if any.

Website

As it is a small project, it uses a simple Makefile. Also check and statistics are integrated in the mo generation, which again is required to be committed each time a po file changes.

Check
# 1 file:
LANG=C make $LOCALE
LANGUAGE=$LOCALE make check
# OR all files:
LANG=C make mos
# make check in all languages …
Reason for 'LANGUAGE=$LOCALE make check'
Translators can break pages by inserting invalid HTML tags.
Important
  1. Commit the updated mo files to make the changes visible!
  2. Sometimes —misconfigured network— www.gnucash.org/ rejects updates from code.gnucash.org. Verify they were accepted:
    1. open one of the changing pages in your browser;
    2. commit the update;
    3. refresh your browser content. The page should now contain the changes.
Update
after messages in source files changed 
# 0. Consider to remove obsolete messages:
for i in po/*.po; do echo "$i:"; LANG=C msgattrib --no-obsolete -o $i $i ; done
# 1. make po/gnucash-htdocs.pot:
make pot
# 2. update po files:
make msgmerge
Create a new po file
# 1. make pot:
make pot
# 2. add a new po file:
cd po/
msginit --no-translator -l $LOCALE
cd ..
# DON'T FORGET: add it to Makefile
Important!
Don't forget to check the existence of the corresponding #Glossary.
Integrate a new translation
into
  1. makefile
    to languages
  2. If a language is new or was bitrotten, add or enable in
    externals/header.phtml
    a line in <span id="language"> and
    lang.php
    a line in 
    # key: locale, value: lang_dir
$supported_languages = array(
        'ca_ES' => 'ca',
        :
        'C' => 'en'
        );
    Note
    A few languages have been disabled here, because they were bitrotten.

Msgmerge on Component Program

When to Msgmerge

Some translators complain, it is so hard to update .po files. Who and when should it be done?

Weblate thinks, it should be done just in time.
But it is at least required on
string freeze,
new public branches,
On Weblate they get their own repos like https://hosted.weblate.org/git/gnucash/program-beta/
but it doesn't hurt to do it more often.

Check for Patch Requirement

Sometimes there are changes, which can be easily applied on the translation without language knowledge like removing a hardcoded linefeed or format tag. 

# if it was not already saved:
# Checkout the last msgmerge commit
ninja pot
# To suppress noise in the diff: 
msgcat --no-location -o ${SRCDIR}/po/gnucash-${OLDDATE}.pot ${BUILDDIR}/po/gnucash-prev.pot
# End if
# Checkout the current HEAD
ninja pot
msgcat --no-location -o ${SRCDIR}/po/gnucash-${NEWDATE}.pot ${BUILDDIR}/po/gnucash.pot

Now you can inspect the diff and if necessary create a preparing patch.

Note
With 
msgcat -s --no-location -o po/gnucash[-$VERSION]-ns.pot po/gnucash[-$VERSION].pot

you can also create the file sorted by msgid to improve messages.

Apply it

Continue with Update in #Program.

Merging with divergent source trees like 2.6 to 2.7

The source tree in GnuCash 2.7 was thoroughly restructured, so it is now very different from 2.6. One side effect of this is that the MsgIds in our 2.7 *.po files are in a completely different order compared to 2.6. This means git merge of maint in unstable will break the 2.7 *.po files. So when merging from 2.6 to 2.7 we have to ignore all changes in the maint po files at this point. To rescue at least parts of the work, try this. 

# Set your language in LL, e.g.: export LL=de
# while you are in the maint branch save your .po file outside of the repo e.g. as maint."$LL".po
git checkout unstable
git merge maint                                       # resolve the full LL.po merge by using the unstable version of LL.po (other files can be merged as normal)
cd ../build                                           # assuming ../build is your build directory
make pot                                              # update the template with changes from maint
cd ../src/po                                          # assuming src is your gnucash git repo
msgmerge --backup=off "$LL".po ../build/po/gnucash.pot -o "$LL".po # update the po of your $LL
msgcat --use-first "$LL".po ../<path_to>/maint."$LL".po -o "$LL".new.po

For the best result you might test different msgcat parameters. Finally 

mv "$LL".new.po "$LL".po

Not PO Based Items

Account Templates

  1. Ask the author/maintainer mentioned in existing files for a review
  2. Account Hierarchy Template#Syntax Check: 
    for i in data/accounts/${LL}/*-xea; do echo $i; xmllint --noout $i; done

References

  1. ↑ https://care.weblate.org/#ticket/zoom/2990
    Watch https://github.com/WeblateOrg/weblate/pull/6451
  2. ↑ Project-Id-Version update from https://github.com/WeblateOrg/hello/blob/main/Makefile
Retrieved from "https://wiki.gnucash.org/wiki/index.php?title=Language_Administration&oldid=22257"