Documentation Improvement

These instructions describe the process to change both the GnuCash "Guide" and the GnuCash "Help" manuals.

Preface and introduction -- What to expect

The recommendation of experienced programmers who work with code and related documentation is to obtain a copy of the full set of the documentation. Besides having the modules you think you will need to change, you will easily be able to update other files, if you discover additional places requiring a modification.

In addition, other persons could be making changes to parts of the documentation at the same time you are. You will need to make sure any changes others have made following your acquiring a set of the documentation are included in what you will install.

After obtaining your full set of the documentation, you will be inserting your modifications into the proper places in the set of documents. Modules in the set are formatted using XML. Later in processing the XML modules are converted to HTML versions for online easy viewing. As a documentation support person your task is to shepherd your enhancements through all stages from start to finish.

At each stage changes must be validated to assure that intended changes occur and other unexpected changes found are understood and either accepted or rejected after proper analysis.

Since your changes will be carried out by software, there is a difference determination process that identifies exactly what and where changes will be made. This process permits you to be sure that only what you intend will actually be installed.

For the sake of quality control there is a review process that you must use in order to make your change "official". The above brief description when executed results in a file of changes. These changes must be explained as to their purpose. Accompanying the explanation in words is everything that makes up the substance of the change. The term for this is "patch". Whether you introduce a major restructuring or a small tweak, every thing is a "patch". Creation of the explanation and the content of the change you are making are combined in a bugzilla event, otherwise called a "bug". In this usage bug means a statement of what is changing and why along with the content that will be the change when approved and applied to all the places needed. It does not necessarily mean a problem is being corrected.

After creating your patch and presenting it in a bugzilla bug, you inform the developers on their list and ask for their review and approval. Your email should list a brief statement of what your bug is about. You also provide the Bugzilla bug ID number so it can be found and reviewed easily.

If everything is accepted without requiring further work, your patch will be applied to the main set of documentation by a developer and you will be notified of that action.

As your first act in getting started, become very familiar with the references given in the REFERENCES section following the NOTES section below.

The Documentation Change Process -- What and how it happens

To write GnuCash documentation the following steps must be completed in the order given. [See the NOTES for each step, which are listed after all the steps!]

1. Use this command (without the quotes) exactly as written to download to your own computer a full set of documentation files.

   svn checkout http://svn.gnucash.org/repo/gnucash-docs/trunk gnucash-docs

2. Find the place in the documentation that you want to change; identify the file(s) that need to be changed to accomplish your documentation objective.
3. Draft your text changes for each module that will be affected by your update. For substantive changes to the documentation it frequently is helpful to open a text editor and develop your ideas there. See [4] in the References section for a link to the jEdit text editor, which installs in both Linux and Windows operating systems. If your changes are minor, you may skip this step and enter your changes directly into the XML modules.
4. Insert your changes into the XML file(s) affected. Make your changes fit the XML tags in the manner of the existing documentation. Adjust your XML tag structure so that the finished structure appears properly and as you intended in the HTML version of the documentation. You may have to return to this step should step 7 reveal a less than desirable arrangement of the data.
5. Validate your xml changes and the file named: “gnucash-guide.xml”. Use this command in a terminal:

   xmllint –valid –noout gnucash-guide.xml

   Repeatedly perform this step until no xml errors are found. Then move on to the next step.

6. Compare the original and changed documentation using the command below. The command must be executed from within the proper directory ("../guide/C" for the GnuCash Guide and "../help/C" for the GnuCash Help). Use this command in a terminal:

   svn diff > gc-diffs-this-date

   The result of this command is a file that lists all modules changed and the exact places in the modules where the change(s) occur. A patch should always be made against the most recent revision of the gnucash-docs project in subversion. In order to make sure your working copy is based on the most recent revision, run svn update (abbreviated, svn up). This will pull in all changes and patches that have been committed to the gnucash-docs module since you checked out your own working copy from subversion. If those changes don't conflict with your local changes, they will be merged automatically into your local working copy.

7. Build the Guide or Help file in HTML. Use this command exactly as written in a terminal (NOTE: there must be a space following the directory name 'output_html/' in the command below):

   xsltproc -o output_html/ ../../xsl/general-customization.xsl gnucash-guide.xml

8. Create the documentation change as a bugzilla bug:
   • At this URL register yourself to create an account.
   • After your account has been created, Login to the section of bugzilla reserved for GnuCash.
   • Enter your userid and password and press 'login'.
   • After logging in you are at this page and you can start the bug creation process by answering the questions on the page.
     • In the comment box explain the nature of the bug fix.
     • Attach to the bug any supporting files (differences found, new modules (if any), changed modules, and anything else that might be required to explain and support your changes.
   • When you press commit, bugzilla creates the bug and a unique id for it.
9. Note the bug number and list yourself as wanting to be notified any time there is an update to the bug. Monitor it until it is confirmed and installed to the trunk.

NOTES for each of the above steps

Step 1 - svn checkout

The terminal command will cause the documentation to be copied to your computer under the name you specify in the command. Executing the command assumes that subversion has previously been installed successfully on your computer. If the system finds that subversion is not installed, it ignores your request and replies with the correct command to download and install the package.

If you need to install svn, make note of the module(s) that it tells you should be deleted. Also, note the command it provides to delete such module(s).Next install subversion with the install command given. Note any instructions given during the install process and carry them out. Finally, use the instruction first noted that tells you what module(s) to remove and carry out the remove instruction.

In the checkout command "gnucash-docs" is the directory name where the checkout process places the full set of documentation modules. You can use that name or some other. I like to include in the directory name the date of download, thus: gc-docs-09302010. Use what is most helpful to you. Remember that the bash shell is case sensitive when typing commands and providing names.

By the above svn checkout command you are making changes in a local subversion working copy of the gnucash-docs library of modules.

Since you are making changes in a local working copy, subversion already keeps track of changes you make. It's just a matter of letting subversion output those changes in a easily understood patch format. For your patch to be useful, the command finding differences (discussed below) should be executed within as large a context as possible. To be sure of that context carry out this command:

Change directory to the top-level directory of the gnucash-docs project. Using my directory name given above to illustrate: cd gc-docs-09302010.

svn checkout

generates a very long listing of modules. At the end it lists a revision number, e.g., "checked out revision 19612". The revision number is important for reference.

Avoid merge conflicts as much as possible. Run svn update regularly, so repository changes are pulled in frequently and in sizes.

Reference [1] (in the next section) is a very good introduction to subversion and how to perform basic tasks.

Step 2 - find update location

This means you have to read the documentation to find exactly where the error is or the best place to insert the additional information that improves the understanding of how the feature works or what its purpose is in the software.

Experienced developers instruct that you should focus first on the modules in either of these two directories (found in the step 1 downloaded files): gnucash-docs/help/C or gnucash-docs/guide/C.

Trying to open each file in each of those directories will show whether there are any errors in the modules you downloaded. If you find errors, ask a more experienced person whether these should first be corrected before proceeding with your updates. You can do that by email to gnucash-devel@gnucash.org. If you choose this method, be sure to be a subscriber to that list in order to receive a prompt answer. If you are not a subscriber, then your email will wait for the list manager to find the time to address emails received from non-subscribers.

It will be useful to have either a printed copy or a PDF copy [3] of the documentation available for reference. The PDF is often useful, because it allows using FIND (ctrl-F) to search for key words. This can be important to assure yourself that you have covered the existing places in the documentation where the issue you are interested in has already had a mention or treatment.

Step 3 - draft your update in a text file

If your changes are few and easily formulated, then you should be able to skip this step and proceed to #Step 4 - place the draft changes in XML files.

If your update is adding something new or reworking existing text rather extensively, it will be very helpful to translate your ideas into words by using a text file not part of the existing documentation as a scratch pad to hone your expression of your ideas.

When your ideas are clear and expressed as you think appropriate, then proceed to #Step 4 - place the draft changes in XML files.

Step 4 - place the draft changes in XML files

Because the source documents are saved in XML code, all changes need to be added to the source modules in that manner. Small changes can be made directly into the XML file itself. Larger and extensive changes may first be prepared in a text editor and later inserted into the module(s) in their proper places. Resources for XML are listed in the References section for this step.

Review the inserted and corrected text to verify that it is presented within the proper XML tags, using existing tags as a guide. To read about XML and its tags, entities, and attributes, see the References section for links to resources.

If text exceeds the vertical text border guides, you should not be alarmed that text outside the vertical border guides would be lost. These border guides show up when the XML module is opened, but are length of line indicators and not warnings that text should be moved within the guides.

Apply the modules structural concepts to your own text after you have cut and pasted it into the XML file. This is done by using the various XML tags in the existing text: chapter, segment, sect1, sect 2, orderedlist, list item, para (to name just a few) and the corresponding closing tags. See the references below these step notes for information about XML and docbook.

Note

if your update adds new modules to the full set of documentation, you should review all modules in the directory in which you are working (gnucash-docs/help/C or gnucash-docs/guide/C) to determine what changes,if any, need to be made to modules outside your original assessment in step 2.

Step 5 - validate xml changes

Execute the 'xmllint' command in a terminal after changing directory to the place where the changed modules are located. For example, if the module you are changing is in the guide/C directory and you had downloaded the documentation files to a directory called GC-docs, then you would cd to the directory GC-docs/guide/C. It is when you are inside that directory that you execute the command.

The file gnucash-guide.xml is found in the C subdirectory of GC-docs/guide/C. Similarly, the file gnucash-help.xml would be in GC-docs/help/C.

If your module(s) are free of XML errors, then the command returns a blank screen (if running in a terminal) or a blank file (if redirecting output to a file.

XML errors must be removed before completing the remaining steps. If you are not able to determine the source of the error, then help could be available via the developers list (gnucash-devel@gnucash.org). If asking for help, provide as much information as possible, including the results of the 'xmllint' command.

Step 6 - svn diff

The command "svn diff" (below, without quotes) contacts the remote repository, verifies that the working copy is up to date, and compares the local, working copy to the master repository.

For the patch to be useful this command should be executed within as large a context as is appropriate. For that reason,

cd to the top-level directory of the gnucash-docs project.

Then run this command:

svn diff > diffs09262010

The command compares all files (in the checked out set of modules to the same modules in the trunk set) for changes. It then places changes found in a difference file. A date in the file name helps identify when the changes were made. Names used for the difference file are entirely up to the person making the update.

A patch should always be made against the most recent revision of the gnucash-docs project in subversion. In order to make sure your working copy is based on the most recent revision, run:

svn update

As a side-note

to avoid merge conflicts as much as possible, it is wise to run "svn up" (svn update) regularly, so repository changes are pulled in frequently and in smaller chunks.

This will pull in all changes and patches that have been committed to the gnucash-docs module since you checked out your own working copy from subversion. If those changes don't conflict with your local changes, they will be merged automatically into your local working copy. If there are conflicting changes, you will have to figure out which changes to keep and which ones to discard. Recently, svn has some tools to help you with this. Should conflicts occur, it will ask you what to do. See [10] for more information on conflict resolution.

Once the conflicts are resolved, run

svn status

This will list all files that are different between your local working copy and the project in subversion. This can be files that have been deleted, new files or modified files. Again, see [1] to understand the output of this command.

A single patch can actually contain all three kinds of changes: adding a new file, modifying an existing file, and deleting an existing file. Just make any and all changes you want, "cd" to the top-level directory of the gnucash-docs project, and run a

diff -ur > changes.patch

from there. That will look through all subdirectories for changes and integrate everything into a single patch. For details on the diff command, run

man diff

and skim through the options there. You can ignore most of them. Press Space to scroll down by a page, and ‘q’ to exit the manual viewer.

To create a patch, you have to make sure that subversion will consider your changes for commit. In practice only files that have status "A" (Added), "M" (Modified) or "D" (Deleted) will be taken into account. If you see files in the list that you know have changes you want to include, but that are not in this state, you should deal with this first.

1. Newly created files (the ones in state "?") you wish to include in the patch should first be added via "svn add"
2. Files you removed with a plain "rm" command (the ones in state "!") should be properly removed via "svn rm"

To create your patch, simply run:

svn diff > changes.patch

Note that the "svn diff" command doesn't require you to give file names. It will go through your working copy and write a proper patch, containing all new files, deleted files, and file modifications. If you wish to be more selective, you can also specify exactly which files you wish to add to the patch like so:

svn diff <list-of-files-and-or-directories> > changes.patch

For example:

svn diff guide/C/ch_capgain.xml AUTHORS > changes.patch

will only include changes to the files guide/C/ch_capgain.xml and the AUTHORS file into the patch. Likewise the command:

svn diff guide > changes.patch

will only include any changes in any file below the guide directory.

Finally, XML and HTML don't really care about trailing whitespace. It's mostly a coding habit to remove them. There are even text editors that do this automatically sometimes. If such a "cleaned up" file is committed to subversion together with actual code/documentation changes, you may have a harder time reading the differences later on. For example, when you are looking at a Trac difference page, the whitespace changes distract from the actual changes. That's what is meant by "cluttering up the repository". So being strict in removing trailing whitespace is not a matter of proper syntax, but rather a matter of making it easier for humans to read changes made. You are recommended highly to remove white spaces.

For that reason also, pure whitespace fixes should be done in separate patches from actual changes. Frequent calls of "svn up" will ease your work. If you wait too long, you risk conflicts when someone else and you are working on the same file. Especially if the other person has just committed a patch that you sent in for review, you should run "svn update" so your local working copy knows that the patch is now in subversion and only new changes should be added to a future patch.

Other resources available to discern differences and review them are listed in the resources below at items [8] and [9].

Step 7 - build the HTML version of the documentation

This step means that after XML has been tested for integrity and the difference file has been determined to contain correct changes and only those, then the Guide must be recreated in HTML and the results examined to verify that the online version of the Guide presents an appearance and reads as expected. The command to make this conversion is

xsltproc -o output_html/ ../../xsl/general-customization.xsl gnucash-guide.xml

NOTE

there is a space following 'output_html/ and preceeding '../'

"output_html" is a directory that will automatically be created and filled with the files the command creates. The directory name can be anything that makes sense to the person executing the command (output_html or something you choose).

Note the '/' following the file name 'output_html'. When that character is in place, the directory (output_html) is created and HTML modules placed in it. When not present, the output_html modules are placed in the current directory. Be sure to retain that '/' in its proper place!.

The command segment (“../../xsl/general-customization.xsl”) is a relative path to the XSL stylesheet used to turn the raw input XML into the output HTML that comprises the online version of the Guide. This command segment must be used exactly as written here.

If your changes involved no images, screenshots, icons, figures of any kind, then you are done at this point. However, if your work does involve figures of any kind, they will not be viewable in the HTML files created up to this point. In that case you should run this quick fix in a terminal:

cd output_html

ln -s ../figures

ln -s ../../../stylesheet

After making this fix, any page containing any figure when reloaded should show the missing figure(s).

Once your inspection shows that the online Guide is now acceptable in all respects, you should make certain that the output_html

 directory and its contents are not included in any respect in your patch.  To do that, before building the patch, remove the directory by this terminal command:

rm -rf output_html

Step 8 - Create the documentation change as a bugzilla bug

See resource item [11].

References - supporting technologies used

Step 1

[1] http://svnbook.red-bean.com/en/1.5/svn.tour.html

[2] http://wiki.gnucash.org/wiki/Subversion

Step 2

[3] http://svn.gnucash.org/docs/guide/

Step 3

[4] http://www.jedit.org/

Step 4

[5] http://en.wikipedia.org/wiki/DocBook

[6] http://www.tldp.org/LDP/LDP-Author-Guide/html/tools-edit.html

[7] http://www.netlingo.com/tips/html-code-cheat-sheet.php

[8] http://svn.gnucash.org/trac/timeline

[9] http://wiki.gnucash.org/wiki/Git

Step 6

[10] http://svnbook.red-bean.com/en/1.5/svn.tour.cycle.html#svn.tour.cycle.resolve

Step 8

[11] http://wiki.gnucash.org/wiki/Bugzilla