Difference between revisions of "Project and Design Documentation"

Revision as of 19:07, 16 September 2014

General

The people behind GnuCash aim to create a world-class GPL'ed Open Source Personal Financial Application for GNU/Linux and other Unix's.

This page documents the technical issues and development directives surrounding this project. It is a kind of an FAQ for developers and contributors, providing status, and guidance.

If you simply want to get a better idea of what GnuCash is and what it does, visit its GnuCash home page or browse the GnuCash Wiki page. The home page contains screen shots, news items, and mailing list archives.

Architectural Goals

There are some over-reaching design principles and philosophies that we intend to maintain. Some of these concepts and terms are introduced in this section.

Separation of GUI and Data

First, we must maintain a clean separation between the data structures and the GUI that manipulates them, along the lines of the Model-View-Controller paradigm.

Model 

Lists of accounts and the transactions in them can be thought of as a representation of financial data, a Model.

Controller 

The GUI that adds, modifies and deletes these should be thought of as a manipulator of the data, a Controller. Thus, the Motif or Gnome GUI's are merely two possible manipulators of the data; others, based on e.g. web/cgi-bin, Qt/KDE, emacs, Java applets or Java servlets ought to be possible.

View 

The View of the data is a subset or slice of the data described by the Model. The View may consist of only the transactions for the month of May, or only the account totals for certain accounts. The View is used in part to generate the reports and graphs, but it is also that which the Controller interacts with. Views are generated by queries to the data store.

GnuCash also needs to deal with multiple distributed datasources: stock quotations from the net or transaction confirmations from online banks and brokerage houses, or from more mundane sources, such as file imports, or merger of data from several users. Amongst these terms, the concept of a global Model-View is dated, and somewhat inappropriate. Rather, we need to be concerned about how data is represented in the local address space of the GUI, how the GUI manipulates it, how data is brought in and merged from external sources, and how that data is again output, whether to a file or a local or remote database.

Thus, the View essentially represents a local data cache of the data that is immediately present and being displayed, reported, and manipulated. The Model is the abstraction of that data that the GUI (the controller) can act on.

Financial Engine

In GnuCash, the Model is implemented via the Engine API,and the View is the data that is currently in the Engine. Thus, the Engine is a set of programming API's that the GUI (or a script, or even a clever command-line-addict) can use to manipulate the data.

Currently, the Engine provides basic accounting structures. These include:

Transactions 

consist of a set of splits or journal entries (JE's) whose values sum to zero. The transaction includes several date fields, a description, and a common-currency field, and a universal unique id (uuid/guid). It also provides hooks to store arbitrary data associated with the transaction (using a URL-based key-value tree).

Split 

(or journal entry) is an amount in the account on which it is drawn. Splits also store reconcile status, dates, a memo field, and also a key-value based hook for arbitrary data.

Accounts 

include a name, a type, a description field, and the type of commodity they store. In principle, the account consists of a list of splits.

Chart of Accounts 

is a hierarchical tree of accounts.

The Engine has a basic two-phase commit model, and a query mechanism for finding the data needed for reports and views. The goal of the two-phase commit and query model is allow the creation of multi-user server based backends, such as an SQL backend, and RPC client-server backend, or an XML-based HTTP/web backend. This design seems to work for the above-named backends.

The Engine currently handles only a basic set of data sources:

• It can import and merge in QIF's (actually, this function has been moved into the GUI, and is no longer part of the engine)

• It can read and write its own XML byte stream; This ability has been used to provide a multi-user client-server demo (which is currently broken).

• It can use a Postgress SQL database as a datastore, thereby enabling multi-user and auditing functions.

• It can talk, via RPC, to a gnucash server. (This code is 'alpha' and incomplete/broken).

• It can get stock quotes from the net (actually, this function is provided by a separate module, the Finance::Quote perl module.)

However, since the Engine is meant to be the interface between the GUI and the financial data, it is really intended to be able to do much more. In particular, it should be possible to create a peer-to-peer network model, where gnucash peers can synchronize data between themselves. The engine should also be expandable to handle other sources of data, such as OFX/IFX, the Open Trading Protocol, or the OMG CORBA General Ledger submission. In particular, it should be possible to use GnuCash not only to view data from these sources, but also to manipulate it and send it back.

Modularity, Extensibility and Customization

The above structure should lead us to view GnuCash not so much as a tightly integrated application, but rather as a loose confederation of component objects, libraries and interfaces. This has a number of advantages for both the developer and the user. For the developer, it allows parts to evolve semi-independently of one-another, and to be used in other, non-gnucash projects. For the user, a good extensibility allows the use of #arrangements": a way of broadly customizing the appearence and behaviour of gnucash, and then allowing users to very easily share these customizations with one another. Such arrangements might be collections of canned reports, e.g. for business or home user. Or they might be the menu contents: one menu arrangement is for beginners, another for 'power users', a third for business owners. The goal is that broad areas should be not only customizable, but it should be possible, even easy, to trade these customizations between users.

In order to facilitate the gluing together of these parts, as well as simplify the questions of customizability, change and rapid development, GnuCash makes use of the Scheme extension language (as implemented in the FSF Guile interpreter), to glue the pieces together. (Note that the engine interface is also available with Perl interfaces, thanks to a SWIG wrapper.

A Web Browser for Financial Data

More and more finacial data is moving onto the web. People shop on-line. They pay bills on-line. There are even some promising e-wallet systems (such as WebFunds). For GnuCash to be relevent in this on-line world, it must be able to interact with these systems. There are several steps that can be taken along this path. First, it must be possible to simply and transparently import financial data off the web. Click on a QIF file, mime-type application/x-qif? Gnucash sucks it in without burping. But, in a more distant future, can GnuCash originate transactions? It should be able to!

GnuCash can be and should be "The Sophisticated Financial Web Browser". All GnuCash reports and the help system are HTML-based. In fact, GnuCash has a built-in web browser that can view ordinary web sites. Interactive gnucash components such as the bar and pie charts, or the graphs and reports, can all be served up by a remote web server, as well as locally through the GnuCash application. This allows GnuCash to blur the line between web browser and financial application, and offer the best of both worlds.

Why is this a good thing? One can always have a pure web-based accounting solution (such as SQL-Ledger) that uses ordinary web browsers to view the financial data. But there are problems: dynamically-generated html and cgi-bins aren't as pretty or easy to use, or as fast, responsive or sophisticated, as what you can do with a custom client. That is, GnuCash can be more interactive, easier to user, slicker and more professional looking in style and presentation than a plain-old dynamic website. Financial ASP's simply cannot currently offer the kind of utility, flexibility, responsiveness and integration that GnuCash can offer.

In theory, the 'Java revolution' was supposed to provide this function, with downloadable Java applets providing the 'fancier- than-plain-html' interface. But a Java applet that is this sophisticated would also be large and slow to download: it offers no inherent advantages over native code. Another problem with Java is the inherent propreitary fragmentation: no two Java applets are alike: every site has thier own; there is no standardization.

In order to solve the 'no gui standardization between websites' problem, there are a number of businesses that have arisen to provide "financial aggregation". You sign up with these sites, and let them get all of your credit-card, bank and investment info, and they can provide a unified interface for your bank statements, with graphs and reports (all for a monthly fee). The existance of these services highlights a problem: the inability to aggregate the same data on the users desktop, in a simple, coherent fashion. Let us posit that GnuCash, with the right on-line interfaces, would be a natural for this. For the paranoid, this has the added advantage that no one business has access to all of your financial records.

To put it another way, GnuCash can provide a centralized, trusted store for financial data that no other application can provide. The convenience factor: if you have trading accounts at e*trade and charles schwab, then GnuCash can be the central place where you can oversee all of your investments. The trust factor: maybe you can trust your web bank. Maybe you can trust your web stock account. But can you trust a single web entity with *all* of your financial data? Someone who won't treat you as 'just another consumer' and sell your 'consumer profile' to anyone who cares to stalk (ahem, target) your financial activity (ahem, purchasing) patterns? Didn't think so. Of course, you

• could* use some proprietary financial software. Assuming, of course, that you trust it not to have any built-in covert channels:

nothing that might send back the make and model of your CPU and the last ten transactions 'home to mommy' for 'diagnostic purposes'. Because open source software, such as GnuCash, can be audited it can be trusted in ways that no proprietary software can be. In an increasingly net-connected world, the ability to build trust through audits will be increasingly important.

Looking Into the Future

Things like support for OFX, and small-business features are hopefully in the near term future of gnucash. Lets take a look at some more far-out possibilities.

E-Wallet 

An e-wallet, just like a real wallet, stores cash. You can use that cash to pay freinds or businesses. Just like a real wallet, an e-wallet should not charge you monthly fees. It should provide you with some measure of anonymity and privacy. An e-wallet should also do something no ordinary wallet can do: show you a monthly report of expenditures. GnuCash should get e-wallet features.

Authentication, Authorization and Access Control 

How does an on-line merchant know that you are who you say you are? How can you use GnuCash to authorize a bill payment? Sooner or later, smart-cards will provide the sort of authentication and access control that is only groped for with PGP/GPG, keyrings, x.509 certificates, and Microsoft Passport/Hailstorm. Of all the electronic things in the electronic world where access control and authentication are important, real money is the most so.

Bill Presentment, Bill Pay 

These magic keywords just mean that a merchant wants to send you a bill, and wants you to go to thier website to pay that bill (they want your eyeballs). Of course, for the user, it would be more convenient to have the bill show up on the desktop, inside of gnucash, and get authenticated and paid out from the desktop (by gnucash), as you would do with all of your other bills.

Requirements

Lets take a look at who the current and potential future users of GnuCash are, and how they might use it.

The classes:

• Basic Household Accounting/Budgeting
• Personal Stock Portfolio Management
• Personal On-line Trading (Day-Trading)
• On-line shopping and bill-pay
• Small Business Users

These different applications may use some of the same financial terminology, and hopefully might share a fair bit of code, but have quite different goals and requirements.

Basic Household Accounting and Budgeting 

Important properties of a personal finance system include:

• Approachable and usable by occasional users who are not knowledgeable about accounting.
• Ease of use and simplicity is critical.
• A reasonable selection of reports, graphs, charts, and tools for personal finance, such as mortgage calculations.
• Budgeting support needs to be targeted at native users.
• Interfaces to on-line banking, shopping, stock systems. Bank and credit-card statements should arrive 'automatically' and always be up-to-date.

Personal Portfolio Management 

• Support for management of stock portfolios that may involve considerable sophistication, since individuals commonly have retirement plans that hold mutual funds, stocks, options, bonds, and the like.
• Reporting infrastructure needs to simplify handling of tax issues (long-term vs. short-term capital gains/losses, cost-basis FIFO accounting, simplfied cost-basis spinoff/merger handling). Need on-line updates of prices, simple portfolio overview, ability to link to websites for additional research.
• Integration with on-line trading systems. This could save time typing.
• Basic Audit features. There's a problem with blindly allowing on-line data (prices, transactions) to enter GnuCash. It may not be clear where it came from, and even if the source is beleived to be 'reputable', there still may be factual errors in the data. Thus, there must be a way of audting newly-arrived (or even old) on-line data, and mark it as 'reconciled', i.e. manually reviewed and checked for accuracy. That is, incoming on-line data must be auditable, and audit trails should mark the history of on-line data import.

Small Business Needs

• With a business system, it is likely that there will be users who use it eight hours a day, which puts the emphasis on efficiency of user interface rather than on its approachability to naive users.
• Business systems require network support, and the ability to support multiple simultaneous users.
• Some business users may want access to the system from an MS Windows 95/98/NT box. For these folks, a web-based interface could be just handy. Web interfaces are also nice a for ASP type deployment.
• Small businesses do not often have sophisticated investment portfolios; they instead need support for additional sophistication in such areas as:
  • Customer and Vendor Lists; Invoicing
  • Payroll (Batch processed and individual)
  • Inventory Control & Asset Management
  • Amortization Schedules, Depreciation
  • Shipping and Receiving
  • Accounts Receivable, Accounts Payable (A/R, A/P)
  • Credit Card Processing<
• Support for calculations associated with accrual accounting.
• Ambitions for the future might include interfaces to online shopping carts, credit card clearing interfaces, and ERP systems.

Reconciling Those Needs

A seemingly contradictory factor is that the kinds of sophistication that are required vary considerably. Consider:

• A home user does not generally require most of the sophistication of accrual accounting that is required by business enterprises. Thus, home users don't need much of the sophistication of an Accounts Receivable or Payable system, or the bizarre depreciation policies that crop up in Asset Management systems.
• On the other hand, home users are in increasing need of surprisingly sophisticated financial tools for managing stock portfolios, including:
  • Mutual funds
  • Retirement savings plans, with such identifiers as 401(k), IRA, Roth IRA, RRSP, or Insurance Annuities
  • Corporate stock purchase plans
  • Corporate option plans

Another set of contradictory requirements has to do with the back-end, and interfacing to other systems:

• Home users need a simple-to-install, simple-to-maintain system. This essentially rules out the use of SQL for the storage medium/back-end for home users. (That is, the current state of the art for SQL on Linux does not offer any simple, fool-proof management for data).
• By contrast, non-SQL systems for business use are almost unimaginable. SQL provides a high degree of data integrity and storage robustness, and also simplifies tremendously the import and export of data. Powerful SQL tools exist that can work magic in the hands of a good DB admin.

It may be that these will require completely different systems, and that GnuCash cannot be "all things to all people." This remains to be seen.

GnuCash Architecture

Decisions

Project Decisions

Design Decisions

Rules

Project Rules

Design Rules

Coding Rules