Difference between revisions of "Budget History"

From GnuCash
Jump to: navigation, search
(Recurrences: link to FreqSpec.h documentation)
Line 1: Line 1:
= Budgets in Gnucash: A History =
+
DOH
This page is intended to document the history, current status and design decisions in the development of Budgeting functionality for the G2 port of Gnucash.  First person pronouns refer to Chris Shoemaker.
 
 
 
 
 
== Preface ==
 
Budgeting functionality in Gnucash has a long and (to me, at least)
 
somewhat frustrating history.  I'm attempting to document some of that
 
history here because it provides some of the rationale for some of the
 
design decisions I've made in my recent design for budgeting.  I make
 
no claim that this recounting is purely unbiased (which, incidentally,
 
no history is) because I'm also using is as a forum for me to weigh-in
 
on debates that raged long before I ever used Gnucash.  I'm "calling
 
'em like I see 'em", and "pulling no punches" to use some idioms, but
 
I do try to remain objective and I think that those places where I'm
 
expressing my opinion are, hopefully, obvious.
 
 
 
== Ancient Times (Before November 2000) ==
 
As early as July 2000, there was some work on implementing budgets in
 
Gnucash.  Bryan Larsen wrote a scheme report titled "budget report",
 
that required the user to enter their budget directly into the scm
 
file.  But when the reporting system in Gnucash was updated, this
 
report was not converted.  By November 2000, Bryan considered this
 
report unmaintained, and hoped it would be replaced with something
 
better.  The lead developers recognized budgeting as one of the most
 
significant "missing features" of Gnucash, and hoped to include it in
 
the next release.
 
 
 
 
 
== Serious Discussions (November 2000 – May 2001) ==
 
In November 2000, one of the earliest recorded collaborative efforts
 
to design a budgeting system began with almost simultaneous mailing
 
list posts by James LewisMoss <jimdres (at) mindspring.com> and Joshua Sled
 
(who later became one of the lead developers).  Jim took an approach
 
of soliciting requirements from list members, while Josh collected a
 
large body of notes and posted them for discussion.  (Incidentally, the
 
lack of budgeting functionality in Gnucash was one of the factors that
 
prompted Josh Sled to join the development effort in the first place.)
 
 
 
Both of these threads generated a fair amount of discussion about
 
budgets, but no clear design consensus was reached.  There was,
 
however, some rejection of what did _not_ constitute budget
 
functionality.  For example, despite what several people claimed to
 
desire, Josh kept the discussion on-track by explaining that budgets
 
cannot actually constrain real-world income and expenses.
 
 
 
One of the most remarkable features of this period, and one that can
 
probably only be appreciated from the comfort of several years
 
distance, was the rather low signal-to-noise ratio in the budget
 
discussions.  Surrounding a small handful of people who had at least
 
some clear vision of the problem space were many well-meaning, vocal,
 
people who only side-tracked the discussions because of a lack
 
of familiarity with software development, double-entry accounting, or
 
even basic accounting practices. 
 
 
 
Much of the discussion was of no particular value for achieving budget
 
functionality, covering instead the types of psychological or physical
 
methods people used for self-imposed fiscal discipline; it often
 
sounded like an overly enthusiastic self-help seminar extolling the
 
virtues of financial planning.  As an example of the difficult
 
environment for collaborative design, take this quote:
 
 
 
    "The principle of budgeting says that we make contributions each
 
    time we are paid into various savings accounts to cover each kind
 
    of expense. &ndash;there are also [various savings accounts], which
 
    we make regular contributions to. From the point of view of our
 
    budget, they are an expense&ndash;" <name protected> 02 Dec 2000
 
 
 
(For non-accounting-minded readers: Money deposited to your savings
 
account is _never_ an expense.)  Nevertheless, there was also much
 
useful discussion, too, often driven by Josh's motivation to
 
contribute to development.
 
 
 
Even though it claimed not to be a design document, Josh's notes
 
included helpful definitions and typical use-cases covering a broad
 
range of budget types.  They also listed some informal requirements
 
for a budgeting system, and even some concrete menu design and
 
GUI-layout.  His notes also appear to have introduced the term
 
"savings goal" (which is apparently a Quicken term for something using
 
virtual accounts to "hide" money in accounts, without reconcilable
 
transactions, unrelated to budgeting) to the Gnucash devel-list
 
discussion.
 
 
 
In these early design discussions, two topics became especially
 
important: the relationship between scheduled transactions and
 
budgeting, and the concept of budgeting categories.
 
 
 
Josh's notes were instrumental in focusing the discussion onto what
 
was generally considered to be the other significant "missing feature"
 
&mdash; scheduled transactions (SX).  There was, however, some disagreement
 
about the exact design relationships between budgeting and SX.
 
 
 
Aaron Peromsik tended to view them as complementary, but distinct:
 
 
 
    "Quicken draws a distinction between "budgeting" functionality and
 
    "scheduled transactions." I find the scheduled transactions useful
 
    as reminders, but I don't use the auto-entry functionality because
 
    I don't want things entered as paid until I actually mail the
 
    check."  Aaron Peromsik <aperomsik (at) mail.com> , 23 Nov 2000
 
 
 
But the nature of the distinction remained unclear:
 
 
 
    "I agree there is a distinction, but I see scheduled transactions
 
    being very useful for budgeting, so for the moment I'm including
 
    them together in my head." James LewisMoss
 
    <jimdres (at) mindspring.com>, 23 Nov 2000
 
 
 
    "There are two major approaches to budgeting that I am aware of,
 
    and while I agree that thinking about them 'together' is necessary
 
    to getting a design, they tend to make the head hurt." Christopher
 
    Browne, <cbbrowne (at) hex.net> 28 Nov 2000.
 
 
 
Christopher Browne then went on to make the distinction quite clear in
 
a post in which he names them the "accountability model" and the
 
"pragmatic adding scheduled transactions".
 
 
 
After a couple of months of discussion, Aaron summed up the current state
 
of the arguments over SX vs budgeting:
 
 
 
    "In all the (recently sparse) discussion of budgeting and
 
    scheduled transactions, people seem to like to lump scheduled
 
    transactions in with budgeting. I still think this is a
 
    mistake. Budgeting is a big scary complex subsystem which many
 
    people will not use at all, whereas scheduled transactions can be
 
    a pretty straightforward affair which is more broadly applicable."
 
    Aaron Peromsik <aperomsik (at) mail.com> 14 Jan 2001
 
 
 
Even though there remained some disagreement about the relationship of
 
budgeting to SX, Josh did something very wise: He focused on what
 
there seemed to be greater agreement about &mdash; SX:
 
 
 
    "This is true [that lumping the two together is a mistake].  I
 
    agree with the assertion that budgeting may not be used by all;
 
    perhaps there is some smaller subsystem for scheduled
 
    transactions, which the budgeting subsystem can utilize&hellip;" Joshua
 
    Sled <jsled (at) asynchronous.org> 14 Jan 2001
 
 
 
This marked Josh's turn toward SX, so that two weeks later Josh wrote:
 
 
 
    "I've been focusing recently on scheduled transactions, since
 
    that's what people more seem to want [more than budgeting], and
 
    the budgeting stuff should make use of a separable
 
    scheduled-transaction subsystem.  My hope is that soon/eventually,
 
    someone else will finish/polish/maintain the scheduled transaction
 
    stuff while I focus on budgeting&hellip;" Joshua Sled
 
    <jsled (at) asynchronous.org> 1 Feb 2001
 
 
 
Even though the exact relationship had not been hammered out, the
 
developers generally viewed SX as somehow either complementary or
 
essential to budgeting and very useful in their own right.  Josh's
 
willingness to work on the part which he viewed as better defined but
 
less satisfying was very much responsible for Gnucash gaining SX
 
functionality.
 
 
 
Nevertheless, serious design decisions about budgets remained
 
unanswered, particularly regarding the concept of budgeting categories:
 
 
 
    "The budgeting categories can span multiple expense
 
    accounts/categories, so they should be external&hellip; I think."
 
    Joshua Sled <jsled (at) asynchronous.org> 1 Dec 2000
 
 
 
    "I think the budgeting categories [bcats] are external from the
 
    accounts to a large degree&hellip; while they may reference the
 
    accounts which the bcat refers to [expense or liability or
 
    whatever], the information about budgeting items crosses over
 
    expense/account category boundaries." Joshua Sled
 
    <jsled (at) asynchronous.org> 13 Feb 2001
 
 
 
To which Gregory Novak responded:
 
 
 
    "True, but I think that asking the user to define an entire
 
    additional additional set of categories for budgeting is asking
 
    for trouble.  I believe I've seen talk of associating a "category"
 
    with every transaction in addition to the account info.  If
 
    budgeting could be folded into this category specification, then
 
    we're golden, but if not, adding budgeting category information on
 
    top of this would be unwieldy.
 
 
 
    One thing may be to just ask the user how much will be transfered
 
    to/from each account, regardless of the account's categorization
 
    or whatever.  It seems to me that this would give you a lot of
 
    generality with very little effort in setting things up (for the
 
    user)." Gregory Novak <Gregory.Novak (at) cern.ch> 14 Feb 2001
 
 
 
Josh remained unconvinced:
 
 
 
    "Maybe, maybe not.  It's all about presentation and user interface.
 
 
 
    I do believe that the budgeting categories are not going to be the
 
    same as the expense categories&hellip; they will generally be a subset
 
    [and, in my case they might very well be a one-to-one map], but
 
    the user should be able to create a set of budgeting categories on
 
    the level which she wishes&hellip;
 
 
 
    For instance, I may have expense [sub-]categories for ea. type of
 
    donation I make to various organizations.  But I would probably
 
    budget this on a yearly scale as 'Donations'.  Entertainment, as
 
    well&hellip; I might have categories for books and movies and painting
 
    supplies and whatnot, but I'd probably just budget the category as
 
    a whole.
 
 
 
    Taken to the other extreme, I might have a lot of budget
 
    categories, but no expense categories. I may not care about
 
    tracking my expenses [no matter how easy GnuCash makes it :)], but
 
    I still want to make a budget over some general expense
 
    categories, and just leave the rest of my expenses [and my
 
    spending allowance] as the difference between known income and
 
    known expenses&hellip;  I don't think I'm quite understanding your
 
    budgeting world view&hellip;" Joshua Sled <jsled (at) asynchronous.org> 15
 
    Feb 2001
 
 
 
There was almost no discussion of the design of budgeting between Feb
 
15 and Jun 15, 2001.  The single thread of discussion during this
 
period evidences the continuing disagreement over budget categories.
 
Aaron clearly understood Josh's definition of budget categories and
 
thought that it had no place in his concept of budgeting.
 
Specifically referring to budget categories, Aaron wrote in response to
 
Josh:
 
 
 
    [Josh:] "The forthcoming budgeting stuff should be very cool,
 
    too&hellip; is budgeting something you'd think you'd use?  If so, I'd
 
    like to bounce ideas off you as we get further along in the
 
    thoughts/design of it all."
 
 
 
    [Aaron:] "Let's put it this way: In theory, yes&hellip; but in
 
    practice, if budgeting is nearly as complicated as it is in
 
    Quicken, or nearly as complicated as your previous descriptions
 
    (was it all the way back in December or February) then I want no
 
    part of it."  <aperomsik (at) mail.com> 20 May 2001
 
 
 
This difference of opinion went unresolved, and this issue would
 
resurface at a later time. 
 
 
 
 
 
== Design Diffusion (June 2001 &ndash; October 2001) ==
 
 
 
September and October generated a huge flurry of budgeting
 
discussion.  The confusion in these discussions was captured by Derek
 
Atkins:
 
 
 
    "I think part of the problem in this discussion is that people are
 
    coming from completely different angles and the nomenclature is
 
    extremely confusing.  Some people seem to be looking at budgeting
 
    as a way to describe Gnucash accounts; some people seem to think
 
    of budgeting as a side line that is separable from Gnucash
 
    accounts.  This dichotomy appears to be causing some conversant
 
    trouble.  Unfortunately I don't have a good approach to solve this
 
    problem." Derek Atkins <warlord (at) MIT.EDU> 01 Oct 2001
 
 
 
In my opinion, there was one important distinction that has been
 
left unmade even since the early discussions: the difference between
 
capital budgeting (sometimes called investment budgeting) and cash
 
budgeting.  (Capital budgeting is planning for some far-term, usually
 
one-time and hypothetical, expense &mdash; often with a feasibility
 
evaluation.  Cash budgeting is the recording of predicted, often
 
recurring, usually not-so-hypothetical, incomes and expenses &mdash;
 
usually with on-going comparison of actuals and planned.)  This
 
confusion was only somewhat troublesome when people were still working
 
out the distinction between SX and budgeting, but now that Josh had
 
worked out the design of SX and the discussion turned toward
 
budgeting, the confusion was more acute. 
 
 
 
The confusion of these two types of budgets had two main negative
 
effects.  First, constant miscommunications on this point prevented
 
any focus on the actual design of budgeting.
 
 
 
Josh used the term savings goals to mean something similar to capital
 
budgeting and he was especially interested in the potential for
 
interactively examining financial hypotheticals.  Therefore he devoted
 
much discussion to a design that would suit these purposes.  On the
 
other hand, he clearly had cash budgeting squarely in view, too:
 
 
 
    "The more important analysis is the historical one, comparing
 
    budgeting constraints to actual spending." Joshua Sled
 
    <jsled (at) asynchronous.org> 22 Jan 2001
 
 
 
Josh would usually lump capital and cash budgeting together in his
 
discussions (Although, in a reposting of his design notes, Josh's
 
"Basic" vs. "Advanced" budget functions vaguely resemble cash
 
vs. capital budgeting.) and others would disagree with his design
 
proposals because they were considering only cash budgeting.  For
 
example, in response to Josh's idea for a GUI that would allow capital
 
budgeting-style analysis, Bryan wrote:
 
 
 
    "I disagree.  As far as I'm concerned, you create a budget, you
 
    balance it, then you report on it continuously for 1 year.  Then
 
    you create a new budget with the information you learned by
 
    completely blowing your budget the previous year.  :)", Bryan
 
    Larsen <blarsen (at) AnalogSynthesis.com> 26 Jul 2001
 
 
 
In response to Josh's allusions to capital budgeting features, the
 
typical reaction was to conclude that there was still disagreement
 
about what a budgeting system really was supposed to do.  This
 
resulted in several more discussions that started out like:
 
 
 
    "This is where we may have to discuss the role of the budgeting
 
    system."  30 Sep 2001 <nigel_gnucash-devel (at) unos.net>
 
 
 
And continued back-and-forth with Josh describing a mixture of cash
 
and capital budgeting features and others saying that some of those
 
features weren't properly part of a budgeting system.
 
 
 
The second negative impact of the confusion of budget types was that,
 
in Josh's continuing design notes, the broader scope of a budgeting
 
system that supported both capital and cash budgeting without any
 
clear distinction between the two made the design unclear and
 
unconvincing.  Josh seemed fully conscious of this, and he occasionally
 
expressed uncertainty with his own design notes.
 
 
 
    "I've been playing around with designing structures to capture
 
    budgeting in GnuCash, and figure out how budgeting integrates with
 
    the rest of the program.
 
 
 
    When you get closer to the end, it starts to unravel a bit&hellip; it's
 
    in a weird state in my head at the moment&hellip; not quite coherent,
 
    definitely not finished." Joshua Sled <jsled (at) asynchronous.org> 22
 
    Jan 2001
 
 
 
    "I've spent many mornings of weekend at the coffee shop up on the
 
    corner writing out pages of notes on budgeting.  Unfortunately,
 
    they always seem to start with 'Okay&hellip; blank slate:' and end with
 
    'hmmm.'" Joshua Sled <jsled (at) asynchronous.org> 25 Jul 2001
 
 
 
    "I tried very persistently for a few months to pin down a good
 
    budgeting design, and failed to come up with something that said
 
    'that's it' to me immediately." Joshua Sled
 
    <jsled (at) asynchronous.org> 18 Sep 2001
 
 
 
One of the contentious design points was the resurfacing budget
 
categories concept.  Josh's position seemed unchanged:
 
 
 
    "And would you agree that, while similar to the account hierarchy
 
    in concept and perhaps application, the budgeting hierarchy
 
    shouldn't be forced to be the same as the expense hierarchy?"
 
    Joshua Sled <jsled (at) asynchronous.org> 30 Sep 2001
 
 
 
There was some struggle to come up with a good example of a need for
 
budget categories that didn't map one-to-one onto accounts.  In the
 
end, the Josh settled on an example where expenses were tracked at a
 
finer grain (Entertainment:Movies, Entertainment:Videos) than they
 
were budgeted for (just Entertainment).  In my opinion, this example
 
is not at all convincing, since the budgeted quantity _is_ for an
 
account, just not the leaf accounts.
 
 
 
The responses to complex (meaning not one-to-one mapping to accounts)
 
budget categories varied from strong disagreement to apathy to
 
reluctant acceptance.
 
 
 
Interestingly, in the responses concerning budget categories, one
 
small exchange appears to have gone almost completely unnoticed:
 
 
 
    [Philip:] "And I come back to a question that was asked some time
 
    ago, (and not answered to my knowledge) where do I place the shoes
 
    I bought for soccer.  Is that a sports expense or a clothes
 
    expense?  How do I let it be both?  Thus is this a sports budget
 
    item or a clothes item.", Phillip Shelton <shelton (at) usq.edu.au>
 
 
 
    [Matt:] "I think that would be a task for the "action" field
 
    listed on the devel page for gnucash.  The idea is to be able to
 
    assign transactions to action fields that would allow you to group
 
    them in addition to categories.", Matt Kowske
 
    <jmkowske (at) students.wisc.edu>
 
 
 
    [Philip:] "What if I want to assign more than one type of action
 
    per split?"  27 Sep 2001
 
 
 
No one responded to Philip's question, but it would come up again&hellip; XXX
 
 
 
My own summary of those discussions would be that Josh was still
 
gathering input about user experiences and still propelling the
 
discussion by his motivation.  But he was also still failing to
 
synthesize capital and cash budgeting into one design.  Nevertheless,
 
despite the lack of a clear design for a budgeting system, 2001 saw
 
tremendous progress on SX.
 
 
 
 
 
== The Dark Ages (November 2001 &ndash; March 2003) ==
 
 
 
The rest of 2001 and all of 2002 passed with no new discussions or
 
progress toward budgeting functionality in Gnucash.
 
 
 
In January 2003, there were some brief discussions about adapting the
 
cash-flow report into a budget report.  Also at that time, Matthew
 
Vanecek announced that he planned to work on budgeting sometime in the
 
future.
 
 
 
 
 
== Revival (April 2003 &ndash; September 2003) ==
 
 
 
By April 2003, Dave Fancella started to design a budgeting system from
 
scratch.  His design called for including the budget data in the KVPs
 
of the Account objects, which Matthew strongly objected to.  His
 
design doesn't appear to have resulted in any implementation, but the
 
mailing list discussion did make one design decision quite obvious:
 
Budgets needed to be first-class entities in order to support multiple
 
budgets.  What also became evident was that Matthew was in firm
 
agreement with Josh regarding BudgetCategories, but he didn't push the
 
issue when Dave's psuedo-code clearly supported only a one-to-one
 
mapping onto Accounts.
 
 
 
On September 1, 2003, in his first post to the devel mailing list,
 
Darin Willits <darin (at) willits.ca> introduced himself, outlined a new
 
design for the budgeting system, and said he was ready to begin
 
implementation.  Darin's design had one substantial advantage: he
 
concentrated exclusively on cash budgeting, which made the project
 
reasonable for the few weeks he expected to be able to spend
 
coding.  (By September 3, the GUI design was complete and by September
 
17, most of Darin's code was complete.)
 
 
 
The gnucash developers had not created a good design for a budgeting
 
system for many years, so is was unfortunate, but not surprising,
 
that Darin had not simply discovered the perfect design out of the blue in
 
one day.  Stewart Wright and others spoke up:
 
 
 
    "I don't want to put a dampener on this, but I think a clearer
 
    focus on what is to be done is a better approach than for someone
 
    to go out and start coding what they would like to see. Code is a
 
    'Good Thing(TM)' but there are still two different camps at
 
    least." Stewart V. Wright <svwright+lists (at) amtp.liv.ac.uk> Sep 1
 
    2003
 
 
 
Also, despite the fact that Darin's design was clearly addressing only the
 
problem of cash budgeting, much of the design commentary was still
 
focused on trying to clarify the distinction between cash and capital
 
budgeting for the people who saw no distinction.  Matthew Vanecek
 
was confusing cash and capital budgeting in his discussions, and this
 
may have been a reason why he was strongly advocating the complex
 
BudgetCategories design.  In fact, he separated the concepts of
 
BudgetCategories and Accounts even more than Josh had ever done:
 
 
 
    "I *would* like to see the capability to create a budget category
 
    w/o an associated account." Matthew Vanecek <mevanecek (at) yahoo.com>
 
    Sep 1 2003
 
 
 
Even while Darin implemented his design, the mailing list rediscussed
 
what exactly a budget was and what a budgeting system should do.
 
Stephen Cuppet even proposed an alternate complete design with a
 
one-to-one mapping.  While most of the conversation was full of
 
miscommunications about cash vs. capital budgeting, some of the
 
discussion did try to respond to Darin's design, and some of that
 
response was highly critical, especially on the point of budget
 
categories:
 
 
 
    "As for the concept of 'categories'.  This does not exist.  The
 
    actual data has accounts.  The budget should have accounts."
 
    Stephen Cuppett <scuppett (at) nc.rr.co> Sep 2 2003
 
 
 
    "I guess doing government accounting for the last 5 years has me
 
    jaded, but I can't for the life of me see why people are wanting
 
    "virtual" budget accounts.  This makes no sense.  You shouldn't be
 
    able to budget to a category in which you can't post actuals.
 
    True often you budget at some roll up higher than where you spend,
 
    but why budget for somewhere you can't spend.  Maintaining a
 
    second set of categories seems silly." Derek Neighbors
 
    <derek (at) gnue.org> Sep 3 2003
 
 
 
The right solution seemed so obvious to the professional accountant
 
that he viewed complex BudgetCategories as crazy:
 
 
 
    "Maybe I'm psycho but why wouldn't you just make your accounts:
 
 
 
    Expenses:Food
 
    Expenses:Food:Dining
 
    Expenses:Food:Groceries
 
 
 
    Then you choose to budget at the "Food" level and expend at the
 
    "sub" level.  This is how most budgets work.  You have a chart of
 
    accounts with each element having different "levels".  Often you
 
    budget at a higher level than you expend.  I see where you are
 
    trying to be flexible, but I think you are making it more
 
    complicated than it needs to be. In order to have some resemblance
 
    to commitment control you really need your budget accounts to
 
    mirror your expense accounts even if the budget is placed at a
 
    different level in the hierarchy tree than the expense."  Derek
 
    Neighbors <derek (at) gnue.org> Sep 3 2003
 
 
 
Then came perhaps the most frustrating point in the history of
 
Gnucash's budgeting: in response to some of the clearest design
 
decision material the list had seen for years, here's what passed for
 
design rationale:
 
 
 
    "Some food is not food. =P Perhaps some people would rather see a
 
    flatter view, rather than drilling down.  Me, I like more of a
 
    tree view like above.  Different folks&hellip;
 
 
 
    In any case, you ought to be able to link multiple accounts to a
 
    single budget category, even if only to satisfy people that set up
 
    like 'Expenses:Grocery' and 'Expenses:Fine Dining' accounts."
 
    <name protected> Sep 3 2003
 
 
 
With an illogical non-sequitur, Derek's concerns were brushed away.  Even
 
when Phil <sublime78ska (at) comcast.net> expressed strong support for
 
Derek Neighbors's evaluation, he was dismissed with the explanation
 
that complex BudgetCategories were needed for capital budgeting (which
 
Darin's design never covered).
 
 
 
There was probably no clearer indication that design decisions were
 
being based on the desire for capital budgeting ("financial
 
simulation") and then applied to an implementation of cash budgeting
 
than Matthew's response to David Ayers:
 
 
 
    [David:] "Are we not confusing budgeting as commonly understood
 
    with financial simulation?  I submit that potential purchases do
 
    not affect a budget. [&hellip;] Otherwise budgeting becomes a game
 
    without any real impact."
 
 
 
    [Matthew:] "YES!! YES!! YES!! YES!! YES!! YES!! My point exactly!!
 
    The above is '''precisely''' what we need out of a budgeting tool!!!"
 
 
 
    [Matthew:] "I '''do''' see a need for profiling possible scenarios,
 
    and how a future purchase will affect my budgetary needs. [&hellip;] An
 
    operating budget, however, like you say below, should experience
 
    minimal changes" Sep 4 2003
 
 
 
But even if the distinction between "profiling possible scenarios" and
 
an "operating budget" was becoming clear, it was too late to affect
 
the design of Darin's implementation.  Darin submitted his patch on
 
Sep 17, 2003.
 
 
 
== Missing Time (October 2003 &ndash; August 2004) ==
 
 
 
Strangely, there's no evidence that anyone even looked at Darin's
 
patch until Derek Atkins reviewed it on Feb 1, 2004.  Derek made no
 
comments on the code's design.  He had only minor concerns about file
 
names, coding style, and infrastructure integration.  Despite the
 
minor nature of Derek's comments, Darin didn't get around to
 
addressing them until I emailed him in Aug 2004, asking for his patch.
 
Darin's budget code was committed to the gnome-2 branch of CVS on Aug
 
4, 2004.
 
 
 
= Budgets in Gnucash: Another Look (August 2003 - present) =
 
 
 
In August 2003, I began to investigate the possibility of budgeting
 
from within Gnucash.  I had been using Gnucash with an external budget
 
for a while, and running budgetary reports had become quite tedious.
 
Like more than 10 developers before me, I knew I wanted budgeting
 
functionality in Gnucash, and I was willing to code it.
 
 
 
I got my feet wet by reviewing 4 years of budget design discussions,
 
and by looking at Darin's code.  One of the things I liked about
 
Darin's design was the clear focus on cash budgeting.  I knew that
 
cash budgeting was exactly what I needed because it was exactly what I
 
was already doing externally.  I had some small interest in, but
 
almost no need for, the capital budgeting envisioned by Josh and
 
others.
 
 
 
One of the more obvious short-comings of Darin's code was the lack of
 
budget persistence.  Eager to climb the learning curve of Gnucash
 
development, I quickly implemented budget persistence in the file
 
back-end for all parts of the budget except for BudgetCategories.  It
 
wasn't until I saw Darin's implementation of BudgetCategories that I
 
realized the design implications for the budgeting system.
 
 
 
 
 
== Budget [Non-]Categories ==
 
 
 
Perhaps seeing the implementation in detail made things clearer for
 
me; or perhaps I'm just naturally opinionated.  In any case, I don't
 
feel _any_ of the ambivalence that was sometimes expressed in the
 
discussions on budget categories.  I am in whole-hearted agreement
 
with Stephen, Phil, Derek Neighbors and everyone else who thought that
 
complex mappings of budget categories to accounts was crazy.
 
 
 
In my mind, cash budgets are very simple: they record predictions
 
about real accounts.  It's difficult for me to understand why complex
 
budget categories were attractive to so many people.  (Is an old DOS
 
version of MYM responsible?)  My only conclusion is that they were
 
perceived to solve some problem that I don't consider to be part of
 
the cash budgeting problem.  It may not be convincing to budget
 
category supporters, but I do have a few points to add to the debate,
 
all really about what cash budgeting is _not_:
 
 
 
  You can always come up with useful, fine-grained reports
 
  after-the-fact.  You don't have to make budgets complicated just to
 
  be able to run complicated reports.
 
 
 
  You can't create a budget before you have an account hierarchy.  You
 
  have to make the account hierarchy first -- really.
 
 
 
  Not every account is a leaf account.  This is good.  Having a good
 
  tree structure is what allows you to "roll up" and look at income,
 
  expenses, and budgets at different levels of detail.
 
 
 
  It's not the place of a budget to try to compensate for a poor choice of
 
  account hierarchy.
 
 
 
  Slicing "across the tree" (sometime called "2D") _is_ sometimes
 
  useful.  But this is not unique or essential to budgets.  "Action
 
  fields" address that problem and budgets shouldn't.
 
 
 
Additionally, after thinking a bit about the design requirements of a capital budgeting system, I'm convinced that even a complex BudgetCategory-style mapping to accounts in insufficient to handle the capital budgeting scenarios.  This is because the capital budgeting task is quite unstructured, and could involve "hypothetical" accounts which could have a much less concrete definition than real accounts do.
 
 
 
Incidentally, I think about 70% of the perceived need for capital budgeting functionality in Gnucash could be satisfied with one (relatively) rudimentary feature: the ability to forcast future account balances based on SX.
 
 
 
 
 
== Current Design ==
 
 
 
Two big pieces of the budgeting system had yet to be written:
 
persistence and reporting.  In addition, changing such a fundamental
 
design feature (as removing budget categories) required substantial
 
modification to the two big pieces of already-implemented budget
 
functionality: budget initialization and budget editing. 
 
 
 
Based on the intrusiveness of the required changes, and the amount of
 
additional functionality remaining to be implemented, I reluctantly
 
decided that the incremental effort of redesigning even the existing
 
portions of budgeting was small compared to the benefit of some
 
improvements to the GUI design: Darin's budget initialization was a
 
multi-page druid that was I felt was GUI-overkill and buggy anyway --
 
I replaced it with a single options dialog.  Darin's budget editing
 
was in a stand-alone "budget workbench" that I thought was too
 
disconnected from the rest of the GUI -- I replaced it with a
 
spreadsheet page in the main window.
 
 
 
As of April 2005, I am reasonably pleased with the current state
 
of my implementation.  I think it's usable and I believe that it has
 
all the functionality I need. 
 
 
 
In addition, I think my budgeting system more cleanly integrated with
 
the existing Gnucash architecture.  E.g. it uses the GncPlugin and
 
GncPluginPage APIs; it uses Gnucash's component manager API for
 
detecting modifications to budgets; it uses Gnucash's TreeViewAccount
 
widget for budget editing; it uses KVPs for hierarchical data storage,
 
etc.  There are still a few minor things I'd like to tweak, but that
 
will probably always be true.
 
 
 
 
 
== Recurrences ==
 
 
 
Even though recurrences are a _very_ small part (~ 300 LOC) of
 
budgeting, they may prove to be the most controversial, because
 
Gnucash already has FreqSpec[http://svn.gnucash.org/docs/HEAD/FreqSpec_8h.html].  I had originally used FreqSpec in my
 
design of budgets, some subtle bugs prompted me to look at the
 
implementation of FreqSpec.  I observed several things that I didn't
 
like about FreqSpec:
 
 
 
* unverifiable algorithms: I understood what the code was supposed to do but some parts (like the parts labeled "CHECK ME") were so complex that I couldn't verify by inspection that they were correct.  This was especially annoying because I knew that the correct algorithm could be expressed in a simple way.
 
 
 
* unnecessary duplication: I would say that the chief design problem with FreqSpec is that it treats each frequency type as a new case, and with different algorithms and more functions.  This is completely unwarranted and overly-complex.  A unified treatment is found in Recurrence.c
 
 
 
* GUI stuff in the engine: The comments ("Ideally this is not here, but what can you do?") indicate that someone knew this was bad, but didn't fix it.
 
 
 
* heavyweight: It sub-classed QofEntity.  My view of Recurrences was that they should be only slightly larger than simple dates, and making them QofEntities was crazy.
 
 
 
* unneeded data fields: I guess if you think every frequency type is unique, you'll find some use for unique data fields. 
 
 
 
* a long TODO list and many FIXMEs: It seemed that the design is suffering from its own complexity.  There are more things to ''fix'' in FreqSpec than ever _went_in_ to Recurrence.
 
 
 
* an ugly union: There are *way* too many fields to keep track of, and for what reason?  This is simply symptomatic of the incorrect assumption that every frequency type is fundamentally different.
 
 
 
* a poor implementation of FreqSpec composition: For much a simpler way, see Recurrence.c.
 
 
 
* misspellings in the public API: e.g. ANUALLY
 
 
 
* warnings about unmaintained functions: e.g. xaccFreqSpecSetUIType
 
 
 
* incorrect names for public API elements: e.g. UIFREQ_TRI_ANUALLY
 
 
 
* Although not really part of FreqSpec, I wanted a GUI presentation that was MUCH simpler than what existed for FreqSpec, and it looked like writing one for FreqSpec was going to be a real pain. (I'm a bit of a GUI minimalist.  see e.g. gnc-recurrence.)
 
 
 
* incompleteness: There's currently no way to specify the recurrence "every month on the 30th (or the last day of the month if there there is no 30th), beginning Feb 28, 2005" or any other recurrence of this class.  (One could argue that this is a fault with the GUI, not with FreqSpec, but, in reality, this is really kludgy to solve in the GUI and very clean to handle in Recurrence.  Strangely enough, considering the amount of code behind FreqSpec, it doesn't even capture the concept of a beginning of the recurrence -- that's left to be handled on your own.  Recurrence handles that by itself.)
 
 
 
 
 
Even though I would have '''loved''' to just use FreqSpec, I knew that I
 
could have something bug-free and verifiably correct in a few hours,
 
with 1/10th the code, and capturing more needed functionality (start
 
date) than FreqSpec did.  So I wrote Recurrence which worked fine, and
 
I never gave it another thought.
 
 
 
That is, until several months later when Derek Atkins gave me a little
 
trouble about it on IRC.  I agreed to reevaluate my decision after I
 
completed the budgeting code (which I guess is about now) and I also
 
promised to find a test case for the bug in FreqSpec before I started
 
any non-budget related hacking.
 
 
 
Well, I've re-evaluated things now, and taken another closer look at
 
FreqSpec.  In hindsight, I think it's even clearer to me that
 
Recurrence was a good idea.  FreqSpec is just plain '''complicated''', and
 
a royal pain to debug (I found that failing test case, and put it into
 
the unit test.)  It does weird stuff like measure the relative offset
 
between two dates by first measuring their relative offsets to some
 
arbitrary third date and then comparing offsets.  Plus, at the time I
 
wrote Recurrence, I didn't even realize that, even if FreqSpec hadn't
 
been quirky, it only offers half of what I wanted it to.
 
 
 
In summary, Recurrence is a much simpler design than FreqSpec and they offer more functionality than FreqSpec.  I believe that the disadvantage of having two ways to represent a recurring event is out-weighed by the advantages of completeness, correctness and simplicity offered by Recurrence.
 
 
 
= APPENDIX =
 
 
 
A collection of devel-list posts from people offering to work on budgeting:
 
 
 
"For people like me, running gnucash as a personal (family) financial
 
program, budgeting is pretty important.  Folks bother to go through the
 
pain of entering in every bloody transaction for years at a time,
 
because they want to achieve greater financial security and prosperity.
 
Planning for income and expenses, and making sure that something is left
 
over to save or invest, is really one of the main points of the exercise.
 
So I hope that these matters will continue to receive some development
 
attention." Mike Sabin <m_sabin (at) mindspring.com>
 
 
 
 
 
"Also, I'm perhaps interested in contributing to Gnucash.
 
Specifically, I want budgeting to work.  Now I'm using a silly system
 
involving a spreadsheet." Gregory Novak <greg (at) novak.cern.ch> 12 Feb
 
2001
 
 
 
 
 
"Hello all,  I've joined this list because I saw in the archives that there
 
was discussion of a "budget GUI" feature.  I currently use q*cken (please
 
excuse my language) for all my finances, but I would love to switch over to
 
gnucash, thus eliminating one more barrier to being 100% linux.
 
:)  However, my primary usage of the aforementioned windoze program right
 
now is the budgeting feature, and I really can't do without it.  How is
 
progress on the budget gui?  If help is needed, I would be glad to
 
contribute.  I have fairly extensive experience in C and C++, but no
 
experience in programming x-windows unfortunately.  But I am eager to get
 
my hands dirty."  Ansel Freniere <ansel (at) lambdares.com> 13 Feb 2001
 
 
 
 
 
"I am a reasonably experienced scheme programmer, and am strongly
 
motivated to help get at least some basic budgeting functionality up
 
and running soon.  Is there anything I can do to help the project
 
along?" Larry Hunter <hunter (at) 180glencoe.com> 30 Jun 2001
 
 
 
 
 
"Hi, I'm converting to linux and I would like to help add budgeting
 
functionality to gnucash so I can use it. I have lots of time and some
 
experience with C/C++, but i'm new to linux programming. I would like to
 
work on the design document and coding as much as possible, and I have lots
 
of ideas for budgeting based on the spreadsheet I currently use (if you want
 
to see it, just tell me and i'll send you a copy). I'll read up on the
 
current documents and discussions, but I'd like to know what to do to get to
 
work as soon as I can." Richard Garand <krogoth2 (at) softhome.net>
 
 
 
 
 
"I'm willing to help with Gnucash development.  My preferred project
 
would be budgeting since that is the weakness in Gnucash that I need
 
filled.  If that is being worked on, contact me and I'll help with
 
design, coding, testing and/or documentation.  If it is not being worked
 
on, I'll start." Phil Longstaff <plongstaff (at) newearth.org> Aug 17 2003
 
 
 
 
 
"As I have now moved completely away from Quicken to GnuCash, I would
 
like to help out.
 
 
 
My main interests (at the moment) are adding budgeting (home) and mysql (or
 
generic sql) backend.
 
 
 
The current version I am using is 1.8.9-1 from Debian/Sarge. I would
 
like to get in and get started on something, but I don't want to
 
reduplicate effort. So I guess I need which project needs the most help
 
at the moment.
 
 
 
I have experience in C/C++/FORTRAN/perl/ etc. Know some lisp. I have
 
done Xtoolkit and Motif coding. Currently I am a unix architect/admin
 
for about every flavor of unix there is (my company has never met an OS
 
it didn't like). I have also done DBA work and coding against mysql,
 
informix, sybase, db/2, and ingres. I have also managed some small web
 
servers and all email for an ISP. I see on the help wanted place on the
 
gnucash site, webmasters are needed. Perhaps I can help out in that
 
area." James Richardson <james (at) jamesrichardson.org> May 28 2004
 
 
 
= Add-On: "The Little Budget Tool" =
 
 
 
A working add-on tool has been programmed by 
 
Gabriel M. Beddingfield called "The Little Budget Tool".
 
Download the source code from http://www.teuton.org/~gabriel/lb
 
It has an optional GUI, using Qt.
 
 
 
* Basic instructions
 
* Configure, make and make install lb.
 
* Go to the example subdirectory ~/lb-1.0.0/example
 
* Examine the budget.csv file.
 
 
 
From your gnucash list of accounts make a suitable  budget.csv for yourself.                             
 
(no, we can't tell you much you should spend each month).
 
If your accounts are in the form Income:job1 Income:interest
 
and Expense:groceries  Expense:rent you won't have to tweak the settings
 
in a .lbrc file to have it work quickly.
 
 
 
== Extract your actual expenditure from gnucash ==
 
 
 
(from the man page)
 
 
 
From the accounts screen (the main window when you open GnuCash), select from
 
the top menu Reports | Income & Expense | Cash Flow.  By default it will give
 
you a summary for all the account for the entire year.  However, you probably
 
want to get a report for last month.  Or for a 4-week period corresponding to
 
your paycheck.  Here's how to tweak it for what you need:
 
 
 
1. Click the Options button.
 
 
 
2. Under the General tab, select the start and end dates for the period you are interested in.  (E.g. From: Start of previous month, To: End of  previous month.)
 
 
 
3.  Under  the Accounts tab, select the account that you want to study.  By default, it has all the Assets accounts selected.  However, for most personal  budgeting,  you  may  want to just select the "Assets:Current Assets:Checking Account."  This will track everything  going  into  and  out  of  that  account.  To unselect the other accounts, just click on  them.
 
 
 
4. Click OK.
 
 
 
5. Click the Export button.
 
 
 
6. In the "Save" dialog box, give the name and location for  where  you want  to save the report.  It's recommended to include the .html extension.  (E.g. report.html)
 
 
 
You're done.  You can run an LB report with this.
 
 
 
Running a LB report:
 
 
 
From the command line, you enter:
 
 
 
    $ lb -b budget.csv -r report.html
 
 
 
And you'll get a report that merges the data from each.
 
 
 
== Running the gui ==
 
 
 
It accepts all the same command-line arguments as lb.  You can use it like this:
 
    $ lbgui -b *.csv -r *.html
 
Then select Edit | Edit Report to tweak it.
 
 
 
== Useful things ==
 
 
 
You can run a different budget for each month (power bills come every 3 months and water bills every four months for me) and run comparisons against the actual expenditure for each month as well as doing a year to date summary.
 

Revision as of 07:05, 16 February 2007

DOH