I’ve been working hard on the data storage, management and interpretation of a user’s actual transactions that are imported from bank statements, and it’s taking quite a bit longer than I had anticipated. The initial upload went pretty smoothly, however the dissemination of said data is quite time-consuming. I actually opened up this post to upload a picture, but there are a number of discoveries that I’ll quickly drop in here to help anyone headed in this direction:
- I’m speeding up bulk imports in Rails using FasterCSV (I’m using it for the CSV exports too) and ar-extensions. The former makes importing and exporting a cinch by helping me easily parse rows, and the latter executes some SQL-fu that consolidates inserts, making bulk updates super-fast when compared to the traditional Rails method of looping and saving. I doubt I’ve scratched the surface of what ar-extensions can offer, but I’m very impressed so far as we’re dealing with a lot of records due to the granular nature of our cashflow forecasts.
- We’ve been having issues with the sqlite3 + Rails installation on Leopard, with PocketSmith throwing odd 500-code “SQLite3::SQLException: SQL logic error or missing database” errors, which disappared upon a refresh. The errors persisted for me even after James upgraded us to MySQL (which was a good move), and the more detailed errors allowed me to track it to the lack of having native C Ruby-MySQL bindings. The fix is nicely documented on Anil Kandangath’s blog.
There. Now, what does “the slide” have to do with this post?
Well, I’ve finally shaped the data well enough to pipe it to a rough dual line-graph to compare actuals to forecasts, and the first rough-cut of the graph showed me this (click on the image for an enlargement):
How low can you go?
Yes, dear reader, those really are my actuals. Albeit mapped against a fictional forecast and starting balance. However, any donations of catfood and hugs are still greatly appreciated.
I can ask for subsidies from the French government to help you Jas ;p btw James and you are awesome, you are doing a fantastic job!!!
Thanks bro. Also, I bet French catfood is pretty good.
[...] be slow, as evidenced by how quickly we’re importing bank transactions via CSV upload using ar-extensions. For the beta, we’ll stay this course and refactor to suit in the near future, if necessary. [...]