Skip to content
This repository has been archived by the owner on Feb 25, 2022. It is now read-only.

Use official ynab api and drop selenium webdriver #9

Merged
merged 16 commits into from
Jul 21, 2018
4 changes: 4 additions & 0 deletions .codeclimate.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
plugins:
rubocop:
enabled: true
channel: rubocop-0-52
6 changes: 6 additions & 0 deletions .dockerignore
Original file line number Diff line number Diff line change
Expand Up @@ -8,3 +8,9 @@ config.sample.yml
.cache
.export
last_imported

.codeclimate.yml
.rubocop.yml
.rubocop_todo.yml
.rspec
spec
2 changes: 2 additions & 0 deletions .rspec
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
--require spec_helper
--color
3 changes: 3 additions & 0 deletions .rubocop.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
inherit_from: .rubocop_todo.yml

require: rubocop-rspec
18 changes: 18 additions & 0 deletions .rubocop_todo.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
# This configuration was generated by
# `rubocop --auto-gen-config`
# on 2018-02-25 01:20:19 +0100 using RuboCop version 0.52.1.
# The point is for the user to remove these configuration records
# one by one as the offenses are removed from the code base.
# Note that changes in the inspected code, or installation of new
# versions of RuboCop, may require this file to be generated again.

# Offense count: 1
# Configuration parameters: CountComments, ExcludedMethods.
Metrics/BlockLength:
Max: 111

# Offense count: 1
# Cop supports --auto-correct.
Style/BlockComments:
Exclude:
- 'spec/spec_helper.rb'
15 changes: 15 additions & 0 deletions .travis.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
language: ruby
cache: bundler
rvm:
- 2.4.2
- 2.5.0
env:
- ENV=test
before_script:
- curl -L https://codeclimate.com/downloads/test-reporter/test-reporter-latest-linux-amd64 > ./cc-test-reporter
- chmod +x ./cc-test-reporter
- ./cc-test-reporter before-build
script:
- bundle exec rspec
after_script:
- ./cc-test-reporter after-build --exit-code $TRAVIS_TEST_RESULT
7 changes: 1 addition & 6 deletions Dockerfile
Original file line number Diff line number Diff line change
@@ -1,6 +1,4 @@
FROM ruby:2.4.1

MAINTAINER Martin Schurig <[email protected]>
FROM ruby:2.5.0

# Set the locale
RUN apt-get clean && apt-get update && apt-get install -y locales
Expand All @@ -11,9 +9,6 @@ ENV LANG=en_US.UTF-8 \
LANGUAGE=en_US:en \
LC_ALL=en_US.UTF-8

RUN git clone https://github.com/schurig/ruby_fints.git && cd ruby_fints && git checkout ing-diba && gem build ruby_fints.gemspec && gem install ruby_fints-0.0.1.gem
# original: https://github.com/playtestcloud/ruby_fints.git

RUN mkdir /usr/app
WORKDIR /usr/app

Expand Down
7 changes: 1 addition & 6 deletions Dockerfile-rpi
Original file line number Diff line number Diff line change
@@ -1,10 +1,8 @@
FROM knjcode/rpi-ruby

MAINTAINER Martin Schurig <[email protected]>

RUN apt-get update && \
apt-get -y install \
gcc mono-mcs build-essential \
build-essential \
locales && \
rm -rf /var/lib/apt/lists/*

Expand All @@ -16,9 +14,6 @@ ENV LANG=en_US.UTF-8 \
LANGUAGE=en_US:en \
LC_ALL=en_US.UTF-8

RUN git clone https://github.com/schurig/ruby_fints.git && cd ruby_fints && git checkout ing-diba && gem build ruby_fints.gemspec && gem install ruby_fints-0.0.1.gem
# original: https://github.com/playtestcloud/ruby_fints.git

RUN mkdir /usr/app
WORKDIR /usr/app

Expand Down
21 changes: 15 additions & 6 deletions Gemfile
Original file line number Diff line number Diff line change
@@ -1,18 +1,27 @@
source 'https://rubygems.org'

gem 'httparty'
gem 'ynab', '~> 1.2.0'

# Dumper: BBVA
gem 'bankscrap'
gem 'bankscrap-bbva', '~> 2.0.2'

# Dumper: N26
gem 'twentysix'

# # Dumper: FinTS
# gem 'ruby_fints', git: 'https://github.com/schurig/ruby_fints.git', branch: 'ing-diba'
# # original https://github.com/playtestcloud/ruby_fints.git
# Dumper: FinTS
gem 'ruby_fints', '0.0.3'

gem 'base32'

gem 'selenium-webdriver'
gem 'nokogiri', '~> 1.8.1'
# included in selenium but because of a security risk it's set to 1.8.1
group :development do
gem 'rubocop', '~> 0.52.1', require: false
gem 'rubocop-rspec'
end

group :test do
gem 'rspec'
gem 'vcr'
gem 'webmock'
end
90 changes: 73 additions & 17 deletions Gemfile.lock
Original file line number Diff line number Diff line change
@@ -1,11 +1,14 @@
GEM
remote: https://rubygems.org/
specs:
activesupport (5.1.4)
activesupport (5.1.5)
concurrent-ruby (~> 1.0, >= 1.0.2)
i18n (~> 0.7)
minitest (~> 5.1)
tzinfo (~> 1.1)
addressable (2.5.2)
public_suffix (>= 2.0.2, < 4.0)
ast (2.4.0)
bankscrap (2.0.6)
activesupport
mechanize
Expand All @@ -15,19 +18,26 @@ GEM
bankscrap-bbva (2.0.2)
bankscrap (~> 2.0.3)
base32 (0.3.2)
childprocess (0.8.0)
ffi (~> 1.0, >= 1.0.11)
cmxl (0.2.1)
rchardet19
concurrent-ruby (1.0.5)
crack (0.4.3)
safe_yaml (~> 1.0.0)
deep_merge (1.2.1)
diff-lcs (1.3)
domain_name (0.5.20170404)
unf (>= 0.0.5, < 1.0.0)
ffi (1.9.18)
ethon (0.11.0)
ffi (>= 1.3.0)
ffi (1.9.25)
hashdiff (0.3.7)
http-cookie (1.0.3)
domain_name (~> 0.5)
httparty (0.15.6)
httparty (0.16.0)
multi_xml (>= 0.5.2)
i18n (0.9.1)
i18n (0.9.5)
concurrent-ruby (~> 1.0)
json (2.1.0)
mechanize (2.7.5)
domain_name (~> 0.5, >= 0.5.1)
http-cookie (~> 1.0)
Expand All @@ -41,31 +51,71 @@ GEM
mime-types-data (~> 3.2015)
mime-types-data (3.2016.0521)
mini_portile2 (2.3.0)
minitest (5.10.3)
minitest (5.11.3)
money (6.10.1)
i18n (>= 0.6.4, < 1.0)
multi_xml (0.6.0)
net-http-digest_auth (1.4.1)
net-http-persistent (2.9.4)
nokogiri (1.8.1)
nokogiri (1.8.2)
mini_portile2 (~> 2.3.0)
ntlm-http (0.1.1)
rubyzip (1.2.1)
selenium-webdriver (3.8.0)
childprocess (~> 0.5)
rubyzip (~> 1.0)
parallel (1.12.1)
parser (2.5.0.2)
ast (~> 2.4.0)
powerpack (0.1.1)
public_suffix (3.0.2)
rainbow (3.0.0)
rchardet19 (1.3.7)
rspec (3.7.0)
rspec-core (~> 3.7.0)
rspec-expectations (~> 3.7.0)
rspec-mocks (~> 3.7.0)
rspec-core (3.7.1)
rspec-support (~> 3.7.0)
rspec-expectations (3.7.0)
diff-lcs (>= 1.2.0, < 2.0)
rspec-support (~> 3.7.0)
rspec-mocks (3.7.0)
diff-lcs (>= 1.2.0, < 2.0)
rspec-support (~> 3.7.0)
rspec-support (3.7.1)
rubocop (0.52.1)
parallel (~> 1.10)
parser (>= 2.4.0.2, < 3.0)
powerpack (~> 0.1)
rainbow (>= 2.2.2, < 4.0)
ruby-progressbar (~> 1.7)
unicode-display_width (~> 1.0, >= 1.0.1)
rubocop-rspec (1.23.0)
rubocop (>= 0.52.1)
ruby-progressbar (1.9.0)
ruby_fints (0.0.3)
cmxl (~> 0.2)
httparty (~> 0.10)
safe_yaml (1.0.4)
thor (0.20.0)
thread_safe (0.3.6)
twentysix (0.1.1)
deep_merge
httparty
tzinfo (1.2.4)
typhoeus (1.3.0)
ethon (>= 0.9.0)
tzinfo (1.2.5)
thread_safe (~> 0.1)
unf (0.1.4)
unf_ext
unf_ext (0.0.7.4)
unf_ext (0.0.7.5)
unicode-display_width (1.3.0)
vcr (4.0.0)
webmock (3.3.0)
addressable (>= 2.3.6)
crack (>= 0.3.2)
hashdiff
webrobots (0.1.2)
ynab (1.2.0)
json (~> 2.1, >= 2.1.0)
typhoeus (~> 1.0, >= 1.0.1)

PLATFORMS
ruby
Expand All @@ -74,9 +124,15 @@ DEPENDENCIES
bankscrap
bankscrap-bbva (~> 2.0.2)
base32
nokogiri (~> 1.8.1)
selenium-webdriver
httparty
rspec
rubocop (~> 0.52.1)
rubocop-rspec
ruby_fints (= 0.0.3)
twentysix
vcr
webmock
ynab (~> 1.2.0)

BUNDLED WITH
1.16.0
1.16.1
46 changes: 12 additions & 34 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,21 +1,21 @@
# Bank YNAB importer [![Docker Build Statu](https://img.shields.io/docker/pulls/schurig/ynab-bank-importer.svg)](https://hub.docker.com/r/schurig/ynab-bank-importer/) [![Docker Build Statu](https://img.shields.io/docker/build/schurig/ynab-bank-importer.svg)](https://hub.docker.com/r/schurig/ynab-bank-importer/builds/)
# Bank YNAB importer [![Docker Build Statu](https://img.shields.io/docker/pulls/schurig/ynab-bank-importer.svg)](https://hub.docker.com/r/schurig/ynab-bank-importer/) [![Docker Build Statu](https://img.shields.io/docker/build/schurig/ynab-bank-importer.svg)](https://hub.docker.com/r/schurig/ynab-bank-importer/builds/) [![Maintainability](https://api.codeclimate.com/v1/badges/4367cde9c1b522b4bcbe/maintainability)](https://codeclimate.com/github/schurig/ynab-bank-importer/maintainability)

This is a ruby script that **pulls your transactions from your banks** and imports them into **You Need A Budget** (YNAB).

## Supported banks

* Most German and Austrian banks _(all banks that [figo.io](https://figo.io) or that implement the FinTS standard)_
* Most German and Austrian banks _(all banks that implement the FinTS standard)_
* BBVA Spain _(private accounts only)_
* N26

## Why

YNAB only supports U.S. and Canadian Banks for now.

_If you're someone from the YNAB-Team: please add a public api-endpoint for an easier import of transactions. ❤️_

## Usage

You will need to obtain a personal access token. [Here is a tutorial on how to do it](https://api.youneedabudget.com/#personal-access-tokens).

**If you clone this repository you don't need to follow step 2!**

1. Install docker and docker-compose
Expand All @@ -27,15 +27,13 @@ _If you're someone from the YNAB-Team: please add a public api-endpoint for an e
```yaml
---
ynab:
username: # email
password: # password
access_token: # ynab access token
budget_id: # budget_id
cash_account_name: # optional
cash_account_id: # optional
accounts:
- dumper: :n26
iban: # iban of your n26
ynab_id: # account id in YNAB
ynab_name: # account name in YNAB
username: # email
password: # password
```
Expand Down Expand Up @@ -72,7 +70,7 @@ The FinTS / HBCI standard is mainly implemented by German banks.

#### Note

* It currently only fetches the last 100 transactions. Please open an issue or PR if you want to change this.
* It currently only fetches the last 100 transactions. Please open an issue or PR if you want to change this. For now I didn't see the use case for it.

#### Options

Expand All @@ -84,37 +82,17 @@ The FinTS / HBCI standard is mainly implemented by German banks.

The field `payee` will be `N/A` because we currently don't get the payee name.

### Figo `:figo`

You need to get a [figo.io](https://figo.io) account first.

#### Options

* `force_download` _(default: false)_

> Since there is a high chance that you use the `:figo` dumper more than once, all transactions from figo will be downloaded once and be cached thoughout the run.
>
> _If you want to turn off this behavior add the option `force_download: true`._

# Technical details on how it works

On [app.youneedabudget.com](https://app.youneedabudget.com) you can upload a `.csv` file with your transactions. The structure looks like this:
The script fetches the transaction information from your bank(s). Then it uses the official YNAB API to create the transactions for you.

```csv
Date,Payee,Category,Memo,Outflow,Inflow
2017-08-13,METRO BARCELONA,Transport & Car,Ticket for the Beach,"",-9.95
```
The script also includes some additional logic like detecting internal transactions by checking if the account transactions go to or come from is one of the other accounts that you set up in the config _(of course this only works if you have multiple accounts configured)_.

*So we need this information for each transaction:*
# Known Problems

* Date _(YYYY-MM-DD)_
* Payee
* Category _(optional)_
* Outflow _(optional *if `Inflow` also uses negative numbers*)_
* Inflow
* Internal transactions _(transfer from one account to an other one)_ don't work yet. This is because the official YNAB API doesn't support the creation of internal transactions yet. **Workaround:** the script flags those transactions in orange, so you can quickly see them and manually edit them to be an internal transaction.

The script fetches the transaction information from your bank and puts it into the format mentioned above.
It exports the result of each individual account in a file. Then it uses selenium-webdriver Chrome to login to your YNAB account, navigates to the right account and imports it for you.
* With N26 it could import a transactions twice from time to time. This is a problem on N26's side, because they change the id of the transaction sometimes. **Workaround:** when you see a transaction showing up twice, you can discard it. It's easy to spot because the payee, date and the price are usually the same.

____________________

Expand Down
Loading