A Clojure library designed to read files from the Local filesystem, SFTP, S3, and github with a simple consistent interface.
Lein: [org.eag.file-access "0.1.1"]
Each reader takes a map of options the value of :source determines the type of file.
{:source :file :filename "test/resources/test1.edn"}
{:source :s3 :access-key <access-key> :secret-key <secret-key> :bucket <bucket> :file-key <filename>}
{:source :github :user <user> :repository <repo> :path "README.md" :branch <branch> :auth <name:password>}
{:source :sftp :port 22 :host localhost }
There is a simple slurp-file function which is identical to core.slurp with the exception that it uses these special readers. There is also a slurp-file-cmdline function which takes a tree as generated by commandline parsing.
{:s3 {:access-key foo :secret-key bar :bucket mybucket :file myfile}}
It is also possible to use the reader directly and place the output on a channel. This is a much better method for larger files.
(require [org.eag.file-access :as fa] )
(defn importer->channel
"Parse a csv file, turn it into records and put it on a channel.
Wrap this with async/thread."
[channel options columns]
(let [headers (or columns
(get-headers options))]
(with-open [rdr (fa/reader options)]
(doseq [record (map #(line->headered-map % headers)
(if columns
(read-lines rdr options)
(rest (read-lines rdr options))))]
(>!! channel record)))
(a/close! channel)))
Copyright © 2017 FIXME
Distributed under the Eclipse Public License either version 1.0 or (at your option) any later version.