Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Symbol column indexing #377

Merged
merged 61 commits into from
Oct 24, 2018
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
61 commits
Select commit Hold shift + click to select a range
d847747
start working on Symbol column indexing
iblislin Oct 6, 2018
02925a4
getindex
iblislin Oct 6, 2018
a2d3a01
deprecation
iblislin Oct 6, 2018
d39f56d
fix readwrite.jl
iblislin Oct 6, 2018
b96edec
modify: rename
iblislin Oct 6, 2018
eac66e2
apply: uniformspace
iblislin Oct 6, 2018
a222f08
fix broadcast
iblislin Oct 6, 2018
1ab8c56
fix replace_dupes
iblislin Oct 6, 2018
c119853
fix test/timearray.jl
iblislin Oct 6, 2018
4e37406
fix test/broadcast.jl
iblislin Oct 6, 2018
a8b6675
fix test/basemisc.jl
iblislin Oct 6, 2018
429a590
combine: add depwarn to `merge`
iblislin Oct 6, 2018
f945707
fix test/combine.jl
iblislin Oct 7, 2018
150940d
fix test/modify.jl
iblislin Oct 7, 2018
2454d63
fix test/split.jl
iblislin Oct 7, 2018
ccc7745
fix depwarn in test/timearray.jl
iblislin Oct 7, 2018
d6c0116
fix depwarn in basemisc
iblislin Oct 7, 2018
e87b595
fix depwarn test/broadcast.jl
iblislin Oct 7, 2018
29f16d5
fix depwarn in test/combine.jl
iblislin Oct 7, 2018
36433b9
fix depwarn in test/meta.jl
iblislin Oct 7, 2018
3d52e24
comment
iblislin Oct 7, 2018
3c52e7e
more symbol
iblislin Oct 7, 2018
02582db
support getproperty
iblislin Oct 7, 2018
3762267
fix indentation and style
iblislin Oct 8, 2018
cb6a247
support Base.propertynames
iblislin Oct 9, 2018
e44e4d8
ta.timestamp -> timestamp(ta) via sed
iblislin Oct 9, 2018
1f37a2b
ta.values -> values(ta) via sed
iblislin Oct 9, 2018
c4f7a9c
ta.colnames -> colnames(ta) via sed
iblislin Oct 9, 2018
6ad637f
ta.meta -> meta(ta) via sed
iblislin Oct 9, 2018
3f78ff0
fix broadcast.jl
iblislin Oct 9, 2018
c1e215d
fix merge function
iblislin Oct 9, 2018
fa3b03c
fix test/timearray.jl
iblislin Oct 9, 2018
d7b94ed
fix test/apply.jl
iblislin Oct 11, 2018
66c9df4
fix apply.jl
iblislin Oct 11, 2018
d49688e
fix test/basemisc.jl
iblislin Oct 11, 2018
09e0726
fix indentation in test/basemisc.jl
iblislin Oct 11, 2018
f84dac0
fix test/broadcast.jl
iblislin Oct 12, 2018
b91accb
fix combine.jl
iblislin Oct 12, 2018
cb90daa
fix test/meta.jl
iblislin Oct 12, 2018
7349f27
fix combine.jl and test/combine.jl
iblislin Oct 12, 2018
75dd518
fix test/modify.jl
iblislin Oct 12, 2018
c2c5680
fix test/readwrite.jl
iblislin Oct 12, 2018
a2495c9
fix test/split.jl
iblislin Oct 12, 2018
4ceed75
doc: getting_started.md
iblislin Oct 13, 2018
2208842
doc: timearray.md
iblislin Oct 13, 2018
7081def
doc: indexing.md
iblislin Oct 13, 2018
132e940
doc: split.md
iblislin Oct 13, 2018
fb1adb8
doc: modify.md
iblislin Oct 13, 2018
3796359
doc: apply.md
iblislin Oct 13, 2018
c7fbf6b
doc: combine.md
iblislin Oct 13, 2018
6f67f84
doc: plotting.md
iblislin Oct 13, 2018
70f5a2b
doc: dotfile.md
iblislin Oct 13, 2018
e1b29e3
doc: readwrite.md
iblislin Oct 13, 2018
48465b9
fix update ohlc
iblislin Oct 13, 2018
47363b8
style stuff
iblislin Oct 13, 2018
670ec9b
fix update function
iblislin Oct 13, 2018
17fef95
doc/dotfile.md: correct edit command
iblislin Oct 13, 2018
5d1dcc7
using `===` (`\euqiv`) in case of singleton
iblislin Oct 17, 2018
9262ce7
fix indentation
iblislin Oct 17, 2018
1afaadd
drop Memoize
iblislin Oct 20, 2018
1987ea9
update NEWS
iblislin Oct 22, 2018
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions .travis.yml
Original file line number Diff line number Diff line change
@@ -1,11 +1,15 @@
language: julia

os:
- linux

julia:
- 0.7
- 1.0

notifications:
email: false

sudo: false

after_success:
Expand Down
44 changes: 44 additions & 0 deletions NEWS.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,47 @@
### 0.14.0

* Symbol column indexing. ([#377])
And the String indexing is deprecated.

E.g.

```julia
using MarketData
ohlc[:Close] # and cl["Close"] is deprecated
```

* `Base.getproperty` support. ([#377])
All the columns can be accessed via the form `ta.column_name`.

E.g.

```julia
using MarketData
ohlc.Open
```

The original `TimeArray` fields getters is available as functions.

* `timestamp(::TimeArray)`
* `values(::TimeArray)`
* `colnames(::TimeArray)`
* `meta(::TimeArray)`

```julia
ohlc.values # this is unavailable due to Base.getproperty support
values(ohlc) # change to this
```

[#377]: https://github.com/JuliaStats/TimeSeries.jl/pull/377


### 0.13.0

* Julia v0.7/1.0 support. ([#370])

[#370]: https://github.com/JuliaStats/TimeSeries.jl/pull/370


### 0.12.0

* Support `Base.copy(::TimeArray)`. ([#352])
Expand Down
1 change: 1 addition & 0 deletions docs/src/apply.md
Original file line number Diff line number Diff line change
Expand Up @@ -135,6 +135,7 @@ In our moving average example, we would pass arguments this way:
```@repl
using TimeSeries
using MarketData
using Statistics
moving(mean, cl, 10)
```

Expand Down
10 changes: 6 additions & 4 deletions docs/src/combine.md
Original file line number Diff line number Diff line change
Expand Up @@ -85,6 +85,7 @@ We can also supply the function that chooses the timestamp and the
function that determines the corresponding value independently:

```@repl collapse
using Statistics
collapse(cl, month, last, mean)
```

Expand All @@ -104,8 +105,8 @@ For example:

```@repl
using TimeSeries
a = TimeArray([Date(2015, 10, 01), Date(2015, 11, 01)], [15, 16], ["Number"])
b = TimeArray([Date(2015, 12, 01)], [17], ["Number"])
a = TimeArray([Date(2015, 10, 01), Date(2015, 11, 01)], [15, 16])
b = TimeArray([Date(2015, 12, 01)], [17])
[a; b]
```

Expand All @@ -124,6 +125,7 @@ series, here by one year:

```@repl
using TimeSeries
a = TimeArray([Date(2015, 10, 01), Date(2015, 11, 01)], [15, 16], ["Number"])
map((timestamp, values) -> (timestamp + Dates.Year(1), values), a)
using Dates
ta = TimeArray([Date(2015, 10, 01), Date(2015, 11, 01)], [15, 16])
map((timestamp, values) -> (timestamp + Year(1), values), ta)
```
4 changes: 3 additions & 1 deletion docs/src/dotfile.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,9 @@ and only controls how values are printed to REPL.
Here is an handy way to edit it:

```julia
julia> edit(Pkg.dir("TimeSeries", "src", ".timeseriesrc.jl"))
julia> using TimeSeries

julia> edit(joinpath(dirname(pathof(TimeSeries)), ".timeseriesrc.jl"))
```

## `DECIMALS`
Expand Down
7 changes: 4 additions & 3 deletions docs/src/getting_started.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,8 @@ the following code block:
using TimeSeries
```

```@example dummy
dates = collect(Date(1999,1,1):Date(2000,12,31))
mytime = TimeArray(dates, rand(length(dates)))
```@repl dummy
using Dates
dates = Date(2018, 1, 1):Day(1):Date(2018, 12, 31)
ta = TimeArray(dates, rand(length(dates)))
```
73 changes: 47 additions & 26 deletions docs/src/indexing.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,67 +2,88 @@

Indexing out a time series is done with common bracketing semantics.

## Integers
## Row indexing

### Integer

| Example | Description | Indexing value |
|--------------|---------------------------------------|--------------------------------|
| `[1]` | First row of data only | single integer |
| `[1:3]` | First through third row only | integer range |
| `[1:2:10]` | Odd row between first to tenth row | integer range with step |
| `[[1:3; 8]]` | First through third row and eight row | integer range & single integer |
| `[end]` | Last row | |

Examples in REPL:

```@repl
```@setup int-indexing
using MarketData
```

```@repl int-indexing
ohlc[1]
ohlc[1:3]
ohlc[1:2:10]
ohlc[[1:3;8]]
ohlc[end]
```

## Strings
### Date and DateTime

| Example | Description | Indexing value |
|---------------------|--------------------------------------|------------------|
| `["Open"]` | The column named "Open" | single string |
| `["Open", "Close"]` | The columns named "Open" and "Close" | multiple strings |
| Example | Description | Indexing value |
|----------------------------------------------|--------------------------------------------|----------------|
| `[Date(2000, 1, 3)]` | The row containing Jan 3, 2000 timestamp | single Date |
| `[[Date(2000, 1, 3), Date(2000, 2, 4)]]` | The rows containing Jan 3 & Jan 4, 2000 | multiple Dates |
| `[Date(2000, 1, 3):Day(1):Date(2000, 2, 4)]` | The rows between Jan 1, 2000 & Feb 1, 2000 | range of Date |

Examples in REPL:

```@repl
```@setup date-indexing
using MarketData
ohlc["Open"]
ohlc["Open", "Close"]
using Dates
```

```@repl date-indexing
ohlc[Date(2000, 1, 3)]
ohlc[Date(2000, 1, 3):Day(1):Date(2000, 2, 4)]
```

# Date and DateTime
## Column indexing

| Example | Description | Indexing value |
|------------------------------------------|--------------------------------------------|----------------|
| `[Date(2000, 1, 3)]` | The row containing Jan 3, 2000 timestamp | single Date |
| `[[Date(2000, 1, 3), Date(2000, 2, 4)]]` | The rows containing Jan 3 & Jan 4, 2000 | multiple Dates |
| `[Date(2000, 1, 3):Date(2000, 2, 4)]` | The rows between Jan 1, 2000 & Feb 1, 2000 | range of Date |
### Symbol

| Example | Description | Indexing value |
|-------------------|----------------------------------------|------------------|
| `[:Open]` | The column named `:Open` | single symbol |
| `[:Open, :Close]` | The columns named `:Open` and `:Close` | multiple symbols |

Examples in REPL:

```@repl
```@setup symbol-indexing
using MarketData
ohlc[Date(2000, 1, 3)]
ohlc[Date(2000, 1, 3):Date(2000, 2, 4)]
using Dates
```

```@repl symbol-indexing
ohlc[:Open]
ohlc[:Open, :Close]
```

## Mixed approach

| Example | Description | Indexing value |
|------------------------------|-------------------------------|-------------------------------|
| `["Open"][1:3]` | "Open" column & first 3 rows | single string & integer range |
| `["Open"][Date(2000, 1, 3)]` | "Open" column and Jan 3, 2000 | single string & Date |
| Example | Description | Indexing value |
|-----------------------------|--------------------------------|-------------------------------|
| `[:Open][1:3]` | `:Open` column & first 3 rows | single symbol & integer range |
| `[:Open][Date(2000, 1, 3)]` | `:Open` column and Jan 3, 2000 | single symbol & Date |

Examples in REPL:

```@repl
```@setup mixed-indexing
using MarketData
ohlc["Open"][1:3]
ohlc["Open"][Date(2000, 1, 3)]
using Dates
```

```@repl mixed-indexing
ohlc[:Open][1:3]
ohlc[:Open][Date(2000, 1, 3)]
```
10 changes: 5 additions & 5 deletions docs/src/modify.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# Modify existing `TimeArrays`
# Modify existing `TimeArray`s

Since `TimeArrays` are immutable, they cannot be altered or changed
in-place. In practical application, an existing `TimeArray` might need to
Expand All @@ -18,7 +18,7 @@ using TimeSeries
using MarketData
update(cl, Date(2002,1,1), 111.11)
update(cl, Date(2002,1,1), [111.11])
update(ohlc, Date(2002,1,1), [111.11, 222.22, 333.33, 444.44])
update(ohlc, Date(2002,1,1), [111.11 222.22 333.33 444.44])
```

## `rename`
Expand All @@ -28,7 +28,7 @@ The `rename` method allows the column name(s) to be changed:
```@repl
using TimeSeries
using MarketData
rename(cl, "New Close")
rename(cl, ["New Close"])
rename(ohlc, ["New Open", "New High", "New Low", "New Close"])
rename(cl, :Close)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is the ' here intentional?

Copy link
Contributor

@femtotrader femtotrader Oct 16, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(not ') is used several times in https://github.com/JuliaStats/TimeSeries.jl/blob/master/src/broadcast.jl

I was surprised by the use of such a character also which can't be found easily on my AZERTY keyboard

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Those render exactly the same for me. Weird.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why not naming simply :NewClose (or :Close2)?

What are the english name of , ' and `

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See screenshot to see how it's rendering here (France)
capture d ecran 2018-10-16 a 22 39 24

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure about the first one, the second is "apostrophe," and the third is "backtick" (assuming that's rendering for me properly).

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's :Close\prime<tab>

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why not naming simply :NewClose (or :Close2)?

well, I personally love the unicode support in Julia.
I always write code (in my research project) like x = 1; x\prime<tab> = x + 42
or
x\vec<tab> ...

julia> x = 1
1

julia> x′ = x + 42
43

julia> function ϕ()
        42
       end 
ϕ (generic function with 1 method)

julia> :Close₂                                                   

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, I see. I was on Linux before when it rendered like an apostrophe for me, but on macOS they're distinct.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ararslan maybe you should open an issue on your Linux distribution bug tracker because that's quite problematic to have 2 differents characters rendered the same.

rename(cl, [:Close])
rename(ohlc, [:Open′, :High′, :Low′, :Close])
```
2 changes: 1 addition & 1 deletion docs/src/plotting.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ More sophisticated plots can be created by using keyword attributes and
subsets:

```julia
plot(MarketData.ohlc["Low"], seriestype = :scatter, markersize = 3, color = :red, markeralpha = 0.4, grid = true)
plot(MarketData.ohlc[:Low], seriestype = :scatter, markersize = 3, color = :red, markeralpha = 0.4, grid = true)
```

![image](images/complexplot.svg)
Expand Down
4 changes: 2 additions & 2 deletions docs/src/readwrite.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ Reading/writing a `csv` file into a `TimeArray` object is supported.

## `readtimearray`

The `readtimearray` method is a wrapper for the `Base.readcsv` method
The `readtimearray` method is a wrapper for the `DelimitedFiles.readdlm` method
that returns a `TimeArray`.

```julia
Expand All @@ -16,7 +16,7 @@ of the `csv` file that you wish to parse into a `TimeArray` object.
Optionally, you can add a value to the `meta` field.

More generally, this function accepts arbitrary delimiters with `delim`,
just like `Base.readcsv`.
just like `DelimitedFiles.readdlm`.

For `DateTime` data that has odd formatting, a `format` argument is
provided where users can pass the format of their data.
Expand Down
10 changes: 5 additions & 5 deletions docs/src/split.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# Splitting by time constraint or when condition is true
# Splitting by conditions

Specific methods for segmenting on time ranges or if condition is met is
supported with the following methods.
Expand Down Expand Up @@ -65,21 +65,21 @@ The `findwhen` method test a condition and returns a vector of `Date` or
using TimeSeries
using MarketData

green = findwhen(ohlc["Close"] .> ohlc["Open"]);
green = findwhen(ohlc[:Close] .> ohlc[:Open]);
typeof(green)
ohlc[green]
```

## `find`
## `findall`

The `find` method tests a condition and returns a vector of `Int`
The `findall` method tests a condition and returns a vector of `Int`
representing the row in the array where the condition is `true`:

```@repl
using TimeSeries
using MarketData

red = find(ohlc["Close"] .< ohlc["Open"]);
red = findall(ohlc[:Close] .< ohlc[:Open]);
typeof(red)
ohlc[red]
```
Expand Down
18 changes: 14 additions & 4 deletions docs/src/timearray.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,10 +4,10 @@ The `TimeArray` time series type is defined here
(with inner constructor code removed for readability):

```julia
struct TimeArray{T,N,D<:TimeType,A<:AbstractArray} <: AbstractTimeSeries
struct TimeArray{T,N,D<:TimeType,A<:AbstractArray{T,N}} <: AbstractTimeSeries{T,N,D}
timestamp::Vector{D}
values::A # some kind of AbstractArray{T,N}
colnames::Vector{String}
colnames::Vector{Symbol}
meta::Any

# inner constructor code enforcing invariants
Expand Down Expand Up @@ -37,7 +37,7 @@ be of the same type.

## `colnames`

The `colnames` field is a vector of type `String` and contains the
The `colnames` field is a vector of `Symbol` and contains the
names of the columns for each column in the `values` field. The length
of this vector must match the column count of the `values` array, or the
constructor will fail. Since `TimeArrays` are indexable on column names,
Expand All @@ -48,8 +48,18 @@ where `n` enumerates from 1.
## `meta`

The `meta` field defaults to holding `nothing`, which is represented by
type `Void`. This default is designed to allow programmers to ignore
type `Nothing`. This default is designed to allow programmers to ignore
this field. For those who wish to utilize this field, `meta` can hold
common types such as `String` or more elaborate user-defined types. One
might want to assign a name to an object that is immutable versus
relying on variable bindings outside of the object's type fields.

## Fields getter functions

There are four field getter functions exported.
They are named as same as the field names.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Field names are effectively an implementation detail, so I don't know that it's worth mentioning that. Good to document these functions though.

Copy link
Collaborator Author

@iblislin iblislin Oct 17, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the field name is revealed to user already here:
http://juliastats.github.io/TimeSeries.jl/latest/timearray.html

So I added that section.


- `timestamp(ta::TimeArray)`
- `values(ta::TimeArray)`
- `colnames(ta::TimeArray)`
- `meta(ta::TimeArray)`
Loading