Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

engine: implement execution engine #179

Merged
merged 81 commits into from
Jul 12, 2020
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
81 commits
Select commit Hold shift + click to select a range
7ec53fa
Remove executor
tsatke Jun 5, 2020
6bc9244
Introduce engine
tsatke Jun 5, 2020
358fa9f
Improve godoc
tsatke Jun 5, 2020
5cde671
Fix typo
tsatke Jun 5, 2020
07c9ebc
Add a few decoding function to the converter
tsatke Jun 6, 2020
a06d5a9
Add a page abstraction
tsatke Jun 6, 2020
893a23c
Add a slotted page implementation
tsatke Jun 6, 2020
0151cfd
Add delete and get method to page
tsatke Jun 6, 2020
9d5644d
Create blackbox tests
tsatke Jun 6, 2020
dffe4b6
Add test and support for deletion of cells
tsatke Jun 6, 2020
bb06a43
Add missing godoc
tsatke Jun 6, 2020
089e19e
Make cell count internal and add a dirty flag
tsatke Jun 7, 2020
7ae4eed
Add a page mock
tsatke Jun 7, 2020
457d4f3
Implement a simple LRU page cache
tsatke Jun 9, 2020
3dbee4e
Remove current page implementation
tsatke Jun 16, 2020
1cee316
Add v1 page implementation without store functionality
tsatke Jun 16, 2020
ca08f13
Remove converter as it's obsolete
tsatke Jun 18, 2020
ff1faf0
Add storage functionality
tsatke Jun 18, 2020
80895af
Rename offset to slot
tsatke Jun 19, 2020
b9c1e99
Make store functionality work (with tests)
tsatke Jun 19, 2020
5f5952f
Add a page loader
tsatke Jun 19, 2020
b53750a
Implement and specify part of the file format
tsatke Jun 22, 2020
0123d40
Add documentation
tsatke Jun 22, 2020
6d1959a
Implement page defragmentation
tsatke Jun 23, 2020
91356c0
Test cell deletion
tsatke Jun 23, 2020
c18ffa5
Only flush if page is dirty
tsatke Jun 23, 2020
154327f
Add more validating functions
tsatke Jun 23, 2020
2a26df8
Add type system for runtime
tsatke Jun 23, 2020
cb798c1
Add pageCache to engine
tsatke Jun 23, 2020
12a388b
Add cache tests
tsatke Jun 23, 2020
16e034a
Define result
tsatke Jun 23, 2020
d92ef6e
Add a simple profiler for the engine
tsatke Jun 24, 2020
fe28198
Implement a type system
tsatke Jun 24, 2020
ca0ab8c
Add basic profiling options
tsatke Jun 24, 2020
5f99184
Add builtin functions and use profiling
tsatke Jun 24, 2020
09f8a9e
Add testing framework for parser->engine e2e tests
tsatke Jun 24, 2020
350d323
Add a date type
tsatke Jun 25, 2020
0e86523
Fix usage and evaluation of function values
tsatke Jun 25, 2020
13cbc34
Refactor the type system
tsatke Jun 25, 2020
ffff33a
Add godoc
tsatke Jun 25, 2020
61c789f
Improve godoc
tsatke Jun 25, 2020
db8d556
Add doc on page layout
tsatke Jun 26, 2020
7b9cfb4
Add info about cells
tsatke Jun 26, 2020
8b66d7f
Test
tsatke Jun 26, 2020
c2eff01
Test
tsatke Jun 26, 2020
a7d49e2
Test
tsatke Jun 26, 2020
c2b8337
Test
tsatke Jun 26, 2020
30124c3
Clean up type system
tsatke Jun 29, 2020
6328d57
Move string tests to value tests
tsatke Jun 29, 2020
daad9b5
Add godoc
tsatke Jun 29, 2020
bd883a8
Improve on the type system and run go generate
tsatke Jul 1, 2020
0c33953
Add literal parser and support for numeric literal
tsatke Jul 1, 2020
161fb04
Create tests to reflect requirements of #188
tsatke Jul 1, 2020
efb21d6
Fix #188
tsatke Jul 1, 2020
ecf1e05
Add more godoc and another test for builtin max
tsatke Jul 2, 2020
78382e7
Introduce a string representation for profiles
tsatke Jul 2, 2020
8c3ea22
Create an example test case with profiling
tsatke Jul 2, 2020
11fa3ac
Add documentation on profile package
tsatke Jul 2, 2020
4f27111
Implement random function
tsatke Jul 2, 2020
c86da7f
Add example tests with engine options
tsatke Jul 2, 2020
a026b24
Introduce a config page
tsatke Jul 8, 2020
86d68b5
Introduce NULLs into Comparator implementations
tsatke Jul 8, 2020
0c85613
Make tests also compare a Go string
tsatke Jul 9, 2020
0e9a36e
Remove obsolete length check of parser errors
tsatke Jul 9, 2020
c7a0170
Merge package ID from branch raft
tsatke Jul 10, 2020
03257a2
Merge branch 'master' into engine
tsatke Jul 10, 2020
164f492
Implement projection
tsatke Jul 10, 2020
e14c42b
Add godoc
tsatke Jul 10, 2020
eb29cdc
Update internal/engine/error.go
tsatke Jul 10, 2020
4f64a98
Remove todos
tsatke Jul 10, 2020
05d1103
Merge branch 'engine' of github.com:tomarrell/lbadd into engine
tsatke Jul 10, 2020
cc49763
Remove database package
tsatke Jul 10, 2020
f023d21
Update internal/engine/expression.go
tsatke Jul 10, 2020
1427b99
Update internal/engine/numeric_parser.go
tsatke Jul 10, 2020
6be3efb
Update internal/engine/expression.go
tsatke Jul 10, 2020
a35032e
Merge branch 'master' into engine
tsatke Jul 10, 2020
1cb364b
Fix test failure
tsatke Jul 10, 2020
a16a3bf
Merge branch 'master' into engine
tsatke Jul 10, 2020
7c2c4b2
Embed a pointer
tsatke Jul 10, 2020
15a41cf
go mod tidy
tsatke Jul 10, 2020
8a51bd4
Update internal/engine/storage/db_test.go
tsatke Jul 10, 2020
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 1 addition & 13 deletions cmd/lbadd/main.go
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,6 @@ import (
"github.com/rs/zerolog"
"github.com/rs/zerolog/diode"
"github.com/spf13/cobra"
"github.com/tomarrell/lbadd/internal/executor"
"github.com/tomarrell/lbadd/internal/node"
)

Expand Down Expand Up @@ -157,9 +156,7 @@ func startNode(cmd *cobra.Command, args []string) {
Str("dbfile", databaseFile).
Logger()

exec := createExecutor(log, databaseFile)

node := node.New(nodeLog, exec)
node := node.New(nodeLog)
if err := node.ListenAndServe(cmd.Context(), addr); err != nil {
log.Error().
Err(err).
Expand Down Expand Up @@ -206,12 +203,3 @@ func createLogger(stdin io.Reader, stdout, stderr io.Writer) zerolog.Logger {

return log
}

func createExecutor(log zerolog.Logger, databaseFile string) executor.Executor {
execLog := log.With().
Str("component", "executor").
Logger()

exec := executor.New(execLog, databaseFile)
return exec
}
Binary file added doc/cell_v1.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
73 changes: 73 additions & 0 deletions doc/file-format.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,73 @@
# File Format
This document describes the v1.x file format of a database.

## Terms
This section will quickly describe the terms, that will be used throughout this
document.

A **database** is a single file, that holds all information of a single
database.

## Format
The database is a single file. Its size is a multiple of the page size, which is
16K or 16384 bytes for v1.x. The file consists of pages only, meaning there is
no fixed size header, only a header page (and maybe overflow pages).

### Header page
The page with the **ID 0** is the header page of the whole database. It holds
values for the following keys. The keys are given as strings, the actual key
bytes are the UTF-8 encoding of that string.

* `pageCount` is a record cell whose entry is an 8 byte big endian unsigned
integer, representing the amount of pages stored in the file.
* `config` is a pointer cell which points to a page, that contains configuration
parameters for this database.
* `tables` is a pointer cell which points to a page, that contains pointers to
all tables that are stored in this database. The format of the table pages is
explained in the next section.

### Table pages
Table pages do not directly hold data of a table. Instead, they hold pointers to
pages, that do, i.e. the index and data page. Table pages do however hold
information about the table data definition. The data definition information is
a single record that is to be interpreted as a data definition (<span
style="color:red;">**TODO: data definitions**</span>).

The keys of the three values, index page, data page and schema are as follows.

* `datadefinition` is a record cell containing the schema information about this
table. That is, columns, column types, references, triggers etc. How the
schema information is to be interpreted, is explained
[here](#data-definition).
* `index` is a pointer cell pointing to the index page of this table. The index
page points to pages that are an actual index in the table. See more
[here](#index-pages)
* `data` is a pointer cell pointing to the data page of this table. See more
[here](#data-pages)

### Index page

### Data pages
A data page stores plain record in a cell. Cell values are the full records,
cell keys are the RID of the record. The RID (row-ID) is an 8 byte unsigned
integer, which may not be reused for other records, even if a record was
deleted. The only scenario where an RID may be re-used is, when a record is
deleted from a page, while it is also being written into the same page or
another page (i.e. on move only) (this means, that the RID is not actually
re-used, just kept when moving or re-writing a cell). This can happen, if the
size of the record grows, and the cell has to be re-written. The cell keys aka.
RIDs are referenced by cells from the index pages. A full table scan is
performed by obtaining all cells in the data page and checking their records.

### Data definition
A data definition follows the following format (everything encoded in big
endian).

* 2 bytes `uint16` the amount of columns
* for each column
* 2 bytes `uint16` frame for the column name
* name bytes
* 1 byte `bool` that is 0 if the table is **NOT**, and 1 if the column is
nullable
* 2 bytes `uint16` frame for the type name
* type name bytes
48 changes: 48 additions & 0 deletions doc/page.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,48 @@
# Page layout
This document describes the layout and format of a single memory page. All pages
are structured like this.

**Please note:** "2 bytes indicating a length" implies, that these two bytes,
interpreted as **big endian** encoded **unsigned two byte integer** indicate
said length. In other words, whenever we talk about bytes forming some kind of
number, it is always the big endian encoding of an integer, either 2, 4 or 8
bytes.

## Page format
Pages implement the concept of slotted pages. A helpful resource for
understanding is
[this](https://db.in.tum.de/teaching/ss17/moderndbs/chapter3.pdf#page=8) PDF.
Please note though, that we do not follow the exact data structure that is
proposed in that file.

![Page Structure](./page_v1.png)

The above image describes the layout of a page. A page has a **fixed size of
16KiB** (16384 byte).

The **first 6 bytes** are the page header. It has a fixed size and will always
have that 6 byte layout. The **first 4 bytes** represent the page ID. This is
globally unique and is set upon page allocation. A page cannot infer it's own ID
without that header field. The **next 2 bytes** represent the cell count in this
page. This is the amount of slots that occur after the header, and is updated
with every call to `storeCell`.

After the header, in **4 byte chunks**, slots are defined. A slot points to an
absolute offset within the page, and holds a size attribute. The **first 2
bytes** are the offset, the **second 2 bytes** are the size.

Between slots and data, there is free space. This is the space, where new cells
(slots on the left, and data on the right) will be inserted. Slots are always
**sorted by the key of the cell that they point to**.

A single "slot data" is a full cell, as described [below](#cell-format).

## Cell format
Cells are simple key-value entities.

![Cell Structure](./cell_v1.png)

The above image describes the layout of a cell. A cell contains of a single key
and a single value, which is called the record. Both in front of the key and in
front of the record, there are **2 bytes** indicating the length of the key
respectively the record.
Binary file added doc/page_v1.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
4 changes: 0 additions & 4 deletions go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,6 @@ module github.com/tomarrell/lbadd
go 1.13

require (
github.com/awnumar/memguard v0.22.2
github.com/google/go-cmp v0.5.0
github.com/kr/text v0.2.0 // indirect
github.com/niemeyer/pretty v0.0.0-20200227124842-a10e7caefd8e // indirect
Expand All @@ -13,12 +12,9 @@ require (
github.com/spf13/cobra v1.0.0
github.com/spf13/pflag v1.0.5 // indirect
github.com/stretchr/testify v1.6.1
golang.org/x/crypto v0.0.0-20200429183012-4b2356b1ed79 // indirect
golang.org/x/net v0.0.0-20200505041828-1ed23360d12c
golang.org/x/sync v0.0.0-20200317015054-43a5402ce75a
golang.org/x/sys v0.0.0-20200501145240-bc7a7d42d5c3 // indirect
golang.org/x/text v0.3.3
golang.org/x/tools v0.0.0-20200521211927-2b542361a4fc
gopkg.in/check.v1 v1.0.0-20200227125254-8fa46927fb4f // indirect
gopkg.in/yaml.v2 v2.2.8 // indirect
)
26 changes: 0 additions & 26 deletions go.sum
Original file line number Diff line number Diff line change
Expand Up @@ -4,10 +4,6 @@ github.com/OneOfOne/xxhash v1.2.2/go.mod h1:HSdplMjZKSmBqAxg5vPj2TmRDmfkzw+cTzAE
github.com/alecthomas/template v0.0.0-20160405071501-a0175ee3bccc/go.mod h1:LOuyumcjzFXgccqObfd/Ljyb9UuFJ6TxHnclSeseNhc=
github.com/alecthomas/units v0.0.0-20151022065526-2efee857e7cf/go.mod h1:ybxpYRFXyAe+OPACYpWeL0wqObRcbAqCMya13uyzqw0=
github.com/armon/consul-api v0.0.0-20180202201655-eb2c6b5be1b6/go.mod h1:grANhF5doyWs3UAsr3K4I6qtAmlQcZDesFNEHPZAzj8=
github.com/awnumar/memcall v0.0.0-20191004114545-73db50fd9f80 h1:8kObYoBO4LNmQ+fLiScBfxEdxF1w2MHlvH/lr9MLaTg=
github.com/awnumar/memcall v0.0.0-20191004114545-73db50fd9f80/go.mod h1:S911igBPR9CThzd/hYQQmTc9SWNu3ZHIlCGaWsWsoJo=
github.com/awnumar/memguard v0.22.2 h1:tMxcq1WamhG13gigK8Yaj9i/CHNUO3fFlpS9ABBQAxw=
github.com/awnumar/memguard v0.22.2/go.mod h1:33OwJBHC+T4eEfFcDrQb78TMlBMBvcOPCXWU9xE34gM=
github.com/beorn7/perks v0.0.0-20180321164747-3a771d992973/go.mod h1:Dwedo/Wpr24TaqPxmxbtue+5NUziq4I4S80YR8gNf3Q=
github.com/beorn7/perks v1.0.0/go.mod h1:KWe93zE9D1o94FZ5RNwFwVgaQK1VOXiVxmqh+CedLV8=
github.com/cespare/xxhash v1.1.0/go.mod h1:XrSqR1VqqWfGrhpAt58auRo0WTKS1nRRg3ghfAqPWnc=
Expand Down Expand Up @@ -40,8 +36,6 @@ github.com/golang/protobuf v1.2.0/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5y
github.com/golang/protobuf v1.3.1/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U=
github.com/google/btree v1.0.0/go.mod h1:lNA+9X1NB3Zf8V7Ke586lFgjr2dZNuvo3lPJSGZ5JPQ=
github.com/google/go-cmp v0.2.0/go.mod h1:oXzfMopK8JAjlY9xF4vHSVASa0yLyX7SntLO5aqRK0M=
github.com/google/go-cmp v0.4.0 h1:xsAVV57WRhGj6kEIi8ReJzQlHHqcBYCElAvkovg3B/4=
github.com/google/go-cmp v0.4.0/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=
github.com/google/go-cmp v0.5.0 h1:/QaMHBdZ26BB3SSst0Iwl10Epc+xhTquomWX0oZEB6w=
github.com/google/go-cmp v0.5.0/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=
github.com/gorilla/websocket v1.4.0/go.mod h1:E7qHFY5m1UJ88s3WnNqhKjPHQ0heANvMoAMk2YaljkQ=
Expand Down Expand Up @@ -97,8 +91,6 @@ github.com/prometheus/procfs v0.0.0-20190507164030-5867b95ac084/go.mod h1:TjEm7z
github.com/prometheus/tsdb v0.7.1/go.mod h1:qhTCs0VvXwvX/y3TZrWD7rabWM+ijKTux40TwIPHuXU=
github.com/rogpeppe/fastuuid v0.0.0-20150106093220-6724a57986af/go.mod h1:XWv6SoW27p1b0cqNHllgS5HIMJraePCO15w5zCzIWYg=
github.com/rs/xid v1.2.1/go.mod h1:+uKXf+4Djp6Md1KODXJxgGQPKngRmWyn10oCKFzNHOQ=
github.com/rs/zerolog v1.18.0 h1:CbAm3kP2Tptby1i9sYy2MGRg0uxIN9cyDb59Ys7W8z8=
github.com/rs/zerolog v1.18.0/go.mod h1:9nvC1axdVrAHcu/s9taAVfBuIdTZLVQmKQyvrUjF5+I=
github.com/rs/zerolog v1.19.0 h1:hYz4ZVdUgjXTBUmrkrw55j1nHx68LfOKIQk5IYtyScg=
github.com/rs/zerolog v1.19.0/go.mod h1:IzD0RJ65iWH0w97OQQebJEvTZYvsCUm9WVLWBQrJRjo=
github.com/russross/blackfriday/v2 v2.0.1/go.mod h1:+Rmxgy9KzJVeS9/2gXHxylqXiyQDYRxCVz55jmeOWTM=
Expand All @@ -108,8 +100,6 @@ github.com/soheilhy/cmux v0.1.4/go.mod h1:IM3LyeVVIOuxMH7sFAkER9+bJ4dT7Ms6E4xg4k
github.com/spaolacci/murmur3 v0.0.0-20180118202830-f09979ecbc72/go.mod h1:JwIasOWyU6f++ZhiEuf87xNszmSA2myDM2Kzu9HwQUA=
github.com/spf13/afero v1.1.2 h1:m8/z1t7/fwjysjQRYbP0RD+bUIF/8tJwPdEZsI83ACI=
github.com/spf13/afero v1.1.2/go.mod h1:j4pytiNVoe2o6bmDsKpLACNPDBIoEAkihy7loJ1B0CQ=
github.com/spf13/afero v1.2.2 h1:5jhuqJyZCZf2JRofRvN/nIFgIWNzPa3/Vz8mYylgbWc=
github.com/spf13/afero v1.2.2/go.mod h1:9ZxEEn6pIJ8Rxe320qSDBk6AsU0r9pR7Q4OcevTdifk=
github.com/spf13/afero v1.3.1 h1:GPTpEAuNr98px18yNQ66JllNil98wfRZ/5Ukny8FeQA=
github.com/spf13/afero v1.3.1/go.mod h1:5KUK8ByomD5Ti5Artl0RtHeI5pTF7MIDuXL3yY520V4=
github.com/spf13/cast v1.3.0 h1:oget//CVOEoFewqQxwr0Ej5yjygnqGkvggSE/gB35Q8=
Expand All @@ -130,16 +120,13 @@ github.com/stretchr/objx v0.1.1 h1:2vfRuCMp5sSVIDSqO8oNnWJq7mPa6KVP3iPIwFBuy8A=
github.com/stretchr/objx v0.1.1/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
github.com/stretchr/testify v1.2.2/go.mod h1:a8OnRcib4nhh0OaRAV+Yts87kKdq0PP7pXfy6kDkUVs=
github.com/stretchr/testify v1.4.0/go.mod h1:j7eGeouHqKxXV5pUuKE4zz7dFj8WfuZ+81PSLYec5m4=
github.com/stretchr/testify v1.5.1 h1:nOGnQDM7FYENwehXlg/kFVnos3rEvtKTjRvOWSzb6H4=
github.com/stretchr/testify v1.5.1/go.mod h1:5W2xD1RspED5o8YsWQXVCued0rvSQ+mT+I5cxcmMvtA=
github.com/stretchr/testify v1.6.1 h1:hDPOHmpOpP40lSULcqw7IrRb/u7w6RpDC9399XyoNd0=
github.com/stretchr/testify v1.6.1/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
github.com/tmc/grpc-websocket-proxy v0.0.0-20190109142713-0ad062ec5ee5/go.mod h1:ncp9v5uamzpCO7NfCPTXjqaC+bZgJeR0sMTm6dMHP7U=
github.com/ugorji/go v1.1.4/go.mod h1:uQMGLiO92mf5W77hV/PUCpI3pbzQx3CRekS0kk+RGrc=
github.com/xiang90/probing v0.0.0-20190116061207-43a291ad63a2/go.mod h1:UETIi67q53MR2AWcXfiuqkDkRtnGDLqkBTpCHuJHxtU=
github.com/xordataexchange/crypt v0.0.3-0.20170626215501-b2862e3d0a77/go.mod h1:aYKd//L2LvnjZzWKhF00oedf4jCCReLcmhLdhm1A27Q=
github.com/yuin/goldmark v1.1.27/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74=
github.com/zenazn/goji v0.9.0/go.mod h1:7S9M489iMyHBNxwZnk9/EHS098H4/F6TATF2mIxtB1Q=
go.etcd.io/bbolt v1.3.2/go.mod h1:IbVyRI1SCnLcuJnV2u8VeU0CEYM7e686BmAb1XKL+uU=
go.uber.org/atomic v1.4.0/go.mod h1:gD2HeocX3+yG+ygLZcrzQJaqmWj9AIm7n08wl/qW/PE=
go.uber.org/multierr v1.1.0/go.mod h1:wR5kodmAFQ0UK8QlbwjlSNy0Z68gJhDJUG5sjR94q/0=
Expand All @@ -148,10 +135,6 @@ golang.org/x/crypto v0.0.0-20180904163835-0709b304e793/go.mod h1:6SG95UA2DQfeDnf
golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w=
golang.org/x/crypto v0.0.0-20190820162420-60c769a6c586/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI=
golang.org/x/crypto v0.0.0-20191011191535-87dc89f01550/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI=
golang.org/x/crypto v0.0.0-20200311171314-f7b00557c8c4 h1:QmwruyY+bKbDDL0BaglrbZABEali68eoMFhTZpCjYVA=
golang.org/x/crypto v0.0.0-20200311171314-f7b00557c8c4/go.mod h1:LzIPMQfyMNhhGPhUkYOs5KpL4U8rLKemX1yGLhDgUto=
golang.org/x/crypto v0.0.0-20200429183012-4b2356b1ed79 h1:IaQbIIB2X/Mp/DKctl6ROxz1KyMlKp4uyvL6+kQ7C88=
golang.org/x/crypto v0.0.0-20200429183012-4b2356b1ed79/go.mod h1:LzIPMQfyMNhhGPhUkYOs5KpL4U8rLKemX1yGLhDgUto=
golang.org/x/lint v0.0.0-20181026193005-c67002cb31c3/go.mod h1:UVdnD1Gm6xHRNCYTkRU2/jEulfH38KcIWyp/GAMgvoE=
golang.org/x/lint v0.0.0-20181026193005-c67002cb31c3/go.mod h1:UVdnD1Gm6xHRNCYTkRU2/jEulfH38KcIWyp/GAMgvoE=
golang.org/x/lint v0.0.0-20190313153728-d0100b6bd8b3/go.mod h1:6SW0HCj/g11FgYtHlgUYUwCkIfeOF89ocIRzGO/8vkc=
Expand Down Expand Up @@ -185,15 +168,8 @@ golang.org/x/sys v0.0.0-20181116152217-5ac8a444bdc5/go.mod h1:STP8DvDyc/dI5b8T5h
golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a h1:1BGLXjeY4akVXGgbC9HugT3Jv3hCI0z56oJR5vAMgBU=
golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20190412213103-97732733099d/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20191001151750-bb3f8db39f24/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200302150141-5c8b2ff67527 h1:uYVVQ9WP/Ds2ROhcaGPeIdVq0RIXVLwsHlnvJ+cT1So=
golang.org/x/sys v0.0.0-20200302150141-5c8b2ff67527/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200323222414-85ca7c5b95cd/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200501145240-bc7a7d42d5c3 h1:5B6i6EAiSYyejWfvc5Rc9BbI3rzIsrrXfAQBWnYfn+w=
golang.org/x/sys v0.0.0-20200501145240-bc7a7d42d5c3/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
golang.org/x/text v0.3.2 h1:tW2bmiBqwgJj/UpqtC8EpXEZVYOwU0yG4iWbprSVAcs=
golang.org/x/text v0.3.2/go.mod h1:bEr9sfX3Q8Zfm5fL9x+3itogRgK3+ptLWKqgva+5dAk=
golang.org/x/text v0.3.3 h1:cokOdA+Jmi5PJGXLlLllQSgYigAEfHXJAERHVMaCc2k=
golang.org/x/text v0.3.3/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
golang.org/x/time v0.0.0-20190308202827-9d24e82272b4/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ=
Expand Down Expand Up @@ -223,8 +199,6 @@ gopkg.in/resty.v1 v1.12.0/go.mod h1:mDo4pnntr5jdWRML875a/NmxYqAlA73dVijT2AXvQQo=
gopkg.in/yaml.v2 v2.0.0-20170812160011-eb3733d160e7/go.mod h1:JAlM8MvJe8wmxCU4Bli9HhUf9+ttbYbLASfIpnQbh74=
gopkg.in/yaml.v2 v2.2.1/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
gopkg.in/yaml.v2 v2.2.2/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
gopkg.in/yaml.v2 v2.2.8 h1:obN1ZagJSUGI0Ek/LBmuj4SNLPfIny3KsKFopxRdj10=
gopkg.in/yaml.v2 v2.2.8/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c h1:dUUwHk2QECo/6vqA44rthZ8ie2QXMNeKRTHCNY2nXvo=
gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
honnef.co/go/tools v0.0.0-20190102054323-c2f93a96b099/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4=
11 changes: 11 additions & 0 deletions internal/compiler/command/command.go
Original file line number Diff line number Diff line change
Expand Up @@ -104,6 +104,7 @@ type (
// select statements.
Table interface {
_table()
QualifiedName() string
}

// SimpleTable is a table that is only specified by schema and table name,
Expand Down Expand Up @@ -313,6 +314,16 @@ func (Values) _list() {}

func (SimpleTable) _table() {}

// QualifiedName returns '<Schema>.<TableName>', or only '<TableName>' if no
// schema is specified.
func (t SimpleTable) QualifiedName() string {
qualifiedName := t.Table
if t.Schema != "" {
qualifiedName = t.Schema + "." + qualifiedName
}
return qualifiedName
}

func (e Explain) String() string {
return fmt.Sprintf("explanation: %v", e.Command)
}
Expand Down
28 changes: 28 additions & 0 deletions internal/compiler/command/insertor_string.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

8 changes: 6 additions & 2 deletions internal/compiler/golden_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@ package compiler

import (
"flag"
"fmt"
"io/ioutil"
"os"
"path/filepath"
Expand Down Expand Up @@ -43,19 +44,22 @@ func runGolden(t *testing.T, input string) {
got, err := c.Compile(stmt)
require.NoError(err)

gotGoString := fmt.Sprintf("%#v", got)
gotString := got.String()
gotFull := gotGoString + "\n\nString:\n" + gotString

testFilePath := "testdata/" + t.Name() + ".golden"

if *record {
t.Logf("overwriting golden file %v", testFilePath)
err := os.MkdirAll(filepath.Dir(testFilePath), 0777)
require.NoError(err)
err = ioutil.WriteFile(testFilePath, []byte(gotString), 0666)
err = ioutil.WriteFile(testFilePath, []byte(gotFull), 0666)
require.NoError(err)
t.Fail()
} else {
data, err := ioutil.ReadFile(testFilePath)
require.NoError(err)
require.Equal(string(data), gotString)
require.Equal(string(data), gotFull)
}
}
11 changes: 11 additions & 0 deletions internal/compiler/simple_compiler_fixture_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,17 @@ func TestCompileGolden(t *testing.T) {
t.Run("delete", _TestCompileDelete)
t.Run("drop", _TestCompileDrop)
t.Run("update", _TestCompileUpdate)
t.Run("expressions", _TestCompileExpressions)
}

func _TestCompileExpressions(t *testing.T) {
tests := []string{
"VALUES (7)",
"VALUES (-7)",
}
for _, test := range tests {
RunGolden(t, test)
}
}

func _TestCompileUpdate(t *testing.T) {
Expand Down
Original file line number Diff line number Diff line change
@@ -1 +1,4 @@
command.Delete{Table:command.SimpleTable{Schema:"", Table:"myTable", Alias:"", Indexed:false, Index:""}, Filter:command.ConstantBooleanExpr{Value:true}}

String:
Delete[filter=true](myTable)
Original file line number Diff line number Diff line change
@@ -1 +1,4 @@
command.Delete{Table:command.SimpleTable{Schema:"mySchema", Table:"myTable", Alias:"", Indexed:false, Index:""}, Filter:command.ConstantBooleanExpr{Value:true}}

String:
Delete[filter=true](mySchema.myTable)
Original file line number Diff line number Diff line change
@@ -1 +1,4 @@
command.Delete{Table:command.SimpleTable{Schema:"", Table:"myTable", Alias:"", Indexed:false, Index:""}, Filter:command.BinaryExpr{Operator:"==", Left:command.LiteralExpr{Value:"col1"}, Right:command.LiteralExpr{Value:"col2"}}}

String:
Delete[filter=col1 == col2](myTable)
3 changes: 3 additions & 0 deletions internal/compiler/testdata/TestCompileGolden/drop/#00.golden
Original file line number Diff line number Diff line change
@@ -1 +1,4 @@
command.DropTable{IfExists:false, Schema:"", Name:"myTable"}

String:
DropTable[table=myTable,ifexists=false]()
Loading