Skip to content

Commit

Permalink
Big Refactor and Cleanup (#134)
Browse files Browse the repository at this point in the history
* Big Refactor and Cleanup

* Delete TNLJ and In-Memory Sort
  • Loading branch information
robertzhao2002 authored and GitHub Enterprise committed Nov 16, 2022
1 parent 9352b97 commit ad27f1b
Show file tree
Hide file tree
Showing 76 changed files with 236 additions and 923 deletions.
63 changes: 2 additions & 61 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -75,71 +75,12 @@ while (outer hasn't reached the end) {
}
```

### Benchmarking

To benchmark, we first developed a [class](./src/main/java/com/dbms/analytics/TupleGenerator.java) to randomly generate tuples. We leveraged our [TupleWriter.java](./src/main/java/com/dbms/utils/TupleWriter.java) class and used it to write randomly generated tuples to files. To keep track of execution time, we used a 3rd party library stopwatch by Google called Guava Stopwatch instead of the Java library one because it uses device native time, instead of a time from a server. The stopwatch recorded the elapsed time for each query, and we wrote the time for each query to a log file. To make benchmarking easier, we also added a function to programmatically apply the configuration, so we don't need to manually update the given config file. Finally, we defined a separate `main` function to run our benchmarking experiments a certain number of times (a command line argument). All the code can be found in the [analytics](./src/main/java/com/dbms/analytics) package.

#### Benchmarking File Structure

In order to reduce the number of command line inputs, we defined constant file paths to keep our benchmarking-related files. Set up the benchmarking file path below to avoid runtime errors.

```
.
└── project/
├── benchmarking/
│ ├── input/
│ │ └── db/
│ │ ├── schema.txt
│ │ └── data/
│ │ └── [rng-data-here]
│ ├── logs/
│ │ └── [generated-execution-time-log-files-here]
│ └── output/
│ └── [generated-query-outputs-here]
```

#### Benchmarking Different Join Types

To run the benchmarking task in [JoinType.java](./src/main/java/com/dbms/analytics/JoinType.java) only once, run the command below.

```
./gradlew benchmarkJoin
```

To run the benchmarking task in [JoinType.java](./src/main/java/com/dbms/analytics/JoinType.java) a certain number of times, run the command below.

```
./gradlew benchmarkJoin --args="[number-of-trials]"
```

#### Benchmarking Data Indexing

To run the benchmarking task in [Indexing.java](./src/main/java/com/dbms/analytics/Indexing.java) only once, run the command below.

```
./gradlew benchmarkIndexing
```

To run the benchmarking task in [Indexing.java](./src/main/java/com/dbms/analytics/Indexing.java) a certain number of times, run the command below.

```
./gradlew benchmarkIndexing --args="[number-of-trials]"
```

### Running the Application

Because we have multiple executable `main` functions, we defined 2 new Gradle tasks in our [build.gradle.kts](build.gradle.kts) file to run either one. To run the `main` function in [Interpreter.java](./src/main/java/com/dbms/Interpreter.java), you can run either of the commands below (the default `main` function is still set to the one in [Interpreter.java](./src/main/java/com/dbms/Interpreter.java)).

#### Interpreter as its own Gradle task

```
./gradlew interpreter --args="[input-directory] [output-directory] [temp-directory]"
```

#### Interpreter as the default Gradle run task
To run the top-level application, use the command below.

```
./gradlew run --args="[input-directory] [output-directory] [temp-directory]"
./gradlew run --args="[config-file-path]"
```

### Formatting Code
Expand Down
15 changes: 0 additions & 15 deletions build.gradle.kts
Original file line number Diff line number Diff line change
Expand Up @@ -38,18 +38,3 @@ tasks.test {
events("passed", "skipped", "failed")
}
}

task("interpreter", JavaExec::class) {
mainClass.set("com.dbms.Interpreter")
classpath = sourceSets["main"].runtimeClasspath
}

task("benchmarkJoin", JavaExec::class) {
mainClass.set("com.dbms.analytics.JoinType")
classpath = sourceSets["main"].runtimeClasspath
}

task("benchmarkIndexing", JavaExec::class) {
mainClass.set("com.dbms.analytics.Indexing")
classpath = sourceSets["main"].runtimeClasspath
}
Binary file removed expected/tnlj/query1
Binary file not shown.
Binary file removed expected/tnlj/query10
Binary file not shown.
Binary file removed expected/tnlj/query11
Binary file not shown.
Binary file removed expected/tnlj/query12
Binary file not shown.
Binary file removed expected/tnlj/query13
Binary file not shown.
Binary file removed expected/tnlj/query14
Binary file not shown.
Binary file removed expected/tnlj/query15
Binary file not shown.
Binary file removed expected/tnlj/query2
Binary file not shown.
Binary file removed expected/tnlj/query3
Binary file not shown.
Binary file removed expected/tnlj/query4
Binary file not shown.
Binary file removed expected/tnlj/query5
Binary file not shown.
Binary file removed expected/tnlj/query6
Binary file not shown.
Empty file removed expected/tnlj/query7
Empty file.
Binary file removed expected/tnlj/query8
Binary file not shown.
Binary file removed expected/tnlj/query9
Binary file not shown.
3 changes: 1 addition & 2 deletions input/bnlj/plan_builder_config.txt
Original file line number Diff line number Diff line change
@@ -1,3 +1,2 @@
1 5
1 3
0
0
3 changes: 1 addition & 2 deletions input/general/plan_builder_config.txt
Original file line number Diff line number Diff line change
@@ -1,3 +1,2 @@
2
0
1
0
1 change: 0 additions & 1 deletion input/index/plan_builder_config.txt
Original file line number Diff line number Diff line change
@@ -1,3 +1,2 @@
0
1 4
0
3 changes: 1 addition & 2 deletions input/indexed/plan_builder_config.txt
Original file line number Diff line number Diff line change
@@ -1,3 +1,2 @@
1 4
1 4
0
1
3 changes: 1 addition & 2 deletions input/smj/plan_builder_config.txt
Original file line number Diff line number Diff line change
@@ -1,3 +1,2 @@
2
0
1
0
5 changes: 0 additions & 5 deletions input/tnlj/config.txt

This file was deleted.

Binary file removed input/tnlj/db/data/Boats
Binary file not shown.
Binary file removed input/tnlj/db/data/Reserves
Binary file not shown.
Binary file removed input/tnlj/db/data/Sailors
Binary file not shown.
1 change: 0 additions & 1 deletion input/tnlj/db/index_info.txt

This file was deleted.

Binary file removed input/tnlj/db/indexes/Boats.E
Binary file not shown.
3 changes: 0 additions & 3 deletions input/tnlj/db/schema.txt

This file was deleted.

3 changes: 0 additions & 3 deletions input/tnlj/db/stats.txt

This file was deleted.

3 changes: 0 additions & 3 deletions input/tnlj/plan_builder_config.txt

This file was deleted.

15 changes: 0 additions & 15 deletions input/tnlj/queries.sql

This file was deleted.

4 changes: 2 additions & 2 deletions src/main/java/com/dbms/Interpreter.java
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
package com.dbms;

import com.dbms.queryplan.LogicalPlanBuilder;
import com.dbms.queryplan.PhysicalPlanBuilder;
import com.dbms.utils.Catalog;
import com.dbms.utils.LogicalPlanBuilder;
import com.dbms.visitors.PhysicalPlanBuilder;
import java.io.BufferedReader;
import java.io.IOException;
import net.sf.jsqlparser.parser.CCJSqlParserUtil;
Expand Down
84 changes: 0 additions & 84 deletions src/main/java/com/dbms/analytics/Benchmarking.java

This file was deleted.

69 changes: 0 additions & 69 deletions src/main/java/com/dbms/analytics/Indexing.java

This file was deleted.

Loading

0 comments on commit ad27f1b

Please sign in to comment.