Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adds postgres implementation for tree_storage #1343

Merged
merged 5 commits into from
Dec 5, 2018

Conversation

vishalkuo
Copy link
Contributor

@vishalkuo vishalkuo commented Nov 11, 2018

This PR adds a partial implementation for the ReadOnlyTreeTX in postgres.

This is mostly a copy from mySQL's tree_storage.go file. The only major difference is the expandPlaceholderSQL method which needs a little more attention due to formatting requiring numeric placeholders for pg instead of ? in mySQL.

I'm not sure how to test this PR without starting work on the log_storage implementation and I was worried at that point it'd be really long and difficult to review. If we're fine merging a NOOP PR in so that it's in a manageable size, I can add the rest in subsequent PRs with tests.

Alternatively, I can try to break this PR down into smaller ones that tests functions solely in tree storage OR create a much larger PR with tests via log storage.

I have no strong preference to either option, I'd just love to get some feedback on how to proceed.

Issue: #1298

@codecov
Copy link

codecov bot commented Nov 12, 2018

Codecov Report

Merging #1343 into master will decrease coverage by 0.91%.
The diff coverage is 7.44%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #1343      +/-   ##
==========================================
- Coverage   60.17%   59.25%   -0.92%     
==========================================
  Files         129      129              
  Lines       10380    10568     +188     
==========================================
+ Hits         6246     6262      +16     
- Misses       3428     3600     +172     
  Partials      706      706
Impacted Files Coverage Δ
storage/postgres/tree_storage.go 7.21% <7.44%> (+7.21%) ⬆️
testonly/hammer/hammer.go 57.22% <0%> (+0.4%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 059e7df...ee08162. Read the comment docs.

storage/postgres/storage_test.go Outdated Show resolved Hide resolved
storage/postgres/storage_test.go Outdated Show resolved Hide resolved
storage/postgres/testdb/testdb.go Outdated Show resolved Hide resolved
storage/postgres/testdb/testdb.go Outdated Show resolved Hide resolved
storage/postgres/tree_storage.go Outdated Show resolved Hide resolved
storage/postgres/tree_storage.go Outdated Show resolved Hide resolved
return err
}
}
t.closed = true
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should the reads/writes to closed not be covered by a mutex?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like it's the same in the mysql version, @Martin2112 do you have thoughts?
The cloudspanner implementation by comparison does hold mu in Close()

storage/postgres/tree_storage_test.go Show resolved Hide resolved
storage/postgres/tree_storage_test.go Show resolved Hide resolved
@@ -12,43 +12,37 @@
// See the License for the specific language governing permissions and
// limitations under the License.

package postgres_test
package postgres
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it not possible to do black box testing of this? (just curious)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Admin storage is actually currently blackbox tested; however I wanted to bring everything under the postgres package because

A) that seems to be the pattern the rest of the codebase follows
B) It allows us to use a unified TestMain function.

Happy to change it back though, I don't have strong feelings about this

@AlCutter
Copy link
Member

I'm alright with going on with this PR and then building on top with log_storage and map_storage in followups, unless @RJPercival has strong feelings?

@vishalkuo
Copy link
Contributor Author

Polite bump

@AlCutter
Copy link
Member

AlCutter commented Dec 5, 2018

@RJPercival you happy?

}

// statementSkeleton contains the structure of a query to create.
type statementSkeleton struct {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I assume this gets used a lot more in future PRs, thereby justifying its existence? It looks rather complicated for the two uses that appear in this PR. All of this string substitution into SQL and use of variables as printf format strings makes me a little uneasy!

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If this implementation follows the MySQL impl - it won't get used elsewhere. I'm not sure if there's a better way to handle this but I'm open to suggestions

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well if @AlCutter is happy with it, I don't want to block you any longer. I'll try and find the time to have a think about whether there's a better way of going about this and come back to it.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks Rob, I just created #1369 to track this.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants