Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Pull master branch updates to resource manager branch #426

Merged
Merged
Changes from 3 commits
Commits
Show all changes
71 commits
Select commit Hold shift + click to select a range
e063bc5
Auth cleanup
Nov 7, 2015
9394d28
Remove DefaultAuthCredentials from serialization tests, since there a…
Nov 7, 2015
6074605
Modify project id lookup order and revert style changes
Nov 9, 2015
2ffac2d
Move generation from BlobInfo to BlobId
mziccard Nov 12, 2015
6c961b6
Create version 0.0.12
Nov 12, 2015
29d9c23
Merge pull request #369 from ajkannan/update-version-0.0.12
aozarov Nov 12, 2015
2fd8066
Updating version in README files.
Nov 12, 2015
af10dbf
Update version to 0.0.13-SNAPSHOT
Nov 13, 2015
6890fa1
Merge pull request #370 from ajkannan/update-version-0.0.13-SNAPSHOT
aozarov Nov 13, 2015
f0ce897
Remove Blob(Source/Get)Option.setGeneration method, add Storage.optio…
mziccard Nov 13, 2015
3311969
Set blob generation before sending requests in DefaultStorageRpc
mziccard Nov 13, 2015
dc71a56
Simplify package-info examples
Nov 13, 2015
22f52ae
Document that we should wait until artifacts are pushed to finish rel…
Nov 13, 2015
f3c8274
Fix code to get app engine ID
Nov 13, 2015
b67dad7
Merge pull request #378 from ajkannan/reflection-fix
aozarov Nov 13, 2015
522ed56
Merge pull request #377 from ajkannan/releasing-docs
aozarov Nov 13, 2015
1bcdf4b
Make package info wording clearer
Nov 13, 2015
cf777fc
Fix setting ContentRange to inclusive positions
mziccard Nov 13, 2015
b7099e0
Merge pull request #376 from ajkannan/update-pkg-info-examples
mziccard Nov 13, 2015
d039537
Merge pull request #375 from mziccard/fix-content-range
aozarov Nov 13, 2015
77dab2c
Remove extra central repository deploy
Nov 14, 2015
3f6d791
Add additional deploy check comment
Nov 14, 2015
29d8ec3
Merge pull request #379 from ajkannan/push-to-maven-once
aozarov Nov 14, 2015
315a0eb
Merge pull request #366 from mziccard/add-generation-blob-id
aozarov Nov 14, 2015
76cb620
Better document delete methods
mziccard Nov 17, 2015
e22e8a3
Make batch delete return false if not found
mziccard Nov 17, 2015
c3b6616
Use GOOGLE_APPLICATION_CREDENTIALS and GCLOUD_PROJECT vars in RemoteG…
mziccard Nov 9, 2015
82ebc71
Add support for signUrl when default credentials are used
mziccard Nov 17, 2015
4ecd840
Merge pull request #339 from mziccard/integration-test-env-vars
aozarov Nov 17, 2015
93da8a9
fix readmes for storage
aozarov Nov 18, 2015
101048e
Merge pull request #389 from aozarov/temp
mziccard Nov 18, 2015
87e4bac
Fix BlobWriteChannel and BlobReadChannel javadoc
mziccard Nov 18, 2015
6537314
BlobReadChannel fails if blob is updated while being read
mziccard Nov 18, 2015
1e7fe54
Move all checks for HTTP_NOT_FOUND to DefaultStorageRpc
mziccard Nov 18, 2015
a4c4273
Merge pull request #380 from mziccard/fix-delete-batch
aozarov Nov 18, 2015
a560777
Use version auth library verison 0.3.1 to avoid NPE
Nov 18, 2015
f995b12
Merge to get AuthCredentials updates
Nov 18, 2015
02694ca
Update javadoc for FullEntity
aozarov Nov 19, 2015
bfe8924
Add message check to IT tests, better javadoc for storage.reader
mziccard Nov 19, 2015
39829a0
Merge pull request #390 from mziccard/rewriter-check-blob-update
aozarov Nov 19, 2015
bbf3cb6
Merge pull request #393 from aozarov/temp
ludoch Nov 20, 2015
a07fbda
Add step-by-step datastore guide
Nov 19, 2015
424f144
fix snippets to match full code
Nov 20, 2015
b48d163
Merge pull request #399 from ajkannan/fix-docs
aozarov Nov 20, 2015
791fba1
Add links to gcloud-common docs in READMEs
Nov 20, 2015
78f8f28
Limit batch deletes to 100, issue serveral batch if limit's exceeded
mziccard Nov 20, 2015
dba9efb
Merge pull request #401 from mziccard/limit-batch-deletes
aozarov Nov 20, 2015
84c91bc
fix link to auth readme
Nov 21, 2015
347d5be
use anchors in links
Nov 21, 2015
08c0f28
Add step-by-step guide for storage README
Nov 21, 2015
a6da17e
Merge pull request #404 from ajkannan/link-docs
aozarov Nov 21, 2015
3af2bba
fix storage readme sample
Nov 23, 2015
8ebfdf8
add link to bucket naming rules
Nov 23, 2015
a27282e
Merge pull request #405 from ajkannan/add-storage-examples
aozarov Nov 23, 2015
539ac61
Add back AppEngineCredentials
Nov 23, 2015
fc2ab10
Avoid throwing exception when checking isLocalHost in Datastore
Nov 24, 2015
8808c54
Add link about datastore indexing in docs
Nov 24, 2015
d8a6159
Merge pull request #411 from ajkannan/fix-socket-api-bug
aozarov Nov 24, 2015
a23d8f9
Merge pull request #412 from ajkannan/datastore-index-link
aozarov Nov 24, 2015
0a05851
Translate SocketTimeoutException to retryable service exception
mziccard Nov 24, 2015
a0f2d20
Set special reason for timeout exceptions
mziccard Nov 24, 2015
1a6dcde
Fix toBuilder so that info.equals(info.toBuilder().build()) is true
mziccard Nov 24, 2015
2ef47e8
Use NetHttpTransport when running on Compute
Nov 24, 2015
f87a394
Fix dependency on two different versions of the GoogleCredential class
Nov 25, 2015
2c90245
Merge pull request #337 from ajkannan/auth-cleanup
aozarov Nov 25, 2015
bbc968e
Merge pull request #414 from mziccard/retry-on-timeout
aozarov Nov 25, 2015
a25e504
Merge pull request #416 from mziccard/fix-storage-to-builder
aozarov Nov 25, 2015
1830275
Enable travis to branches other than master
mziccard Nov 30, 2015
80c7a96
Merge pull request #424 from mziccard/test-all-branches
aozarov Nov 30, 2015
5e46fba
Merge branch 'master' of https://github.com/GoogleCloudPlatform/gclou…
Nov 30, 2015
0160867
sync pom version to parent project
Nov 30, 2015
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
@@ -28,6 +28,7 @@
import com.google.api.client.json.jackson.JacksonFactory;
import com.google.auth.http.HttpCredentialsAdapter;
import com.google.auth.oauth2.GoogleCredentials;
import com.google.auth.oauth2.ServiceAccountCredentials;

import java.io.IOException;
import java.io.InputStream;
@@ -212,7 +213,7 @@ public RestorableState<AuthCredentials> capture() {
}
}

private static class ApplicationDefaultAuthCredentials extends AuthCredentials {
public static class ApplicationDefaultAuthCredentials extends AuthCredentials {

private GoogleCredentials googleCredentials;

@@ -255,6 +256,15 @@ protected HttpRequestInitializer httpRequestInitializer(HttpTransport transport,
return new HttpCredentialsAdapter(googleCredentials.createScoped(scopes));
}

public ServiceAccountAuthCredentials toServiceAccountCredentials() {
if (googleCredentials instanceof ServiceAccountCredentials) {
ServiceAccountCredentials credentials = (ServiceAccountCredentials) googleCredentials;
return new ServiceAccountAuthCredentials(credentials.getClientEmail(),
credentials.getPrivateKey());
}
return null;
}

@Override
public RestorableState<AuthCredentials> capture() {
return STATE;
Original file line number Diff line number Diff line change
@@ -43,6 +43,8 @@
import com.google.common.hash.Hashing;
import com.google.common.io.BaseEncoding;
import com.google.common.primitives.Ints;
import com.google.gcloud.AuthCredentials;
import com.google.gcloud.AuthCredentials.ApplicationDefaultAuthCredentials;
import com.google.gcloud.AuthCredentials.ServiceAccountAuthCredentials;
import com.google.gcloud.PageImpl;
import com.google.gcloud.BaseService;
@@ -584,9 +586,15 @@ public URL signUrl(BlobInfo blobInfo, long duration, TimeUnit unit, SignUrlOptio
ServiceAccountAuthCredentials cred =
(ServiceAccountAuthCredentials) optionMap.get(SignUrlOption.Option.SERVICE_ACCOUNT_CRED);
if (cred == null) {
checkArgument(options().authCredentials() instanceof ServiceAccountAuthCredentials,
"Signing key was not provided and could not be derived");
cred = (ServiceAccountAuthCredentials) this.options().authCredentials();
AuthCredentials serviceCred = this.options().authCredentials();
if (serviceCred instanceof ServiceAccountAuthCredentials) {
cred = (ServiceAccountAuthCredentials) serviceCred;
} else {
if (serviceCred instanceof ApplicationDefaultAuthCredentials) {
cred = ((ApplicationDefaultAuthCredentials) serviceCred).toServiceAccountCredentials();
}
}
checkArgument(cred != null, "Signing key was not provided and could not be derived");
}
// construct signature - see https://cloud.google.com/storage/docs/access-control#Signed-URLs
StringBuilder stBuilder = new StringBuilder();
Original file line number Diff line number Diff line change
@@ -45,8 +45,6 @@ public class RemoteGcsHelper {

private static final Logger log = Logger.getLogger(RemoteGcsHelper.class.getName());
private static final String BUCKET_NAME_PREFIX = "gcloud-test-bucket-temp-";
private static final String PROJECT_ID_ENV_VAR = "GCLOUD_TESTS_PROJECT_ID";
private static final String PRIVATE_KEY_ENV_VAR = "GCLOUD_TESTS_KEY";
private final StorageOptions options;

private RemoteGcsHelper(StorageOptions options) {
@@ -107,13 +105,7 @@ public static RemoteGcsHelper create(String projectId, InputStream keyStream)
StorageOptions storageOptions = StorageOptions.builder()
.authCredentials(AuthCredentials.createForJson(keyStream))
.projectId(projectId)
.retryParams(RetryParams.builder()
.retryMaxAttempts(10)
.retryMinAttempts(6)
.maxRetryDelayMillis(30000)
.totalRetryPeriodMillis(120000)
.initialRetryDelayMillis(250)
.build())
.retryParams(retryParams())
.connectTimeout(60000)
.readTimeout(60000)
.build();
@@ -145,41 +137,30 @@ public static RemoteGcsHelper create(String projectId, String keyPath)
log.log(Level.WARNING, ex.getMessage());
}
throw GcsHelperException.translate(ex);
} catch (IOException ex) {
if (log.isLoggable(Level.WARNING)) {
log.log(Level.WARNING, ex.getMessage());
}
throw GcsHelperException.translate(ex);
}
}

/**
* Creates a {@code RemoteGcsHelper} object. Project id and path to JSON key are read from two
* environment variables: {@code GCLOUD_TESTS_PROJECT_ID} and {@code GCLOUD_TESTS_KEY}.
*
* @return A {@code RemoteGcsHelper} object for the provided options.
* @throws com.google.gcloud.storage.testing.RemoteGcsHelper.GcsHelperException if environment
* variables {@code GCLOUD_TESTS_PROJECT_ID} and {@code GCLOUD_TESTS_KEY} are not set or if
* the file pointed by {@code GCLOUD_TESTS_KEY} does not exist
* Creates a {@code RemoteGcsHelper} object using default project id and authentication
* credentials.
*/
public static RemoteGcsHelper create() throws GcsHelperException {
String projectId = System.getenv(PROJECT_ID_ENV_VAR);
String keyPath = System.getenv(PRIVATE_KEY_ENV_VAR);
if (projectId == null) {
String message = "Environment variable " + PROJECT_ID_ENV_VAR + " not set";
if (log.isLoggable(Level.WARNING)) {
log.log(Level.WARNING, message);
}
throw new GcsHelperException(message);
}
if (keyPath == null) {
String message = "Environment variable " + PRIVATE_KEY_ENV_VAR + " not set";
if (log.isLoggable(Level.WARNING)) {
log.log(Level.WARNING, message);
}
throw new GcsHelperException(message);
}
return create(projectId, keyPath);
StorageOptions storageOptions = StorageOptions.builder()
.retryParams(retryParams())
.connectTimeout(60000)
.readTimeout(60000)
.build();
return new RemoteGcsHelper(storageOptions);
}

private static RetryParams retryParams() {
return RetryParams.builder()
.retryMaxAttempts(10)
.retryMinAttempts(6)
.maxRetryDelayMillis(30000)
.totalRetryPeriodMillis(120000)
.initialRetryDelayMillis(250)
.build();
}

private static class DeleteBucketTask implements Callable<Boolean> {
4 changes: 2 additions & 2 deletions utilities/integration_test_env.sh
Original file line number Diff line number Diff line change
@@ -1,3 +1,3 @@
# Export test env variables
export GCLOUD_TESTS_PROJECT_ID="gcloud-devel"
export GCLOUD_TESTS_KEY=$TRAVIS_BUILD_DIR/signing-tools/gcloud-devel-travis.json
export GCLOUD_PROJECT="gcloud-devel"
export GOOGLE_APPLICATION_CREDENTIALS=$TRAVIS_BUILD_DIR/signing-tools/gcloud-devel-travis.json