Skip to content

Commit

Permalink
PR comments
Browse files Browse the repository at this point in the history
  • Loading branch information
Lin Jian committed Jun 11, 2020
1 parent 2d7fcc3 commit 2a961fb
Show file tree
Hide file tree
Showing 15 changed files with 52 additions and 155 deletions.
24 changes: 6 additions & 18 deletions sdk/storage/storage-blob-changefeed/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ The purpose of the change feed is to provide transaction logs of all the changes

This project provides a client library in JavaScript that makes it easy to consume the change feed.

Use the client libararies in this package to:
Use the client libraries in this package to:
- Reading change feed events, all or within a time range
- Resuming reading events from a saved position

Expand Down Expand Up @@ -34,19 +34,7 @@ This library uses an authenticated `BlobServiceClient` to initialize. Refer to [

### Compatibility

For this perview, this library is only compatible with Node.js.

### CORS

You need to set up [Cross-Origin Resource Sharing (CORS)](https://docs.microsoft.com/rest/api/storageservices/cross-origin-resource-sharing--cors--support-for-the-azure-storage-services) rules for your storage account if you need to develop for browsers. Go to Azure portal and Azure Storage Explorer, find your storage account, create new CORS rules for blob/queue/file/table service(s).

For example, you can create following CORS settings for debugging. But please customize the settings carefully according to your requirements in production environment.

- Allowed origins: \*
- Allowed verbs: DELETE,GET,HEAD,MERGE,POST,OPTIONS,PUT
- Allowed headers: \*
- Exposed headers: \*
- Maximum age (seconds): 86400
For this preview, this library is only compatible with Node.js.

## Key concepts

Expand All @@ -69,7 +57,7 @@ The `BlobChangeFeedClient` requires a `BlobServiceClient` to initialize. Refer t
const account = "<account>";
const accountKey = "<accountkey>";
// Use StorageSharedKeyCredential with storage account and account key
// StorageSharedKeyCredential is only avaiable in Node.js runtime, not in browsers
// StorageSharedKeyCredential is only available in Node.js runtime, not in browsers
const sharedKeyCredential = new StorageSharedKeyCredential(account, accountKey);
const blobServiceClient = new BlobServiceClient(
`https://${account}.blob.core.windows.net`,
Expand Down Expand Up @@ -157,9 +145,9 @@ setLogLevel("info");

More code samples:

- [Blob Storage Samples (JavaScript)](https://github.com/Azure/azure-sdk-for-js/tree/master/sdk/storage/storage-blob-changefeed/samples/javascript)
- [Blob Storage Samples (TypeScript)](https://github.com/Azure/azure-sdk-for-js/tree/master/sdk/storage/storage-blob-changefeed/samples/typescript)
- [Blob Storage Test Cases](https://github.com/Azure/azure-sdk-for-js/tree/master/sdk/storage/storage-blob-changefeed/test/)
- [Blob Storage Change Feed Samples (JavaScript)](https://github.com/Azure/azure-sdk-for-js/tree/master/sdk/storage/storage-blob-changefeed/samples/javascript)
- [Blob Storage Change Feed Samples (TypeScript)](https://github.com/Azure/azure-sdk-for-js/tree/master/sdk/storage/storage-blob-changefeed/samples/typescript)
- [Blob Storage Change Feed Test Cases](https://github.com/Azure/azure-sdk-for-js/tree/master/sdk/storage/storage-blob-changefeed/test/)

## Contributing

Expand Down

This file was deleted.

6 changes: 3 additions & 3 deletions sdk/storage/storage-blob-changefeed/src/AvroReaderFactory.ts
Original file line number Diff line number Diff line change
@@ -1,16 +1,16 @@
import { AvroReadable, AvroReader } from "../../storage-internal-avro/src";

export class AvroReaderFactory {
public buildAvroReader(headerAndDataStream: AvroReadable): AvroReader;
public create(headerAndDataStream: AvroReadable): AvroReader;

public buildAvroReader(
public create(
dataStream: AvroReadable,
headerStream: AvroReadable,
blockOffset: number,
eventIndex: number
): AvroReader;

public buildAvroReader(
public create(
dataStream: AvroReadable,
headerStream?: AvroReadable,
blockOffset?: number,
Expand Down
10 changes: 5 additions & 5 deletions sdk/storage/storage-blob-changefeed/src/BlobChangeFeedClient.ts
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ import { PagedAsyncIterableIterator, PageSettings } from "@azure/core-paging";
import { BlobChangeFeedEvent } from "./models/BlobChangeFeedEvent";
import { ChangeFeedFactory } from "./ChangeFeedFactory";
import { ChangeFeed } from "./ChangeFeed";
import { CHANGE_FEED_DEFAULT_PAGE_SIZE } from "./utils/constants";
import { CHANGE_FEED_MAX_PAGE_SIZE } from "./utils/constants";

export interface ChangeFeedGetChangesOptions {
start?: Date;
Expand Down Expand Up @@ -65,7 +65,7 @@ export class BlobChangeFeedClient {
private async *getChange(
options: ChangeFeedGetChangesOptions = {}
): AsyncIterableIterator<BlobChangeFeedEvent> {
const changeFeed: ChangeFeed = await this._changeFeedFactory.buildChangeFeed(
const changeFeed: ChangeFeed = await this._changeFeedFactory.create(
this._blobServiceClient,
undefined,
options.start,
Expand All @@ -88,15 +88,15 @@ export class BlobChangeFeedClient {
maxPageSize?: number,
options: ChangeFeedGetChangesOptions = {}
): AsyncIterableIterator<BlobChangeFeedEventPage> {
const changeFeed: ChangeFeed = await this._changeFeedFactory.buildChangeFeed(
const changeFeed: ChangeFeed = await this._changeFeedFactory.create(
this._blobServiceClient,
continuationToken,
options.start,
options.end
);

if (!maxPageSize || maxPageSize > CHANGE_FEED_DEFAULT_PAGE_SIZE) {
maxPageSize = CHANGE_FEED_DEFAULT_PAGE_SIZE;
if (!maxPageSize || maxPageSize > CHANGE_FEED_MAX_PAGE_SIZE) {
maxPageSize = CHANGE_FEED_MAX_PAGE_SIZE;
}
while (changeFeed.hasNext()) {
let eventPage = new BlobChangeFeedEventPage();
Expand Down
6 changes: 3 additions & 3 deletions sdk/storage/storage-blob-changefeed/src/ChangeFeed.ts
Original file line number Diff line number Diff line change
Expand Up @@ -99,7 +99,7 @@ export class ChangeFeed {

return {
cursorVersion: 1,
urlHash: hashString(getURI(this._containerClient!.url)!),
urlHash: hashString(getURI(this._containerClient!.url)),
endTime: this._endTime?.toJSON(),
currentSegmentCursor: this._currentSegment!.getCursor()
};
Expand All @@ -117,7 +117,7 @@ export class ChangeFeed {

// If the current segment is completed, remove it
if (this._segments.length > 0) {
this._currentSegment = await this._segmentFactory!.buildSegment(
this._currentSegment = await this._segmentFactory!.create(
this._containerClient!,
this._segments.shift()!
);
Expand All @@ -133,7 +133,7 @@ export class ChangeFeed {
);

if (this._segments.length > 0) {
this._currentSegment = await this._segmentFactory!.buildSegment(
this._currentSegment = await this._segmentFactory!.create(
this._containerClient!,
this._segments.shift()!
);
Expand Down
6 changes: 3 additions & 3 deletions sdk/storage/storage-blob-changefeed/src/ChangeFeedFactory.ts
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,7 @@ export class ChangeFeedFactory {
}
}

public async buildChangeFeed(
public async create(
blobServiceClient: BlobServiceClient,
continuationToken?: string,
startTime?: Date,
Expand Down Expand Up @@ -107,7 +107,7 @@ export class ChangeFeedFactory {
if (segments.length === 0) {
return new ChangeFeed();
}
const currentSegment: Segment = await this._segmentFactory.buildSegment(
const currentSegment: Segment = await this._segmentFactory.create(
containerClient,
segments.shift()!,
cursor?.currentSegmentCursor
Expand All @@ -126,7 +126,7 @@ export class ChangeFeedFactory {
}

private static validateCursor(containerClient: ContainerClient, cursor: ChangeFeedCursor): void {
if (hashString(getURI(containerClient.url)!) !== cursor.urlHash) {
if (hashString(getURI(containerClient.url)) !== cursor.urlHash) {
throw new Error("Cursor URL does not match container URL.");
}
}
Expand Down
6 changes: 3 additions & 3 deletions sdk/storage/storage-blob-changefeed/src/ChunkFactory.ts
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ export class ChunkFactory {
this._avroReaderFactory = avroReaderFactory;
}

public async buildChunk(
public async create(
containerClient: ContainerClient,
chunkPath: string,
blockOffset?: number,
Expand All @@ -28,14 +28,14 @@ export class ChunkFactory {
if (blockOffset !== 0) {
const headerDownloadRes = await blobClient.download(0);
const headerStream = bodyToAvroReadable(headerDownloadRes);
avroReader = this._avroReaderFactory.buildAvroReader(
avroReader = this._avroReaderFactory.create(
dataStream,
headerStream,
blockOffset,
eventIndex
);
} else {
avroReader = this._avroReaderFactory.buildAvroReader(dataStream);
avroReader = this._avroReaderFactory.create(dataStream);
}

return new Chunk(avroReader, blockOffset, eventIndex);
Expand Down
4 changes: 2 additions & 2 deletions sdk/storage/storage-blob-changefeed/src/SegmentFactory.ts
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ export class SegmentFactory {
this._shardFactory = shardFactory;
}

public async buildSegment(
public async create(
containerClient: ContainerClient,
manifestPath: string,
cursor?: SegmentCursor
Expand All @@ -44,7 +44,7 @@ export class SegmentFactory {

const containerPrefixLength = CHANGE_FEED_CONTAINER_NAME.length + 1; // "$blobchangefeed/"
for (const shardPath of segmentManifest.chunkFilePaths) {
const shard: Shard = await this._shardFactory.buildShard(
const shard: Shard = await this._shardFactory.create(
containerClient,
shardPath.substring(containerPrefixLength),
cursor?.shardCursors[i++]
Expand Down
2 changes: 1 addition & 1 deletion sdk/storage/storage-blob-changefeed/src/Shard.ts
Original file line number Diff line number Diff line change
Expand Up @@ -40,7 +40,7 @@ export class Shard {

// Remove currentChunk if it doesn't have more events.
if (!this._currentChunk.hasNext() && this._chunks.length > 0) {
this._currentChunk = await this._chunkFactory.buildChunk(
this._currentChunk = await this._chunkFactory.create(
this._containerClient,
this._chunks.shift()!
);
Expand Down
4 changes: 2 additions & 2 deletions sdk/storage/storage-blob-changefeed/src/ShardFactory.ts
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ export class ShardFactory {
this._chunkFactory = chunkFactory;
}

public async buildShard(
public async create(
containerClient: ContainerClient,
shardPath: string,
shardCursor?: ShardCursor
Expand All @@ -37,7 +37,7 @@ export class ShardFactory {
chunks.splice(0, chunkIndex);
}

const currentChunk = await this._chunkFactory.buildChunk(
const currentChunk = await this._chunkFactory.create(
containerClient,
chunks.shift()!,
blockOffset,
Expand Down
2 changes: 1 addition & 1 deletion sdk/storage/storage-blob-changefeed/src/utils/constants.ts
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
export const CHANGE_FEED_CONTAINER_NAME: string = "$blobchangefeed";
export const CHANGE_FEED_META_SEGMENT_PATH: string = "meta/segments.json";
export const CHANGE_FEED_DEFAULT_PAGE_SIZE: number = 5000; // align with rest API list operations
export const CHANGE_FEED_MAX_PAGE_SIZE: number = 5000; // align with rest API list operations
export const CHANGE_FEED_STATUS_FINALIZED: string = "Finalized";
export const CHANGE_FEED_SEGMENT_PREFIX: string = "idx/segments/";
export const CHANGE_FEED_INITIALIZATION_SEGMENT: string = "1601";
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ export function floorToNearestHour(date: Date | undefined): Date | undefined {
* @param {string} url Source URL string
* @returns {(string | undefined)}
*/
export function getURI(url: string): string | undefined {
export function getURI(url: string): string {
const urlParsed = URLBuilder.parse(url);
return `${urlParsed.getHost()}${urlParsed.getPort()}${urlParsed.getPath()}`;
}
Expand Down
Loading

0 comments on commit 2a961fb

Please sign in to comment.