Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(log-viewer-webui): Refactor S3Manager into a fastify plugin. #689

Open
wants to merge 3 commits into
base: main
Choose a base branch
from
+33 −24
Open

feat(log-viewer-webui): Refactor S3Manager into a fastify plugin. #689

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
4f40126
backup
haiqi96 Jan 23, 2025
ca3a674
change
haiqi96 Jan 23, 2025
77826ce
Linter
haiqi96 Jan 23, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
34 changes: 26 additions & 8 deletions components/log-viewer-webui/server/src/S3Manager.js
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,3 +1,5 @@
import fastifyPlugin from "fastify-plugin";

import {
GetObjectCommand,
S3Client,
Expand All @@ -16,15 +18,27 @@ const PRE_SIGNED_URL_EXPIRY_TIME_SECONDS = 3600;
class S3Manager {
#s3Client;

#enabled;

/**
* @param {string} region
* @param {string|null} region
haiqi96 marked this conversation as resolved.
Show resolved Hide resolved
*/
constructor (
region,
) {
this.#s3Client = new S3Client({
region: region,
});
constructor (region) {
haiqi96 marked this conversation as resolved.
Show resolved Hide resolved
console.log(`Region's length is ${region.length}`)
console.log(`Region is ${region}`)
console.log(region)
this.#enabled = null !== region;
console.log(`Enabled is ${this.#enabled}`)
if (true === this.#enabled) {
console.log(`Initializing S3 client`)
this.#s3Client = new S3Client({
region: region,
});
}
}

isEnabled () {
return this.#enabled;
}

/**
Expand Down Expand Up @@ -55,4 +69,8 @@ class S3Manager {
}
}

export default S3Manager;
export default fastifyPlugin(async (app, region) => {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fastifyPlugin requires the second argument in the callback to be an options object: See https://fastify.dev/docs/latest/Reference/Plugins/#plugin-options

because Fastify specific options like logLevel, logSerializers, prefix are expected to show up in the options object if they exist.

Suggested change
export default fastifyPlugin(async (app, region) => {
export default fastifyPlugin(async (app, options) => {

console.log(`Region in Plugin init is ${region}`)
console.log(region)
await app.decorate("s3Manager", new S3Manager(region));
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we can directly pass all options to the S3Manager constructor, assuming we will be adding more options in the future.

Suggested change
await app.decorate("s3Manager", new S3Manager(region));
await app.decorate("s3Manager", new S3Manager(options));

});
3 changes: 3 additions & 0 deletions components/log-viewer-webui/server/src/app.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@ import DbManager from "./DbManager.js";
import exampleRoutes from "./routes/example.js";
import queryRoutes from "./routes/query.js";
import staticRoutes from "./routes/static.js";
import S3Manager from "./S3Manager.js";


/**
Expand Down Expand Up @@ -41,6 +42,8 @@ const app = async ({
port: settings.MongoDbPort,
},
});
console.log(settings.StreamFilesS3Region)
await server.register(S3Manager, settings.StreamFilesS3Region);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Then we will be calling this plugin registration with an option object:

Suggested change
await server.register(S3Manager, settings.StreamFilesS3Region);
await server.register(S3Manager, {region: settings.StreamFilesS3Region});

}

await server.register(staticRoutes);
Expand Down
25 changes: 10 additions & 15 deletions components/log-viewer-webui/server/src/routes/query.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,22 +2,15 @@ import {StatusCodes} from "http-status-codes";

import settings from "../../settings.json" with {type: "json"};
import {EXTRACT_JOB_TYPES} from "../DbManager.js";
import S3Manager from "../S3Manager.js";


const S3_MANAGER = (
null === settings.StreamFilesS3PathPrefix ||
null === settings.StreamFilesS3Region
) ?
null :
new S3Manager(settings.StreamFilesS3Region);


/**
* Submits a stream extraction job and returns the metadata of the extracted stream.
*
* @param {object} props
* @param {import("fastify").FastifyInstance | {dbManager: DbManager}} props.fastify
* @param {import("fastify").FastifyInstance |
* {dbManager: DbManager} |
* {s3Manager: S3Manager}} props.fastify
* @param {EXTRACT_JOB_TYPES} props.jobType
* @param {number} props.logEventIdx
* @param {string} props.streamId
Expand Down Expand Up @@ -63,7 +56,9 @@ const extractStreamAndGetMetadata = async ({
/**
* Creates query routes.
*
* @param {import("fastify").FastifyInstance | {dbManager: DbManager}} fastify
* @param {import("fastify").FastifyInstance |
* {dbManager: DbManager} |
* {s3Manager: S3Manager}} fastify
* @param {import("fastify").FastifyPluginOptions} options
* @return {Promise<void>}
*/
Expand Down Expand Up @@ -96,12 +91,12 @@ const routes = async (fastify, options) => {
});
}

if (null === S3_MANAGER) {
streamMetadata.path = `/streams/${streamMetadata.path}`;
} else {
streamMetadata.path = await S3_MANAGER.getPreSignedUrl(
if (fastify.s3Manager.isEnabled()) {
streamMetadata.path = await fastify.s3Manager.getPreSignedUrl(
`s3://${settings.StreamFilesS3PathPrefix}${streamMetadata.path}`
);
} else {
streamMetadata.path = `/streams/${streamMetadata.path}`;
Comment on lines +94 to +99
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue

Ensure s3Manager is defined before use

Accessing fastify.s3Manager without checking if it is defined could lead to runtime errors, especially in environments where s3Manager is not registered (e.g., when NODE_ENV === "test"). Modify the condition to check for the existence of s3Manager before calling isEnabled().

Apply this diff to prevent potential null reference errors:

-if (fastify.s3Manager.isEnabled()) {
+if (fastify.s3Manager && fastify.s3Manager.isEnabled()) {
     streamMetadata.path = await fastify.s3Manager.getPreSignedUrl(
         `s3://${settings.StreamFilesS3PathPrefix}${streamMetadata.path}`
     );
 } else {
     streamMetadata.path = `/streams/${streamMetadata.path}`;
 }
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
if (fastify.s3Manager.isEnabled()) {
streamMetadata.path = await fastify.s3Manager.getPreSignedUrl(
`s3://${settings.StreamFilesS3PathPrefix}${streamMetadata.path}`
);
} else {
streamMetadata.path = `/streams/${streamMetadata.path}`;
if (fastify.s3Manager && fastify.s3Manager.isEnabled()) {
streamMetadata.path = await fastify.s3Manager.getPreSignedUrl(
`s3://${settings.StreamFilesS3PathPrefix}${streamMetadata.path}`
);
} else {
streamMetadata.path = `/streams/${streamMetadata.path}`;

}

return streamMetadata;
Expand Down
Loading