Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: added support for url which imports files as data-URIs or ES #60

Merged
merged 4 commits into from
Nov 12, 2024

Conversation

prisis
Copy link
Contributor

@prisis prisis commented Nov 12, 2024

Summary by CodeRabbit

Release Notes

  • New Features

    • Added support for handling various image formats (GIF, JPEG, JPG, PNG, SVG, WEBP) through new dedicated files for easier asset management.
    • Introduced a new Rollup plugin for copying and inlining asset files, enhancing build capabilities.
    • Updated licensing information for the mime dependency to ensure compliance.
  • Bug Fixes

    • Improved error handling and clarity in integration tests for URL handling functionality.
  • Updates

    • Version updated to 1.5.1 in the package configuration.
    • Refined loader mappings for file extensions in the constants.
  • Chores

    • Removed outdated dependencies and cleaned up configuration files for better maintainability.

Copy link
Contributor

coderabbitai bot commented Nov 12, 2024

Warning

Rate limit exceeded

@prisis has exceeded the limit for the number of commits or files that can be reviewed per hour. Please wait 0 minutes and 49 seconds before requesting another review.

⌛ How to resolve this issue?

After the wait time has elapsed, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

We recommend that you space out your commits to avoid hitting the rate limit.

🚦 How do rate limits work?

CodeRabbit enforces hourly rate limits for each developer per organization.

Our paid plans have higher rate limits than the trial, open-source and free plans. In all cases, we re-allow further reviews after a brief timeout.

Please see our FAQ for further information.

📥 Commits

Reviewing files that changed from the base of the PR and between 3f68f93 and 9899d5a.

Walkthrough

The pull request introduces several changes to the @visulima/packem package, including the addition of a new dependency, mime, with its licensing information documented in LICENSE.md. Multiple files for different image formats (GIF, JPEG, JPG, PNG, SVG, WebP) have been created to facilitate importing these assets. Additionally, various modifications have been made to configuration and utility files, including the introduction of new functions for cleaning distribution directories and managing cache folders. Test files have also been added and updated to ensure comprehensive coverage of the new functionalities.

Changes

File Path Change Summary
packages/packem/LICENSE.md Added licensing information for the new mime dependency (MIT license, author: Robert Kieffer, repository link provided).
packages/packem/__fixtures__/url/gif.js New file created to export a GIF image.
packages/packem/__fixtures__/url/jpeg.js New file created to export a JPEG image.
packages/packem/__fixtures__/url/jpg.js New file created to export a JPG image.
packages/packem/__fixtures__/url/png.js New file created to export a PNG image.
packages/packem/__fixtures__/url/svg.js New file created to export an SVG image.
packages/packem/__fixtures__/url/webp.js New file created to export a WebP image.
packages/packem/__tests__/intigration/css.test.ts Modifications made to improve clarity and maintainability of the test code.
packages/packem/__tests__/intigration/url.test.ts New integration test file for URL handling functionality, covering various image formats and their processing.
packages/packem/package.json Updated version to 1.5.1, removed mime-types, added mime dependency (version 4.0.4).
packages/packem/packem.config.ts Removed commented-out sections related to typedoc builder configuration.
packages/packem/src/constants.ts Updated DEFAULT_LOADERS constant to change loaders for .ctsx and .mtsx, removed several file extensions.
packages/packem/src/packem.ts Removed cleanDistributionDirectories and removeOldCacheFolders functions, added new properties to generateOptions.
packages/packem/src/rollup/get-rollup-options.ts Added urlPlugin to the Rollup plugins array for enhanced URL handling.
packages/packem/src/rollup/plugins/css/loaders/postcss/url/inline.ts Updated MIME type lookup to use liteMime instead of mime-types.
packages/packem/src/rollup/plugins/url.ts Introduced a new Rollup plugin for handling asset files, including configuration options for file management.
packages/packem/src/types.ts Imported UrlOptions, updated interfaces for RollupDynamicImportVariablesOptions and RollupBuildOptions.
packages/packem/src/utils/clean-distribution-directories.ts New file added with a function to clean distribution directories based on the build context.
packages/packem/src/utils/remove-old-cache-folders.ts New file added with a function to manage cache directories based on a specific JSON file.

Possibly related PRs

🐰 In the garden of code, we hop with glee,
New images to share, as bright as can be!
With mime in our basket, we dance and we play,
Bringing colors to life, in a wonderful way.
Dependencies grow, like carrots in rows,
In the world of Packem, our creativity flows! 🌼


Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media?

❤️ Share
🪧 Tips

Chat

There are 3 ways to chat with CodeRabbit:

  • Review comments: Directly reply to a review comment made by CodeRabbit. Example:
    • I pushed a fix in commit <commit_id>, please review it.
    • Generate unit testing code for this file.
    • Open a follow-up GitHub issue for this discussion.
  • Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
    • @coderabbitai generate unit testing code for this file.
    • @coderabbitai modularize this function.
  • PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
    • @coderabbitai gather interesting stats about this repository and render them as a table. Additionally, render a pie chart showing the language distribution in the codebase.
    • @coderabbitai read src/utils.ts and generate unit testing code.
    • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.
    • @coderabbitai help me debug CodeRabbit configuration file.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (Invoked using PR comments)

  • @coderabbitai pause to pause the reviews on a PR.
  • @coderabbitai resume to resume the paused reviews.
  • @coderabbitai review to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
  • @coderabbitai full review to do a full review from scratch and review all the files again.
  • @coderabbitai summary to regenerate the summary of the PR.
  • @coderabbitai resolve resolve all the CodeRabbit review comments.
  • @coderabbitai configuration to show the current CodeRabbit configuration for the repository.
  • @coderabbitai help to get help.

Other keywords and placeholders

  • Add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.
  • Add @coderabbitai summary to generate the high-level summary at a specific location in the PR description.
  • Add @coderabbitai anywhere in the PR title to generate the title automatically.

CodeRabbit Configuration File (.coderabbit.yaml)

  • You can programmatically configure CodeRabbit by adding a .coderabbit.yaml file to the root of your repository.
  • Please see the configuration documentation for more information.
  • If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json

Documentation and Community

  • Visit our Documentation for detailed information on how to use CodeRabbit.
  • Join our Discord Community to get help, request features, and share feedback.
  • Follow us on X/Twitter for updates and announcements.

Copy link
Contributor

github-actions bot commented Nov 12, 2024

Thank you for following the naming conventions! 🙏

Copy link

pkg-pr-new bot commented Nov 12, 2024

Open in Stackblitz

pnpm add https://pkg.pr.new/visulima/packem/@visulima/packem@60

commit: 9899d5a

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 10

🧹 Outside diff range and nitpick comments (14)
packages/packem/src/utils/clean-distribution-directories.ts (3)

9-15: Simplify the Set creation and type handling.

The type assertion as unknown as Set<string> is unnecessary since we're creating a new Set from the array.

-        for (const directory of new Set(
-            // eslint-disable-next-line @typescript-eslint/require-array-sort-compare
-            context.options.entries
-                .map((entry) => entry.outDir)
-                .filter(Boolean)
-                .sort() as unknown as Set<string>,
-        )) {
+        const directories = [...new Set(
+            context.options.entries
+                .map((entry) => entry.outDir)
+                .filter(Boolean)
+        )].sort();
+        for (const directory of directories) {

16-24: Consider improving directory path validation.

The directory path checking logic could be more robust by:

  1. Normalizing paths before comparison
  2. Using path manipulation utilities for safer path checks
+            const normalizedDir = directory.endsWith("/") ? directory : `${directory}/`;
+            const normalizedRootDir = context.options.rootDir.endsWith("/") 
+                ? context.options.rootDir 
+                : `${context.options.rootDir}/`;
             if (
-                directory === context.options.rootDir ||
+                normalizedDir === normalizedRootDir ||
                 directory === context.options.sourceDir ||
-                context.options.rootDir.startsWith(directory.endsWith("/") ? directory : `${directory}/`) ||
+                normalizedRootDir.startsWith(normalizedDir) ||
                 cleanedDirectories.some((c) => directory.startsWith(c))
             ) {

29-31: Consider parallel directory cleaning for better performance.

The current implementation cleans directories sequentially. For better performance, consider cleaning them in parallel.

-            // eslint-disable-next-line no-await-in-loop
-            await emptyDir(directory);
+        const cleaningPromises = directories.map((directory) => emptyDir(directory));
+        await Promise.all(cleaningPromises);

Note: Only implement this if parallel cleaning is safe for your use case and directories are not nested.

packages/packem/src/utils/remove-old-cache-folders.ts (2)

11-16: Consider optimizing directory filtering.

The current implementation reads all entries and then filters directories. Consider combining these operations for better efficiency.

Here's a more efficient approach:

-        // eslint-disable-next-line security/detect-non-literal-fs-filename
-        let cacheDirectories = await readdir(cachePath, {
-            withFileTypes: true,
-        });
-
-        cacheDirectories = cacheDirectories.filter((dirent) => dirent.isDirectory());
+        // eslint-disable-next-line security/detect-non-literal-fs-filename
+        const cacheDirectories = (await readdir(cachePath, { withFileTypes: true }))
+            .filter((dirent) => dirent.isDirectory());

20-39: Enhance logging and code readability.

A few suggestions to improve this section:

  1. Use template literals for the log message
  2. Consider simplifying the logging logic

Apply these improvements:

             if (!keyStore[dirent.name]) {
                 // eslint-disable-next-line no-await-in-loop
                 await rm(join(cachePath, dirent.name), {
                     force: true,
                     recursive: true,
                 });
 
-                if (hasLogged) {
+                if (hasLogged) {
                     logger.raw("\n\n");
-                }
+                    hasLogged = false;
+                }
 
                 logger.info({
-                    message: "Removing " + dirent.name + " file cache, the cache key is not used anymore.",
+                    message: `Removing ${dirent.name} file cache, the cache key is not used anymore.`,
                     prefix: "file-cache",
                 });
-
-                hasLogged = false;
             }
packages/packem/__tests__/intigration/url.test.ts (1)

1-14: Fix typo in directory name: "intigration" should be "integration"

The directory name contains a spelling error that should be corrected for better maintainability and consistency.

packages/packem/package.json (1)

Line range hint 2-2: Consider bumping minor version instead of patch

Since this PR adds new functionality (URL imports as data-URIs), according to semver, it should bump the minor version to "1.6.0" instead of the patch version to "1.5.1".

-    "version": "1.5.1",
+    "version": "1.6.0",
packages/packem/src/packem.ts (2)

30-31: Good modularization of utility functions!

Moving cleanDistributionDirectories and removeOldCacheFolders to separate utility files improves code organization and follows the Single Responsibility Principle.

This modularization:

  • Improves maintainability by isolating specific functionalities
  • Makes the code more testable
  • Reduces the complexity of the main file

Also applies to: 40-41


Line range hint 1-824: Well-structured code with robust error handling!

The overall implementation demonstrates several good practices:

  • Comprehensive error handling with proper enhancement
  • Clean process termination handling
  • Strong TypeScript typing
  • Detailed logging

Consider adding JSDoc comments for the main exported functions to improve documentation.

packages/packem/__tests__/intigration/css.test.ts (4)

Line range hint 1-24: Fix typo in directory name

The directory name contains a typo: intigration should be integration.


75-77: Simplify path handling in cpSync

The join operation is redundant here as fixturePath already includes the directory path. You can directly use string concatenation with the input path.

-        cpSync(join(fixturePath, dirname(input[0] as string)), temporaryDirectoryPath, { recursive: true });
+        cpSync(`${fixturePath}/${dirname(input[0] as string)}`, temporaryDirectoryPath, { recursive: true });

150-158: Address TODO comment and deprecation warning

There's a TODO comment about changing the readdir implementation to use @visulima/fs. Additionally, the code uses a deprecated feature.

Would you like me to help implement the @visulima/fs readdir solution to address both the TODO and deprecation warning?


Multiple Skipped Tests Found

The shell script detected numerous files with skipped tests, indicating widespread gaps in test coverage. Please review and address these skipped tests to ensure comprehensive testing.

🔗 Analysis chain

Line range hint 590-600: Address skipped and commented tests

Several test cases are either skipped or commented out:

  1. The entire stylus test suite is skipped
  2. Some sass import tests are commented out
  3. DTS-related tests are commented out

These gaps in test coverage should be addressed to ensure comprehensive testing.

🏁 Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Check for other skipped tests in the codebase
rg -l "describe\.skip|it\.skip|xdescribe|xit" --type ts

Length of output: 4044

packages/packem/src/rollup/plugins/url.ts (1)

170-170: Consider using a stronger hash function

The code uses the SHA-1 hashing algorithm, which is considered outdated and has known vulnerabilities. While it's used here for file naming and not for security purposes, it's recommended to use a stronger algorithm like SHA-256 to prevent potential hash collisions.

Apply this diff to use SHA-256:

- const hash = crypto.createHash("sha1").update(buffer).digest("hex").slice(0, 16);
+ const hash = crypto.createHash("sha256").update(buffer).digest("hex").slice(0, 16);
📜 Review details

Configuration used: CodeRabbit UI
Review profile: CHILL

📥 Commits

Reviewing files that changed from the base of the PR and between 7261f2c and 3f68f93.

⛔ Files ignored due to path filters (7)
  • packages/packem/__fixtures__/url/gif.gif is excluded by !**/*.gif
  • packages/packem/__fixtures__/url/jpeg.jpeg is excluded by !**/*.jpeg
  • packages/packem/__fixtures__/url/jpg.jpg is excluded by !**/*.jpg
  • packages/packem/__fixtures__/url/png.png is excluded by !**/*.png
  • packages/packem/__fixtures__/url/svg.svg is excluded by !**/*.svg
  • packages/packem/__tests__/intigration/__snapshots__/url.test.ts.snap is excluded by !**/*.snap
  • pnpm-lock.yaml is excluded by !**/pnpm-lock.yaml
📒 Files selected for processing (19)
  • packages/packem/LICENSE.md (1 hunks)
  • packages/packem/__fixtures__/url/gif.js (1 hunks)
  • packages/packem/__fixtures__/url/jpeg.js (1 hunks)
  • packages/packem/__fixtures__/url/jpg.js (1 hunks)
  • packages/packem/__fixtures__/url/png.js (1 hunks)
  • packages/packem/__fixtures__/url/svg.js (1 hunks)
  • packages/packem/__fixtures__/url/webp.js (1 hunks)
  • packages/packem/__tests__/intigration/css.test.ts (3 hunks)
  • packages/packem/__tests__/intigration/url.test.ts (1 hunks)
  • packages/packem/package.json (2 hunks)
  • packages/packem/packem.config.ts (1 hunks)
  • packages/packem/src/constants.ts (1 hunks)
  • packages/packem/src/packem.ts (5 hunks)
  • packages/packem/src/rollup/get-rollup-options.ts (2 hunks)
  • packages/packem/src/rollup/plugins/css/loaders/postcss/url/inline.ts (1 hunks)
  • packages/packem/src/rollup/plugins/url.ts (1 hunks)
  • packages/packem/src/types.ts (3 hunks)
  • packages/packem/src/utils/clean-distribution-directories.ts (1 hunks)
  • packages/packem/src/utils/remove-old-cache-folders.ts (1 hunks)
✅ Files skipped from review due to trivial changes (7)
  • packages/packem/fixtures/url/gif.js
  • packages/packem/fixtures/url/jpeg.js
  • packages/packem/fixtures/url/jpg.js
  • packages/packem/fixtures/url/png.js
  • packages/packem/fixtures/url/svg.js
  • packages/packem/fixtures/url/webp.js
  • packages/packem/packem.config.ts
🧰 Additional context used
🪛 Markdownlint
packages/packem/LICENSE.md

77-77: null
Bare URL used

(MD034, no-bare-urls)

🪛 GitHub Check: CodeQL
packages/packem/src/rollup/plugins/url.ts

[failure] 112-117: Incomplete multi-character sanitization
This string may still contain <!--, which may cause an HTML element injection vulnerability.

🔇 Additional comments (14)
packages/packem/src/rollup/plugins/css/loaders/postcss/url/inline.ts (1)

1-1: LGTM! Verify MIME type coverage for all supported file types.

The switch to mime/lite is a good optimization. However, let's ensure it supports all necessary MIME types for the URL plugin.

packages/packem/src/utils/clean-distribution-directories.ts (3)

1-4: LGTM! Clean and focused imports.

The imports are minimal and specific to the requirements.


5-7: LGTM! Well-typed function declaration.

The function signature is clear and properly typed.


35-35: LGTM! Clean export.

The default export is appropriate for this utility function.

packages/packem/src/constants.ts (2)

Line range hint 1-45: Verify handling of removed image formats.

The removal of image format loaders (.gif, .jpeg, .jpg, .png, .svg, .webp) from DEFAULT_LOADERS suggests these will now be handled by the URL plugin.

Let's verify the new handling mechanism:

#!/bin/bash
# Description: Verify that the URL plugin properly handles the removed formats
# and that there are no orphaned image imports.

# Check for the URL plugin implementation
echo "Checking URL plugin implementation..."
rg -A 5 "urlPlugin" packages/packem/src/

# Check for image imports in the codebase
echo "Checking image imports..."
rg -g '*.{ts,tsx,js,jsx}' '\.(gif|jpe?g|png|svg|webp)'

9-9: LGTM! Loader changes align with TypeScript conventions.

The change from "ts" to "tsx" loader for .ctsx and .mtsx extensions is correct as these are TypeScript JSX files.

Let's verify the usage of these extensions in the codebase:

Also applies to: 18-18

packages/packem/src/utils/remove-old-cache-folders.ts (1)

1-7: LGTM! Clean imports and well-typed function signature.

The imports are appropriate, and the TypeScript types are properly defined.

packages/packem/src/types.ts (1)

34-34: LGTM! URL plugin configuration properly integrated.

The new URL plugin configuration has been properly integrated into the type system:

  • The UrlOptions type import follows the established pattern
  • The optional url property in RollupBuildOptions maintains consistency with other plugin configurations

Also applies to: 110-110

packages/packem/__tests__/intigration/url.test.ts (2)

15-25: LGTM! Well-structured test setup and teardown

Good implementation of test isolation using temporary directories with proper cleanup.


82-313: LGTM! Comprehensive test coverage

The test suite provides excellent coverage with:

  • Tests for various file types (PNG, JPG, JPEG, GIF, WEBP, SVG)
  • Different scenarios (inlining, copying, size limits)
  • Edge cases (nested directories, file naming patterns)
  • Configuration options (publicPath, sourceDir, destDir)
packages/packem/package.json (1)

386-387: LGTM: Dependency update looks good

The replacement of mime-types with mime is a good choice as it provides better TypeScript support and is more suitable for handling data-URIs. The version is compatible with the project's Node.js engine requirements.

packages/packem/src/packem.ts (1)

317-323: LGTM! URL configuration looks well-structured.

The URL configuration is properly configured with:

  • Appropriate file size limit (14KB) for inlining
  • Common image format support
  • Content-based hash naming
  • Proper exclusion patterns

Let's verify the EXCLUDE_REGEXP pattern matches the URL configuration:

packages/packem/src/rollup/get-rollup-options.ts (2)

54-54: LGTM!

The import statement follows the existing import style and is properly positioned with related plugin imports.


498-499: LGTM!

The URL plugin is correctly configured:

  • Follows the existing pattern of conditional plugin addition
  • Properly positioned in the plugins array
  • Configuration is passed through from the context options

Let's verify the URL plugin configuration:

✅ Verification successful

URL plugin configuration verified.

🏁 Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Description: Check for URL plugin configuration and usage patterns

# Check for URL plugin configuration patterns
rg -A 5 'rollup\.url' 

# Check for URL plugin imports and usage
ast-grep --pattern 'import { urlPlugin } from "./plugins/url"'

Length of output: 754


Script:

#!/bin/bash
# Description: Verify the positioning and conditional addition of urlPlugin in the plugins array

# Locate the plugins array in get-rollup-options.ts and display surrounding lines for context
rg -A 10 'plugins\s*:\s*\[' packages/packem/src/rollup/get-rollup-options.ts

# Check the placement of urlPlugin within the plugins array
rg 'urlPlugin' packages/packem/src/rollup/get-rollup-options.ts -A 2 -B 2

Length of output: 1632

Comment on lines 3 to 7
export default (file: string, source: Uint8Array): string => {
const mime = lookup(file) || "application/octet-stream";
const mime = liteMime.getType(file) ?? "application/octet-stream";
const data = Buffer.from(source).toString("base64");

return `data:${mime};base64,${data}`;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🛠️ Refactor suggestion

Add input validation and size limits for robustness.

The function works but could benefit from some safety improvements:

  1. Add input validation for parameters
  2. Implement size limits to prevent memory issues
  3. Consider handling empty or invalid source buffers

Consider applying these improvements:

-export default (file: string, source: Uint8Array): string => {
+const MAX_FILE_SIZE = 1024 * 1024; // 1MB limit for inline files
+
+export default (file: string, source: Uint8Array): string => {
+    if (!file || typeof file !== 'string') {
+        throw new Error('File path must be a non-empty string');
+    }
+    
+    if (!source || !(source instanceof Uint8Array)) {
+        throw new Error('Source must be a Uint8Array');
+    }
+    
+    if (source.length > MAX_FILE_SIZE) {
+        throw new Error(`File size exceeds maximum limit of ${MAX_FILE_SIZE} bytes`);
+    }
+
     const mime = liteMime.getType(file) ?? "application/octet-stream";
     const data = Buffer.from(source).toString("base64");
 
     return `data:${mime};base64,${data}`;
 };
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
export default (file: string, source: Uint8Array): string => {
const mime = lookup(file) || "application/octet-stream";
const mime = liteMime.getType(file) ?? "application/octet-stream";
const data = Buffer.from(source).toString("base64");
return `data:${mime};base64,${data}`;
const MAX_FILE_SIZE = 1024 * 1024; // 1MB limit for inline files
export default (file: string, source: Uint8Array): string => {
if (!file || typeof file !== 'string') {
throw new Error('File path must be a non-empty string');
}
if (!source || !(source instanceof Uint8Array)) {
throw new Error('Source must be a Uint8Array');
}
if (source.length > MAX_FILE_SIZE) {
throw new Error(`File size exceeds maximum limit of ${MAX_FILE_SIZE} bytes`);
}
const mime = liteMime.getType(file) ?? "application/octet-stream";
const data = Buffer.from(source).toString("base64");
return `data:${mime};base64,${data}`;
};

Comment on lines +1 to +35
import { relative } from "@visulima/path";
import { emptyDir } from "@visulima/fs";
import type { BuildContext } from "../types";

const cleanDistributionDirectories = async (context: BuildContext): Promise<void> => {
const cleanedDirectories: string[] = [];

if (context.options.clean) {
for (const directory of new Set(
// eslint-disable-next-line @typescript-eslint/require-array-sort-compare
context.options.entries
.map((entry) => entry.outDir)
.filter(Boolean)
.sort() as unknown as Set<string>,
)) {
if (
directory === context.options.rootDir ||
directory === context.options.sourceDir ||
context.options.rootDir.startsWith(directory.endsWith("/") ? directory : `${directory}/`) ||
cleanedDirectories.some((c) => directory.startsWith(c))
) {
// eslint-disable-next-line no-continue
continue;
}

cleanedDirectories.push(directory);
context.logger.info(`Cleaning dist directory: \`./${relative(context.options.rootDir, directory)}\``);

// eslint-disable-next-line no-await-in-loop
await emptyDir(directory);
}
}
};

export default cleanDistributionDirectories;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codebase verification

Add tests for cleanDistributionDirectories utility.

No test file clean-distribution-directories.test.ts found. Ensure proper test coverage.

🔗 Analysis chain

Verify test coverage and similar utilities.

Let's ensure this utility is properly tested and there's no duplication in the codebase.

🏁 Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Check for test file
fd "clean-distribution-directories.test.ts" packages/packem/src

# Look for similar utilities
rg -g "!*.test.ts" -g "!*.d.ts" "emptyDir|cleanDir" packages/packem/src

Length of output: 337

Comment on lines 8 to 9
if (cachePath && (await isAccessible(join(cachePath, "keystore1.json")))) {
const keyStore: Record<string, string> = await readJson(join(cachePath, "keystore.json"));
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue

Fix mismatch in keystore filename.

There's a critical inconsistency in the keystore file naming:

  • Line 8 checks for keystore1.json
  • Line 9 reads from keystore.json

This mismatch will cause the function to fail silently as it checks for one file but reads from another.

Apply this fix:

-    if (cachePath && (await isAccessible(join(cachePath, "keystore1.json")))) {
-        const keyStore: Record<string, string> = await readJson(join(cachePath, "keystore.json"));
+    if (cachePath && (await isAccessible(join(cachePath, "keystore.json")))) {
+        const keyStore: Record<string, string> = await readJson(join(cachePath, "keystore.json"));
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
if (cachePath && (await isAccessible(join(cachePath, "keystore1.json")))) {
const keyStore: Record<string, string> = await readJson(join(cachePath, "keystore.json"));
if (cachePath && (await isAccessible(join(cachePath, "keystore.json")))) {
const keyStore: Record<string, string> = await readJson(join(cachePath, "keystore.json"));

Comment on lines 7 to 41
const removeOldCacheFolders = async (cachePath: string | undefined, logger: Pail, logged: boolean): Promise<void> => {
if (cachePath && (await isAccessible(join(cachePath, "keystore1.json")))) {
const keyStore: Record<string, string> = await readJson(join(cachePath, "keystore.json"));

// eslint-disable-next-line security/detect-non-literal-fs-filename
let cacheDirectories = await readdir(cachePath, {
withFileTypes: true,
});

cacheDirectories = cacheDirectories.filter((dirent) => dirent.isDirectory());

let hasLogged = logged;

for (const dirent of cacheDirectories) {
if (!keyStore[dirent.name]) {
// eslint-disable-next-line no-await-in-loop
await rm(join(cachePath, dirent.name), {
force: true,
recursive: true,
});

if (hasLogged) {
logger.raw("\n\n");
}

logger.info({
message: "Removing " + dirent.name + " file cache, the cache key is not used anymore.",
prefix: "file-cache",
});

hasLogged = false;
}
}
}
};
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🛠️ Refactor suggestion

Add error handling for file operations.

The function should handle potential errors from file operations (readdir, rm) to prevent silent failures and provide better debugging information.

Consider wrapping the operations in try-catch blocks:

 const removeOldCacheFolders = async (cachePath: string | undefined, logger: Pail, logged: boolean): Promise<void> => {
-    if (cachePath && (await isAccessible(join(cachePath, "keystore.json")))) {
-        const keyStore: Record<string, string> = await readJson(join(cachePath, "keystore.json"));
+    if (!cachePath) {
+        return;
+    }
+
+    try {
+        if (!(await isAccessible(join(cachePath, "keystore.json")))) {
+            return;
+        }
+
+        const keyStore: Record<string, string> = await readJson(join(cachePath, "keystore.json"));
 
         // eslint-disable-next-line security/detect-non-literal-fs-filename
         const cacheDirectories = (await readdir(cachePath, { withFileTypes: true }))
             .filter((dirent) => dirent.isDirectory());
 
         let hasLogged = logged;
 
         for (const dirent of cacheDirectories) {
             if (!keyStore[dirent.name]) {
                 // eslint-disable-next-line no-await-in-loop
-                await rm(join(cachePath, dirent.name), {
-                    force: true,
-                    recursive: true,
-                });
+                try {
+                    await rm(join(cachePath, dirent.name), {
+                        force: true,
+                        recursive: true,
+                    });
+                } catch (error) {
+                    logger.error({
+                        message: `Failed to remove cache directory ${dirent.name}: ${error instanceof Error ? error.message : String(error)}`,
+                        prefix: "file-cache",
+                    });
+                    continue;
+                }
 
                 if (hasLogged) {
                     logger.raw("\n\n");
                     hasLogged = false;
                 }
 
                 logger.info({
                     message: `Removing ${dirent.name} file cache, the cache key is not used anymore.`,
                     prefix: "file-cache",
                 });
             }
         }
+    } catch (error) {
+        logger.error({
+            message: `Failed to process cache cleanup: ${error instanceof Error ? error.message : String(error)}`,
+            prefix: "file-cache",
+        });
     }
 };
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
const removeOldCacheFolders = async (cachePath: string | undefined, logger: Pail, logged: boolean): Promise<void> => {
if (cachePath && (await isAccessible(join(cachePath, "keystore1.json")))) {
const keyStore: Record<string, string> = await readJson(join(cachePath, "keystore.json"));
// eslint-disable-next-line security/detect-non-literal-fs-filename
let cacheDirectories = await readdir(cachePath, {
withFileTypes: true,
});
cacheDirectories = cacheDirectories.filter((dirent) => dirent.isDirectory());
let hasLogged = logged;
for (const dirent of cacheDirectories) {
if (!keyStore[dirent.name]) {
// eslint-disable-next-line no-await-in-loop
await rm(join(cachePath, dirent.name), {
force: true,
recursive: true,
});
if (hasLogged) {
logger.raw("\n\n");
}
logger.info({
message: "Removing " + dirent.name + " file cache, the cache key is not used anymore.",
prefix: "file-cache",
});
hasLogged = false;
}
}
}
};
const removeOldCacheFolders = async (cachePath: string | undefined, logger: Pail, logged: boolean): Promise<void> => {
if (!cachePath) {
return;
}
try {
if (!(await isAccessible(join(cachePath, "keystore.json")))) {
return;
}
const keyStore: Record<string, string> = await readJson(join(cachePath, "keystore.json"));
// eslint-disable-next-line security/detect-non-literal-fs-filename
const cacheDirectories = (await readdir(cachePath, { withFileTypes: true }))
.filter((dirent) => dirent.isDirectory());
let hasLogged = logged;
for (const dirent of cacheDirectories) {
if (!keyStore[dirent.name]) {
// eslint-disable-next-line no-await-in-loop
try {
await rm(join(cachePath, dirent.name), {
force: true,
recursive: true,
});
} catch (error) {
logger.error({
message: `Failed to remove cache directory ${dirent.name}: ${error instanceof Error ? error.message : String(error)}`,
prefix: "file-cache",
});
continue;
}
if (hasLogged) {
logger.raw("\n\n");
hasLogged = false;
}
logger.info({
message: `Removing ${dirent.name} file cache, the cache key is not used anymore.`,
prefix: "file-cache",
});
}
}
} catch (error) {
logger.error({
message: `Failed to process cache cleanup: ${error instanceof Error ? error.message : String(error)}`,
prefix: "file-cache",
});
}
};

Comment on lines +75 to +77
// @TODO: Change this readdir to @visulima/fs readdir
// eslint-disable-next-line deprecation/deprecation
.map((dirent) => join(dirent.path, dirent.name));
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue

Address TODO comment and deprecated feature usage

  1. The TODO comment indicates that readdir should be changed to use @visulima/fs.
  2. The code uses the deprecated dirent.path property.

Consider updating this to use the recommended approach:

- .map((dirent) => join(dirent.path, dirent.name));
+ .map((dirent) => join(distributionPath, dirent.name));

Would you like me to help implement the switch to @visulima/fs readdir?

Committable suggestion skipped: line range outside the PR's diff.

[
join(temporaryDirectoryPath, "/dist/png.cjs"),
join(temporaryDirectoryPath, "/dist/png.mjs"),
join(temporaryDirectoryPath, "/dist/", temporaryDirectoryPath.replace("/tmp/", ""),"/src/6b71fbe07b498a82.png"),
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue

Use path.sep for cross-platform compatibility

The hardcoded path separator /tmp/ might cause issues on Windows platforms.

- join(temporaryDirectoryPath, "/dist/", temporaryDirectoryPath.replace("/tmp/", ""),"/src/6b71fbe07b498a82.png"),
+ const tmpDirPrefix = temporaryDirectoryPath.split(path.sep).slice(0, 2).join(path.sep);
+ join(temporaryDirectoryPath, "dist", temporaryDirectoryPath.replace(tmpDirPrefix + path.sep, ""), "src", "6b71fbe07b498a82.png"),

Committable suggestion skipped: line range outside the PR's diff.

});

it("should copy the file according to destDir option", async () => {
expect.assertions(6);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codebase verification

Inconsistent assertion count

One test uses expect.assertions(6), while others use expect.assertions(5). Please ensure that this difference is intentional.

🔗 Analysis chain

Verify assertion count consistency

Most tests use expect.assertions(5), but this test uses expect.assertions(6). Please verify if this is intentional or if it should be consistent with other tests.

🏁 Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Description: Check assertion counts in test file
rg -B 2 -A 2 "expect\.assertions\([0-9]+\)" packages/packem/__tests__/intigration/url.test.ts

Length of output: 3714

@@ -435,6 +434,7 @@
"icss-utils": "^5.1.0",
"less": "^4.2.0",
"lightningcss": "1.28.1",
"mime": "4.0.4",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🛠️ Refactor suggestion

Remove duplicate mime dependency

The mime package is listed in both dependencies and devDependencies. Since it's a runtime dependency needed for data-URI handling, it should only be listed in dependencies.

-        "mime": "4.0.4",

Committable suggestion skipped: line range outside the PR's diff.

.replaceAll("\t", " ")
// eslint-disable-next-line regexp/no-unused-capturing-group
.replaceAll(/<!--(.*(?=-->))-->/g, "")
.replaceAll("'", "\\i"),
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue

Typo in string replacement: Incorrect escape sequence

The .replaceAll("'", "\\i") appears to have an incorrect escape sequence. It likely intends to escape single quotes by replacing ' with \\'. Consider updating it to properly escape single quotes.

Apply this diff to fix the typo:

-.replaceAll("'", "\\i"),
+.replaceAll("'", "\\'")
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
.replaceAll("'", "\\i"),
.replaceAll("'", "\\'"),

const hash = crypto.createHash("sha1").update(buffer).digest("hex").slice(0, 16);
const extension = extname(id);
const name = basename(id, extension);
const relativeDirectory = sourceDirectory ? relative(sourceDirectory, dirname(id)) : (dirname(id).split("/").pop() ?? "");
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🛠️ Refactor suggestion

Ensure cross-platform path compatibility

Using .split("/") to manipulate paths may cause issues on Windows systems where the path separator is \. To ensure cross-platform compatibility, consider using path utilities.

Apply this diff to improve path handling:

- const relativeDirectory = sourceDirectory ? relative(sourceDirectory, dirname(id)) : (dirname(id).split("/").pop() ?? "");
+ const relativeDirectory = sourceDirectory
+   ? relative(sourceDirectory, dirname(id))
+   : basename(dirname(id));

Ensure you have imported basename from @visulima/path:

+ import { basename, dirname, extname, join, relative } from "@visulima/path";

Committable suggestion skipped: line range outside the PR's diff.

Copy link

New and removed dependencies detected. Learn more about Socket for GitHub ↗︎

Package New capabilities Transitives Size Publisher

🚮 Removed packages: npm/[email protected]

View full report↗︎

@prisis prisis merged commit 7342aad into main Nov 12, 2024
23 of 26 checks passed
@prisis prisis deleted the add-url branch November 12, 2024 23:52
prisis pushed a commit that referenced this pull request Nov 13, 2024
…em [1.6.0](https://github.com/visulima/packem/compare/@visulima/[email protected]...@visulima/[email protected]) (2024-11-13)

### Features

* added support for url which imports files as data-URIs or ES ([#60](#60)) ([7342aad](7342aad))

### Miscellaneous Chores

* **deps:** update dependency postcss to >=8.4.47 ([#55](#55)) ([7261f2c](7261f2c))
Copy link
Contributor

This pull request has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.
Please note this issue tracker is not a help forum. We recommend using our GitHub Discussions tab for questions.

@github-actions github-actions bot locked as resolved and limited conversation to collaborators Dec 13, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant