Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore: Git resource map conversions #37920

Merged
merged 3 commits into from
Dec 3, 2024
Merged

Conversation

nidhi-nair
Copy link
Contributor

@nidhi-nair nidhi-nair commented Dec 3, 2024

Description

Tip

Add a TL;DR when the description is longer than 500 words or extremely technical (helps the content, marketing, and DevRel team).

Please also include relevant motivation and context. List any dependencies that are required for this change. Add links to Notion, Figma or any other documents that might be relevant to the PR.

Fixes #Issue Number
or
Fixes Issue URL

Warning

If no issue exists, please create an issue first, and check with the maintainers if the issue is valid.

Automation

/ok-to-test tags="@tag.Git"

🔍 Cypress test results

Tip

🟢 🟢 🟢 All cypress tests have passed! 🎉 🎉 🎉
Workflow run: https://github.com/appsmithorg/appsmith/actions/runs/12145100933
Commit: c78857b
Cypress dashboard.
Tags: @tag.Git
Spec:


Tue, 03 Dec 2024 18:37:08 UTC

Communication

Should the DevRel and Marketing teams inform users about this change?

  • Yes
  • No

Summary by CodeRabbit

Release Notes

  • New Features

    • Enhanced Git resource handling with the ability to save artifacts to Git repositories.
    • Introduced new methods for managing modified resources and artifact exchange JSON.
    • Improved tracking of updated entities during export operations, including custom JavaScript libraries and new pages.
    • New functionality for handling application artifacts and contexts in Git.
  • Bug Fixes

    • Refined error handling and control flow in auto-commit and migration processes.
  • Documentation

    • Updated comments and method signatures for clarity and improved understanding.
  • Tests

    • Added new test cases to validate the conversion processes and resource comparisons.
    • Enhanced existing tests to utilize new data structures and improve clarity.

@nidhi-nair nidhi-nair added the ok-to-test Required label for CI label Dec 3, 2024
@nidhi-nair nidhi-nair requested review from sondermanish and a team as code owners December 3, 2024 14:45
Copy link
Contributor

coderabbitai bot commented Dec 3, 2024

Walkthrough

The pull request introduces significant enhancements to the handling of Git resources across multiple classes in the Appsmith project. Key modifications include the addition of new methods for saving artifacts and retrieving existing files in Git repositories, updates to constructors to incorporate ObjectMapper, and the introduction of new data structures like GitResourceMap. Additionally, several classes have been updated to improve the clarity and functionality of resource management, particularly in relation to application artifacts and export processes.

Changes

File Change Summary
app/server/appsmith-git/src/main/java/com/appsmith/git/files/FileUtilsCEImpl.java - Added methods: saveArtifactToGitRepo, fetchGitResourceMap, getExistingFilesInRepo, getGitResourceIdentity, saveResourceCommon.
- Modified constructor to include ObjectMapper.
- Updated updateEntitiesInRepo.
app/server/appsmith-git/src/main/java/com/appsmith/git/files/FileUtilsImpl.java - Updated constructor to include ObjectMapper parameter.
app/server/appsmith-git/src/main/java/com/appsmith/git/helpers/DSLTransformerHelper.java - Replaced constants with static imports for better readability.
- Updated control flow in calculateParentDirectories and other methods for improved logic.
app/server/appsmith-git/src/test/java/com/appsmith/git/helpers/FileUtilsImplTest.java - Updated constructor to include ObjectMapper in tests for FileUtilsImpl.
app/server/appsmith-interfaces/src/main/java/com/appsmith/external/dtos/ModifiedResources.java - Changed to extend ModifiedResourcesCE, removing previous fields and methods.
app/server/appsmith-interfaces/src/main/java/com/appsmith/external/dtos/ce/ModifiedResourcesCE.java - Introduced new class with methods for managing modified resources.
app/server/appsmith-interfaces/src/main/java/com/appsmith/external/git/FileInterface.java - Added method: saveArtifactToGitRepo.
app/server/appsmith-interfaces/src/main/java/com/appsmith/external/git/models/GitResourceIdentity.java - Reintroduced filePath as a non-nullable property.
app/server/appsmith-server/src/main/java/com/appsmith/server/actioncollections/exportable/ActionCollectionExportableServiceCEImpl.java - Updated getExportableEntities to improve context update handling with Git resource tracking.
app/server/appsmith-server/src/main/java/com/appsmith/server/applications/git/ApplicationGitFileUtilsCEImpl.java - Added methods for creating and setting properties in ArtifactExchangeJson. Updated logic for handling application components.
app/server/appsmith-server/src/main/java/com/appsmith/server/applications/git/ApplicationGitFileUtilsImpl.java - Updated constructor to include ObjectMapper.
app/server/appsmith-server/src/main/java/com/appsmith/server/dtos/ce/ApplicationJsonCE.java - Added methods for setting artifacts and context lists.
app/server/appsmith-server/src/main/java/com/appsmith/server/dtos/ce/ArtifactExchangeJsonCE.java - Added methods for setting artifacts and context lists in the interface.
app/server/appsmith-server/src/main/java/com/appsmith/server/dtos/ce/MappedExportableResourcesCE_DTO.java - Added new field contextNameToGitSyncIdMap.
app/server/appsmith-server/src/main/java/com/appsmith/server/git/autocommit/AutoCommitEventHandlerCEImpl.java - Updated migratePageDsl to return a new data structure and refined logic for handling page identifiers.
app/server/appsmith-server/src/main/java/com/appsmith/server/helpers/ce/ArtifactGitFileUtilsCE.java - Added methods for creating and manipulating ArtifactExchangeJson.
app/server/appsmith-server/src/main/java/com/appsmith/server/helpers/ce/CommonGitFileUtilsCE.java - Added method saveArtifactToLocalRepoNew and updated existing methods to handle new structures.
app/server/appsmith-server/src/main/java/com/appsmith/server/jslibs/exportable/CustomJSLibExportableServiceCEImpl.java - Updated getExportableEntities to track modifications for custom JavaScript libraries.
app/server/appsmith-server/src/main/java/com/appsmith/server/newactions/exportable/NewActionExportableServiceCEImpl.java - Updated getExportableEntities to track updated action identities.
app/server/appsmith-server/src/main/java/com/appsmith/server/newpages/exportable/NewPageExportableServiceCEImpl.java - Enhanced getExportableEntities to include tracking for updated page identities.
app/server/appsmith-server/src/test/java/com/appsmith/server/git/resourcemap/ExchangeJsonConversionTests.java - Replaced Gson with ObjectMapper, added new test methods for JSON conversion and serialization.
app/server/appsmith-server/src/test/java/com/appsmith/server/git/resourcemap/templates/providers/ce/ExchangeJsonTestTemplateProviderCE.java - Added new method for comparing resource counts in ArtifactExchangeJson.
app/server/appsmith-server/src/main/java/com/appsmith/server/helpers/CommonGitFileUtils.java - Updated constructor to include ObjectMapper.

Possibly related PRs

  • chore: Allow all file I/Os for git in parallel #36872: The changes in FileUtilsCEImpl.java regarding parallel processing of file I/O operations are directly related to the enhancements made in the main PR, which also modifies the FileUtilsCEImpl class.
  • chore: Added git resource map types #37746: The introduction of GitResourceIdentity, GitResourceMap, and GitResourceType in this PR aligns with the changes in the main PR that enhance handling Git resources, indicating a strong connection in functionality.
  • chore: autocommit feature branch #34062: The modifications to FileUtilsCEImpl.java in this PR, particularly related to the handling of entities in the Git repository, relate closely to the new methods and functionalities introduced in the main PR.
  • chore: skipping file change #37774: The changes in the workflow that affect file change detection may indirectly relate to the overall file handling improvements in the main PR, although the connection is less direct compared to others.

Suggested labels

Git Product, Enhancement, Task

Suggested reviewers

  • sharat87
  • abhvsn
  • ashit-rath
  • mohanarpit

"In the realm of code, where Git flows free,
New methods emerge, as bright as can be.
Artifacts saved, resources refined,
A dance of logic, beautifully aligned.
With ObjectMapper's grace, we craft and we mold,
Enhancing our tools, as stories unfold." 🎉


Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media?

❤️ Share
🪧 Tips

Chat

There are 3 ways to chat with CodeRabbit:

  • Review comments: Directly reply to a review comment made by CodeRabbit. Example:
    • I pushed a fix in commit <commit_id>, please review it.
    • Generate unit testing code for this file.
    • Open a follow-up GitHub issue for this discussion.
  • Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
    • @coderabbitai generate unit testing code for this file.
    • @coderabbitai modularize this function.
  • PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
    • @coderabbitai gather interesting stats about this repository and render them as a table. Additionally, render a pie chart showing the language distribution in the codebase.
    • @coderabbitai read src/utils.ts and generate unit testing code.
    • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.
    • @coderabbitai help me debug CodeRabbit configuration file.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (Invoked using PR comments)

  • @coderabbitai pause to pause the reviews on a PR.
  • @coderabbitai resume to resume the paused reviews.
  • @coderabbitai review to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
  • @coderabbitai full review to do a full review from scratch and review all the files again.
  • @coderabbitai summary to regenerate the summary of the PR.
  • @coderabbitai resolve resolve all the CodeRabbit review comments.
  • @coderabbitai configuration to show the current CodeRabbit configuration for the repository.
  • @coderabbitai help to get help.

Other keywords and placeholders

  • Add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.
  • Add @coderabbitai summary to generate the high-level summary at a specific location in the PR description.
  • Add @coderabbitai anywhere in the PR title to generate the title automatically.

Documentation and Community

  • Visit our Documentation for detailed information on how to use CodeRabbit.
  • Join our Discord Community to get help, request features, and share feedback.
  • Follow us on X/Twitter for updates and announcements.

@github-actions github-actions bot added the skip-changelog Adding this label to a PR prevents it from being listed in the changelog label Dec 3, 2024
sondermanish
sondermanish previously approved these changes Dec 3, 2024
Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 7

🧹 Outside diff range and nitpick comments (14)
app/server/appsmith-git/src/main/java/com/appsmith/git/helpers/DSLTransformerHelper.java (3)

51-51: Consider improving readability of the conditional expression.

The line combines multiple operations in a single statement. Consider breaking it down for better maintainability.

-String childPrefix = isCanvasWidget(childObject) ? prefix + widgetName + DELIMITER_POINT : prefix;
+String childPrefix;
+if (isCanvasWidget(childObject)) {
+    childPrefix = prefix + widgetName + DELIMITER_POINT;
+} else {
+    childPrefix = prefix;
+}

109-119: Add logging for invalid paths.

The method silently ignores invalid paths. Consider adding debug logging to help with troubleshooting.

 if (lastDirectoryIndex <= 0) {
+    log.debug("Ignoring invalid path: {}", path);
     // This is not a valid path anymore, ignore
     continue;
 }

239-248: Extract string manipulation logic into separate methods.

The multiple string replacements and regex operations make the code harder to maintain. Consider extracting these operations into well-named helper methods.

+private static String cleanMainContainerPath(String path) {
+    return path.replace(MAIN_CONTAINER, EMPTY_STRING)
+               .replace(DELIMITER_POINT, DELIMITER_PATH);
+}
+
+private static String removeCanvasWidget(String path) {
+    return path.replaceAll(CANVAS_WIDGET, EMPTY_STRING);
+}
+
 public static String getPathToWidgetFile(String key, JSONObject jsonObject, String widgetName) {
-    String childPath = key.replace(MAIN_CONTAINER, EMPTY_STRING).replace(DELIMITER_POINT, DELIMITER_PATH);
-    childPath = childPath.replaceAll(CANVAS_WIDGET, EMPTY_STRING);
+    String childPath = cleanMainContainerPath(key);
+    childPath = removeCanvasWidget(childPath);
app/server/appsmith-server/src/test/java/com/appsmith/server/git/resourcemap/templates/providers/ce/ExchangeJsonTestTemplateProviderCE.java (1)

136-173: Reduce Code Duplication in assertResourceComparisons

There is repetitive code when asserting resource counts. Consider refactoring to eliminate duplication by iterating over resource types.

app/server/appsmith-server/src/main/java/com/appsmith/server/applications/git/ApplicationGitFileUtilsCEImpl.java (1)

Line range hint 161-173: Avoid Using 'New' in Method Names

Rename saveArtifactToLocalRepoNew to reflect its purpose more clearly, such as saveArtifactToLocalRepoWithGitResourceMap.

app/server/appsmith-server/src/main/java/com/appsmith/server/helpers/ce/CommonGitFileUtilsCE.java (1)

161-173: Rename Method for Clarity

The method name saveArtifactToLocalRepoNew is unclear. Consider renaming it to saveArtifactToLocalRepoUsingGitResourceMap for better clarity.

app/server/appsmith-interfaces/src/main/java/com/appsmith/external/git/models/GitResourceIdentity.java (1)

26-27: LGTM! Consider documenting constructor behavior.

The addition of @nonnull filePath is appropriate for Git operations. Since this class uses @requiredargsconstructor, this field will be automatically added to the constructor parameters.

Consider adding a brief comment explaining the expected format/structure of filePath (e.g., relative to repo root, forward slashes, etc.).

app/server/appsmith-server/src/main/java/com/appsmith/server/helpers/ce/ArtifactGitFileUtilsCE.java (2)

16-17: Consider reordering methods for better cohesion

Group related methods together:

  1. Move createArtifactExchangeJsonObject() next to createArtifactReferenceObject()
  2. Place setArtifactDependentPropertiesInJson() next to setArtifactDependentResources()

Also applies to: 30-30


29-29: Remove unnecessary empty line

The empty line at line 29 creates unnecessary spacing between interface methods.

app/server/appsmith-server/src/test/java/com/appsmith/server/git/resourcemap/ExchangeJsonConversionTests.java (2)

104-106: Add error handling for JSON deserialization

The JSON parsing could fail silently. Consider wrapping it in a try-catch block with appropriate test failure messages.

-        ArtifactExchangeJson artifactExchangeJson =
-                objectMapper.copy().disable(MapperFeature.USE_ANNOTATIONS).readValue(artifactJson, exchangeJsonType);
+        try {
+            return objectMapper.copy()
+                    .disable(MapperFeature.USE_ANNOTATIONS)
+                    .readValue(artifactJson, exchangeJsonType);
+        } catch (IOException e) {
+            throw new RuntimeException("Failed to parse test artifact JSON: " + e.getMessage(), e);
+        }

132-145: Use try-with-resources for directory handling

The directory cleanup might be skipped if an exception occurs. Consider using try-with-resources or @tempdir.

-        Files.createDirectories(Path.of("./container-volumes/git-storage/test123"));
-
-        Mono<Path> responseMono =
-                commonGitFileUtils.saveArtifactToLocalRepoNew(Path.of("test123"), originalArtifactJson, "irrelevant");
-
-        StepVerifier.create(responseMono)
-                .assertNext(response -> {
-                    Assertions.assertThat(response).isNotNull();
-                })
-                .verifyComplete();
-
-        FileUtils.deleteDirectory(
-                Path.of("./container-volumes/git-storage/test123").toFile());
+        Path testPath = Path.of("./container-volumes/git-storage/test123");
+        try {
+            Files.createDirectories(testPath);
+            Mono<Path> responseMono =
+                    commonGitFileUtils.saveArtifactToLocalRepoNew(Path.of("test123"), originalArtifactJson, "irrelevant");
+            StepVerifier.create(responseMono)
+                    .assertNext(response -> {
+                        Assertions.assertThat(response).isNotNull();
+                    })
+                    .verifyComplete();
+        } finally {
+            FileUtils.deleteDirectory(testPath.toFile());
+        }
app/server/appsmith-git/src/test/java/com/appsmith/git/helpers/FileUtilsImplTest.java (1)

46-47: Consider configuring ObjectMapper for test scenarios

The default ObjectMapper configuration might not cover all test cases. Consider customizing it for testing edge cases.

+        ObjectMapper objectMapper = new ObjectMapper()
+                .configure(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES, false)
+                .configure(SerializationFeature.FAIL_ON_EMPTY_BEANS, false);
         fileUtils = new FileUtilsImpl(
-                gitServiceConfig, gitExecutor, fileOperations, ObservationHelper.NOOP, new ObjectMapper());
+                gitServiceConfig, gitExecutor, fileOperations, ObservationHelper.NOOP, objectMapper);
app/server/appsmith-interfaces/src/main/java/com/appsmith/external/dtos/ce/ModifiedResourcesCE.java (1)

48-60: Consider consolidating similar methods.

isResourceUpdated and isResourceUpdatedNew have nearly identical logic. Consider consolidating them into a single generic method to reduce code duplication.

-    public boolean isResourceUpdated(String resourceType, String resourceName) {
-        return StringUtils.isNotEmpty(resourceType)
-                && (isAllModified
-                        || (!CollectionUtils.isEmpty(modifiedResourceMap.get(resourceType))
-                                && modifiedResourceMap.get(resourceType).contains(resourceName)));
-    }
-
-    public boolean isResourceUpdatedNew(GitResourceType resourceType, String resourceIdentifier) {
-        return StringUtils.isNotEmpty(resourceIdentifier)
-                && (isAllModified
-                        || (!CollectionUtils.isEmpty(modifiedResourceIdentifiers.get(resourceType))
-                                && modifiedResourceIdentifiers.get(resourceType).contains(resourceIdentifier)));
-    }
+    private <T> boolean isResourceUpdatedGeneric(T resourceType, String resourceIdentifier, Map<T, Set<String>> resourceMap) {
+        return StringUtils.isNotEmpty(resourceIdentifier)
+                && (isAllModified
+                        || (!CollectionUtils.isEmpty(resourceMap.get(resourceType))
+                                && resourceMap.get(resourceType).contains(resourceIdentifier)));
+    }
+
+    public boolean isResourceUpdated(String resourceType, String resourceName) {
+        return isResourceUpdatedGeneric(resourceType, resourceName, modifiedResourceMap);
+    }
+
+    public boolean isResourceUpdatedNew(GitResourceType resourceType, String resourceIdentifier) {
+        return isResourceUpdatedGeneric(resourceType, resourceIdentifier, modifiedResourceIdentifiers);
+    }
app/server/appsmith-interfaces/src/main/java/com/appsmith/external/git/FileInterface.java (1)

38-39: Add JavaDoc for saveArtifactToGitRepo method.

Other methods in the interface have detailed documentation. Please add JavaDoc explaining the purpose, parameters, return value, and when exceptions are thrown.

+    /**
+     * Saves artifact resources to the Git repository using the provided resource map
+     * @param baseRepoSuffix path suffix used to create a repo path
+     * @param gitResourceMap map containing Git resources to be saved
+     * @param branchName target branch name
+     * @return Path where the artifact is stored
+     * @throws GitAPIException when Git operations fail
+     * @throws IOException when file operations fail
+     */
     Mono<Path> saveArtifactToGitRepo(Path baseRepoSuffix, GitResourceMap gitResourceMap, String branchName)
             throws GitAPIException, IOException;
📜 Review details

Configuration used: .coderabbit.yaml
Review profile: CHILL

📥 Commits

Reviewing files that changed from the base of the PR and between 008a946 and 28c7911.

📒 Files selected for processing (22)
  • app/server/appsmith-git/src/main/java/com/appsmith/git/files/FileUtilsCEImpl.java (10 hunks)
  • app/server/appsmith-git/src/main/java/com/appsmith/git/files/FileUtilsImpl.java (2 hunks)
  • app/server/appsmith-git/src/main/java/com/appsmith/git/helpers/DSLTransformerHelper.java (7 hunks)
  • app/server/appsmith-git/src/test/java/com/appsmith/git/helpers/FileUtilsImplTest.java (2 hunks)
  • app/server/appsmith-interfaces/src/main/java/com/appsmith/external/dtos/ModifiedResources.java (2 hunks)
  • app/server/appsmith-interfaces/src/main/java/com/appsmith/external/dtos/ce/ModifiedResourcesCE.java (1 hunks)
  • app/server/appsmith-interfaces/src/main/java/com/appsmith/external/git/FileInterface.java (2 hunks)
  • app/server/appsmith-interfaces/src/main/java/com/appsmith/external/git/models/GitResourceIdentity.java (1 hunks)
  • app/server/appsmith-server/src/main/java/com/appsmith/server/actioncollections/exportable/ActionCollectionExportableServiceCEImpl.java (5 hunks)
  • app/server/appsmith-server/src/main/java/com/appsmith/server/applications/git/ApplicationGitFileUtilsCEImpl.java (9 hunks)
  • app/server/appsmith-server/src/main/java/com/appsmith/server/applications/git/ApplicationGitFileUtilsImpl.java (2 hunks)
  • app/server/appsmith-server/src/main/java/com/appsmith/server/dtos/ce/ApplicationJsonCE.java (3 hunks)
  • app/server/appsmith-server/src/main/java/com/appsmith/server/dtos/ce/ArtifactExchangeJsonCE.java (2 hunks)
  • app/server/appsmith-server/src/main/java/com/appsmith/server/dtos/ce/MappedExportableResourcesCE_DTO.java (1 hunks)
  • app/server/appsmith-server/src/main/java/com/appsmith/server/git/autocommit/AutoCommitEventHandlerCEImpl.java (5 hunks)
  • app/server/appsmith-server/src/main/java/com/appsmith/server/helpers/ce/ArtifactGitFileUtilsCE.java (2 hunks)
  • app/server/appsmith-server/src/main/java/com/appsmith/server/helpers/ce/CommonGitFileUtilsCE.java (12 hunks)
  • app/server/appsmith-server/src/main/java/com/appsmith/server/jslibs/exportable/CustomJSLibExportableServiceCEImpl.java (2 hunks)
  • app/server/appsmith-server/src/main/java/com/appsmith/server/newactions/exportable/NewActionExportableServiceCEImpl.java (4 hunks)
  • app/server/appsmith-server/src/main/java/com/appsmith/server/newpages/exportable/NewPageExportableServiceCEImpl.java (4 hunks)
  • app/server/appsmith-server/src/test/java/com/appsmith/server/git/resourcemap/ExchangeJsonConversionTests.java (3 hunks)
  • app/server/appsmith-server/src/test/java/com/appsmith/server/git/resourcemap/templates/providers/ce/ExchangeJsonTestTemplateProviderCE.java (5 hunks)
👮 Files not reviewed due to content moderation or server errors (3)
  • app/server/appsmith-server/src/main/java/com/appsmith/server/actioncollections/exportable/ActionCollectionExportableServiceCEImpl.java
  • app/server/appsmith-server/src/main/java/com/appsmith/server/newactions/exportable/NewActionExportableServiceCEImpl.java
  • app/server/appsmith-server/src/main/java/com/appsmith/server/newpages/exportable/NewPageExportableServiceCEImpl.java
🔇 Additional comments (10)
app/server/appsmith-git/src/main/java/com/appsmith/git/helpers/DSLTransformerHelper.java (2)

19-22: LGTM! Clean static imports for commonly used constants.

The static imports improve code readability by reducing verbosity.


31-31: LGTM! Consistent usage of static import.

app/server/appsmith-server/src/main/java/com/appsmith/server/jslibs/exportable/CustomJSLibExportableServiceCEImpl.java (2)

3-3: LGTM: Import addition is appropriate

The GitResourceType import is correctly added and utilized in the resource identifier updates.


Line range hint 108-114: Verify intended behavior of isContextAgnostic parameter

The overloaded method ignores the isContextAgnostic parameter. Is this intentional or should this parameter influence the export behavior?

✅ Verification successful

Parameter isContextAgnostic is intentionally ignored

The overloaded method is part of a delegation pattern where the new method with isContextAgnostic parameter simply delegates to the original method. This is a valid implementation as the comment indicates that only unpublished JS libraries are considered for export, making the context agnostic parameter irrelevant for this specific service.

🏁 Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Search for other implementations using isContextAgnostic
ast-grep --pattern 'getExportableEntities($_, $_, $_, $_, Boolean isContextAgnostic)'

Length of output: 87


Script:

#!/bin/bash
# Let's try a broader search for the method name and check its implementations
ast-grep --pattern 'getExportableEntities($$$)'

# Also search for the interface definition
rg "getExportableEntities.*isContextAgnostic" -A 3

Length of output: 67494


Script:

#!/bin/bash
# Let's search for the method implementation in the file under review
rg -A 10 "getExportableEntities.*isContextAgnostic" app/server/appsmith-server/src/main/java/com/appsmith/server/jslibs/exportable/CustomJSLibExportableServiceCEImpl.java

# Also search for any interface or abstract class that might define this method
rg -A 10 "getExportableEntities.*isContextAgnostic" app/server/appsmith-server/src/main/java/com/appsmith/server/jslibs/exportable/

Length of output: 306


Script:

#!/bin/bash
# Let's try a broader search for the method name in the file
rg -A 10 "getExportableEntities" app/server/appsmith-server/src/main/java/com/appsmith/server/jslibs/exportable/CustomJSLibExportableServiceCEImpl.java

# Also search for the interface definition
rg -A 10 "interface.*ExportableService" app/server/appsmith-server/src/main/java/com/appsmith/server/jslibs/exportable/

Length of output: 1372

app/server/appsmith-interfaces/src/main/java/com/appsmith/external/dtos/ModifiedResources.java (1)

3-3: LGTM! Clean CE implementation.

Good implementation of CE separation pattern. The class documentation accurately describes the purpose and usage.

Also applies to: 14-14

app/server/appsmith-server/src/main/java/com/appsmith/server/dtos/ce/MappedExportableResourcesCE_DTO.java (1)

18-18: LGTM! Verify map usage in export flow.

The new contextNameToGitSyncIdMap follows the established pattern.

Let's verify the map usage in the export flow:

✅ Verification successful

Let me check the usage context of this map to ensure it's properly integrated into the export flow.


Let me check how this DTO is used in the export functionality.


Let me check one final thing to see how this DTO is populated and used.


LGTM - Map addition follows established pattern in export DTO

The new contextNameToGitSyncIdMap is properly integrated into the export flow:

  • Follows same initialization pattern as other maps in the DTO
  • Used in both full and partial export services
  • Consistent with the DTO's purpose of mapping resources during export
🏁 Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Search for usage of contextNameToGitSyncIdMap in export-related files
rg -l "contextNameToGitSyncIdMap" | grep -i "export"

Length of output: 156


Script:

#!/bin/bash
# Check the implementation details around the map usage
rg "contextNameToGitSyncIdMap" -A 5 -B 5

Length of output: 1445


Script:

#!/bin/bash
# Search for references to MappedExportableResourcesCE_DTO in export-related code
rg -l "MappedExportableResourcesCE_DTO" | grep -i "export"

# Also check for any direct usage patterns
ast-grep --pattern 'contextNameToGitSyncIdMap.put($$$)' 

Length of output: 320


Script:

#!/bin/bash
# Look for implementations that use this DTO in export context
rg -A 5 -B 5 "new MappedExportableResources.*DTO\(\)"

Length of output: 3782

app/server/appsmith-git/src/main/java/com/appsmith/git/files/FileUtilsImpl.java (1)

8-8: LGTM: Clean constructor modification

The addition of ObjectMapper follows good dependency injection practices and maintains consistency with the superclass.

Also applies to: 26-28

app/server/appsmith-server/src/main/java/com/appsmith/server/applications/git/ApplicationGitFileUtilsImpl.java (1)

9-9: LGTM: Consistent constructor modification

The ObjectMapper integration follows the same pattern as other implementations in the codebase.

Also applies to: 19-24

app/server/appsmith-server/src/main/java/com/appsmith/server/dtos/ce/ApplicationJsonCE.java (1)

127-130: 🛠️ Refactor suggestion

Add type safety checks in setArtifact method

The unchecked cast to Application could fail at runtime. Consider adding type validation before the cast.

 @Override
 public <T extends Artifact> void setArtifact(T application) {
+    if (!(application instanceof Application)) {
+        throw new IllegalArgumentException("Artifact must be an Application instance");
+    }
     this.exportedApplication = (Application) application;
 }
app/server/appsmith-server/src/main/java/com/appsmith/server/dtos/ce/ArtifactExchangeJsonCE.java (1)

33-34: LGTM! Well-designed generic methods.

The generic methods with bounded type parameters provide good flexibility while maintaining type safety.

Also applies to: 74-74

Comment on lines +97 to +107
.filter(action -> !PluginType.JS.equals(action.getPluginType())
&& action.getUnpublishedAction().getActionConfiguration() != null
&& !(action.getUnpublishedAction()
.getActionConfiguration()
.getBody()
== null
|| (action.getPluginType().equals(PluginType.REMOTE)
&& action.getUnpublishedAction()
.getActionConfiguration()
.getFormData()
== null)))
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🛠️ Refactor suggestion

Refactor Complex Condition for Readability

The filter condition is complex and hard to read. Consider extracting parts into well-named variables or helper methods to enhance readability.

Comment on lines +226 to +229
modifiedResources
.getModifiedResourceIdentifiers()
.get(GitResourceType.CONTEXT_CONFIG)
.addAll(pageIdentifiersSet);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue

Prevent Potential NullPointerException

Ensure modifiedResources.getModifiedResourceIdentifiers().get(GitResourceType.CONTEXT_CONFIG) is not null before calling addAll(). Initialize it if necessary.

Apply this diff:

modifiedResources
    .getModifiedResourceIdentifiers()
-    .get(GitResourceType.CONTEXT_CONFIG)
+    .computeIfAbsent(GitResourceType.CONTEXT_CONFIG, k -> new HashSet<>())
    .addAll(pageIdentifiersSet);
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
modifiedResources
.getModifiedResourceIdentifiers()
.get(GitResourceType.CONTEXT_CONFIG)
.addAll(pageIdentifiersSet);
modifiedResources
.getModifiedResourceIdentifiers()
.computeIfAbsent(GitResourceType.CONTEXT_CONFIG, k -> new HashSet<>())
.addAll(pageIdentifiersSet);

Comment on lines +825 to +831
filesInRepo.parallelStream()
.filter(path -> !Files.isDirectory(baseRepoPath.resolve(path)))
.forEach(filePath -> {
Tuple2<GitResourceIdentity, Object> identity = getGitResourceIdentity(baseRepoPath, filePath);

resourceMap.put(identity.getT1(), identity.getT2());
});
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue

Handle Potential Null Values

getGitResourceIdentity may return null. Add a null check before using the returned value to prevent a NullPointerException.

Apply this diff:

.stream()
    .filter(path -> !Files.isDirectory(baseRepoPath.resolve(path)))
    .forEach(filePath -> {
        Tuple2<GitResourceIdentity, Object> identity = getGitResourceIdentity(baseRepoPath, filePath);
+       if (identity != null) {
            resourceMap.put(identity.getT1(), identity.getT2());
+       }
    });
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
filesInRepo.parallelStream()
.filter(path -> !Files.isDirectory(baseRepoPath.resolve(path)))
.forEach(filePath -> {
Tuple2<GitResourceIdentity, Object> identity = getGitResourceIdentity(baseRepoPath, filePath);
resourceMap.put(identity.getT1(), identity.getT2());
});
filesInRepo.stream()
.filter(path -> !Files.isDirectory(baseRepoPath.resolve(path)))
.forEach(filePath -> {
Tuple2<GitResourceIdentity, Object> identity = getGitResourceIdentity(baseRepoPath, filePath);
if (identity != null) {
resourceMap.put(identity.getT1(), identity.getT2());
}
});

Comment on lines +148 to +151
@Override
public <T extends Context> void setContextList(List<T> contextList) {
this.pageList = (List<NewPage>) contextList;
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🛠️ Refactor suggestion

Add type safety checks in setContextList method

The unchecked cast of the entire list could fail at runtime. Consider validating list elements.

 @Override
 public <T extends Context> void setContextList(List<T> contextList) {
+    if (contextList != null && !contextList.stream().allMatch(context -> context instanceof NewPage)) {
+        throw new IllegalArgumentException("All contexts must be NewPage instances");
+    }
     this.pageList = (List<NewPage>) contextList;
 }
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
@Override
public <T extends Context> void setContextList(List<T> contextList) {
this.pageList = (List<NewPage>) contextList;
}
@Override
public <T extends Context> void setContextList(List<T> contextList) {
if (contextList != null && !contextList.stream().allMatch(context -> context instanceof NewPage)) {
throw new IllegalArgumentException("All contexts must be NewPage instances");
}
this.pageList = (List<NewPage>) contextList;
}

Comment on lines +67 to +85
public void putResource(String resourceType, String resourceName) {
if (!this.modifiedResourceMap.containsKey(resourceType)) {
this.modifiedResourceMap.put(resourceType, new HashSet<>());
}
this.modifiedResourceMap.get(resourceType).add(resourceName);
}

/**
* Adds a set of resources to the map. Will create a new set if no set found for the provided resource type.
* It'll append the resources to the set.
* @param resourceType String, type of the resource e.g. PAGE_LST
* @param resourceNames Set of String, names of the resource e.g. Home Page, About page
*/
public void putResource(String resourceType, Set<String> resourceNames) {
if (!this.modifiedResourceMap.containsKey(resourceType)) {
this.modifiedResourceMap.put(resourceType, new HashSet<>());
}
this.modifiedResourceMap.get(resourceType).addAll(resourceNames);
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue

Add null checks in putResource methods.

Both putResource methods should validate input parameters to prevent NullPointerException.

     public void putResource(String resourceType, String resourceName) {
+        if (resourceType == null || resourceName == null) {
+            return;
+        }
         if (!this.modifiedResourceMap.containsKey(resourceType)) {
             this.modifiedResourceMap.put(resourceType, new HashSet<>());
         }
         this.modifiedResourceMap.get(resourceType).add(resourceName);
     }

     public void putResource(String resourceType, Set<String> resourceNames) {
+        if (resourceType == null || resourceNames == null) {
+            return;
+        }
         if (!this.modifiedResourceMap.containsKey(resourceType)) {
             this.modifiedResourceMap.put(resourceType, new HashSet<>());
         }
         this.modifiedResourceMap.get(resourceType).addAll(resourceNames);
     }
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
public void putResource(String resourceType, String resourceName) {
if (!this.modifiedResourceMap.containsKey(resourceType)) {
this.modifiedResourceMap.put(resourceType, new HashSet<>());
}
this.modifiedResourceMap.get(resourceType).add(resourceName);
}
/**
* Adds a set of resources to the map. Will create a new set if no set found for the provided resource type.
* It'll append the resources to the set.
* @param resourceType String, type of the resource e.g. PAGE_LST
* @param resourceNames Set of String, names of the resource e.g. Home Page, About page
*/
public void putResource(String resourceType, Set<String> resourceNames) {
if (!this.modifiedResourceMap.containsKey(resourceType)) {
this.modifiedResourceMap.put(resourceType, new HashSet<>());
}
this.modifiedResourceMap.get(resourceType).addAll(resourceNames);
}
public void putResource(String resourceType, String resourceName) {
if (resourceType == null || resourceName == null) {
return;
}
if (!this.modifiedResourceMap.containsKey(resourceType)) {
this.modifiedResourceMap.put(resourceType, new HashSet<>());
}
this.modifiedResourceMap.get(resourceType).add(resourceName);
}
/**
* Adds a set of resources to the map. Will create a new set if no set found for the provided resource type.
* It'll append the resources to the set.
* @param resourceType String, type of the resource e.g. PAGE_LST
* @param resourceNames Set of String, names of the resource e.g. Home Page, About page
*/
public void putResource(String resourceType, Set<String> resourceNames) {
if (resourceType == null || resourceNames == null) {
return;
}
if (!this.modifiedResourceMap.containsKey(resourceType)) {
this.modifiedResourceMap.put(resourceType, new HashSet<>());
}
this.modifiedResourceMap.get(resourceType).addAll(resourceNames);
}

Copy link

github-actions bot commented Dec 3, 2024

Failed server tests

  • com.appsmith.server.refactors.ce.RefactoringServiceCETest#testRefactorWidgetName_forDefaultWidgetsInList_updatesBothWidgetsAndTemplateReferences

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

🧹 Outside diff range and nitpick comments (1)
app/server/appsmith-server/src/test/java/com/appsmith/server/git/autocommit/AutoCommitEventHandlerImplTest.java (1)

164-164: Consider using dynamic UUID for gitSyncId in tests

While setting gitSyncId is correct, using a dynamic UUID instead of hardcoded "p1" would ensure better test isolation.

-        newPage.setGitSyncId("p1");
+        newPage.setGitSyncId(UUID.randomUUID().toString());
📜 Review details

Configuration used: .coderabbit.yaml
Review profile: CHILL

📥 Commits

Reviewing files that changed from the base of the PR and between ca6935e and c78857b.

📒 Files selected for processing (1)
  • app/server/appsmith-server/src/test/java/com/appsmith/server/git/autocommit/AutoCommitEventHandlerImplTest.java (1 hunks)
🔇 Additional comments (1)
app/server/appsmith-server/src/test/java/com/appsmith/server/git/autocommit/AutoCommitEventHandlerImplTest.java (1)

164-164: Verify test coverage for gitSyncId scenarios

Let's verify that the test coverage remains comprehensive with the addition of gitSyncId.

✅ Verification successful

Let me check the test coverage more thoroughly by examining the actual test implementations.


Test coverage for gitSyncId is adequate

The codebase already contains comprehensive test coverage for gitSyncId in ImportServiceTests.java, including:

  • Verification of gitSyncId preservation during import/export
  • Explicit assertions checking gitSyncId equality
  • Tests covering scenarios with page addition and removal
🏁 Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Check if there are any test cases specifically validating gitSyncId behavior
rg -l "gitSyncId" --type java app/server/appsmith-server/src/test/java/

# Check for any existing assertions on gitSyncId
ast-grep --pattern 'assertThat($x.getGitSyncId())' 

Length of output: 226


Script:

#!/bin/bash
# Check the content of test files containing gitSyncId
rg "gitSyncId" -B 2 -A 2 app/server/appsmith-server/src/test/java/com/appsmith/server/git/autocommit/AutoCommitEventHandlerImplTest.java
rg "gitSyncId" -B 2 -A 2 app/server/appsmith-server/src/test/java/com/appsmith/server/imports/internal/ImportServiceTests.java

# Look for any test methods that might be testing gitSyncId functionality
ast-grep --pattern 'void $testMethod() {
  $$$
  gitSyncId
  $$$
}'

Length of output: 1585

Copy link

github-actions bot commented Dec 3, 2024

Failed server tests

  • com.appsmith.server.services.LayoutActionServiceTest#OnLoadActionsWhenActionDependentOnWidgetButNotPageLoadCandidate

@nidhi-nair
Copy link
Contributor Author

Force merging since changes since approval are for compile errors and test changes

@nidhi-nair nidhi-nair merged commit a2c5caa into release Dec 3, 2024
43 checks passed
@nidhi-nair nidhi-nair deleted the chore/git-map-to-json branch December 3, 2024 18:31
github-actions bot pushed a commit to Zeral-Zhang/appsmith that referenced this pull request Dec 9, 2024
## Description
> [!TIP]  
> _Add a TL;DR when the description is longer than 500 words or
extremely technical (helps the content, marketing, and DevRel team)._
>
> _Please also include relevant motivation and context. List any
dependencies that are required for this change. Add links to Notion,
Figma or any other documents that might be relevant to the PR._


Fixes #`Issue Number`  
_or_  
Fixes `Issue URL`
> [!WARNING]  
> _If no issue exists, please create an issue first, and check with the
maintainers if the issue is valid._

## Automation

/ok-to-test tags="@tag.Git"

### 🔍 Cypress test results
<!-- This is an auto-generated comment: Cypress test results  -->
> [!IMPORTANT]
> 🟣 🟣 🟣 Your tests are running.
> Tests running at:
<https://github.com/appsmithorg/appsmith/actions/runs/12145100933>
> Commit: c78857b
> Workflow: `PR Automation test suite`
> Tags: `@tag.Git`
> Spec: ``
> <hr>Tue, 03 Dec 2024 17:36:22 UTC
<!-- end of auto-generated comment: Cypress test results  -->


## Communication
Should the DevRel and Marketing teams inform users about this change?
- [ ] Yes
- [ ] No


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

## Release Notes

- **New Features**
- Enhanced Git resource handling with the ability to save artifacts to
Git repositories.
- Introduced new methods for managing modified resources and artifact
exchange JSON.
- Improved tracking of updated entities during export operations,
including custom JavaScript libraries and new pages.
- New functionality for handling application artifacts and contexts in
Git.

- **Bug Fixes**
- Refined error handling and control flow in auto-commit and migration
processes.

- **Documentation**
- Updated comments and method signatures for clarity and improved
understanding.

- **Tests**
- Added new test cases to validate the conversion processes and resource
comparisons.
- Enhanced existing tests to utilize new data structures and improve
clarity.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
ok-to-test Required label for CI skip-changelog Adding this label to a PR prevents it from being listed in the changelog
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants