Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update postman metadata #3852

Merged
merged 13 commits into from
Jan 30, 2025
Merged

Update postman metadata #3852

merged 13 commits into from
Jan 30, 2025

Conversation

casey-tran
Copy link
Contributor

@casey-tran casey-tran commented Jan 23, 2025

Description:

  • Updated Postman metadata fields to contain location uniqueness and took out the unused fields of global_id, field_name, and variable_type.
  • Disabled body scanning for now since the only body that is scanned is the currently selected radio button but secrets can still be saved in the other unselected radio button options.
  • Updated link generation for more accuracy.
  • Updated tests to not use global constant.

Checklist:

  • Tests passing (make test-community)?
  • Lint passing (make lint this requires golangci-lint)?

@casey-tran casey-tran requested a review from joeleonjr January 29, 2025 15:41
@casey-tran casey-tran marked this pull request as ready for review January 29, 2025 15:41
@casey-tran casey-tran requested review from a team as code owners January 29, 2025 15:41
Copy link
Collaborator

@rosecodym rosecodym left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

cool stuff! i left comment reminders of what we discussed synchronously. let's also wait for joe to sign-off on the request body stuff.

Comment on lines 114 to 116
if reflect.TypeOf(obj) == reflect.TypeOf(&source_metadatapb.MetaData_Postman{}) {
m["Postman"]["location"] = obj.(*source_metadatapb.MetaData_Postman).Postman.Location.String()
}
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

probably no need to reflect; just check the generated dictionary

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

also pls add an explanatory (apologetic) comment

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

Comment on lines 277 to 283
metadata.Location = source_metadatapb.PostmanLocation_UNKNOWN_POSTMAN
ctx.Logger().V(2).Info("finished scanning environment vars", "environment_uuid", metadata.FullID)
metadata.Type = ""
metadata.Link = ""
metadata.FullID = ""
metadata.EnvironmentID = ""
metadata.EnvironmentName = ""
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

log before or after, but not in the middle!

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

string request_name = 9;
string folder_id = 10;
string folder_name = 11;
string field_type = 12;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

leave a comment saying that this field is used for local output but not in transport

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

PostmanLocation location = 13;
}

enum PostmanLocation {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

rename to PostmanLocationType

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

string folder_id = 10;
string folder_name = 11;
string field_type = 12;
PostmanLocation location = 13;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

rename to location_type

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

Comment on lines -312 to -313
string field_name = 14;
string variable_type = 15;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done and for globals_id as well. I also reserved the names too.

}
// We would scan the body, but currently the body has different radio buttons that can be scanned but only the selected one is scanned. The unselected radio button options can still
// have secrets in them but will not be scanned. The selction of the radio button will also change the secret metadata for that particular scanning pass and can create confusion for
// the user as to the status of a secret. We will reimplement at some point.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm a little unclear - does the API prevent us from scanning the body data from the unselected fields?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The Postman API will need further exploration to answer that question Joe. As it is written in main currently, I'm not seeing unselected fields getting scanned.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Understood. I think it's still worth it to scan the body data that we can see, but if that confuses the user, then I understand punting it down the road. I'll just add that if the Postman API doesn't allow us to scan all body types now, it likely never will, so the existing implementation might be the only option.

Copy link
Collaborator

@rosecodym rosecodym left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

great work man!

@@ -110,6 +110,16 @@ func structToMap(obj any) (m map[string]map[string]any, err error) {
return
}
err = json.Unmarshal(data, &m)
// Due to PostmanLocationType protobuf field being an enum, we want to be able to assign the string value of the enum to the field without needing to create another Protobuf field.
// To have the "UNKNOWN_POSTMAN = 0" value be assigned correctly to the field, we need to check if the Postman workspace ID is filled since every secret in the Postman source
// should have a valid workspace ID and the 0 value is considered nil for integers.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I love this comment - it explains the problem as well as why you picked the solution you did.

@casey-tran casey-tran merged commit 9ecaf07 into main Jan 30, 2025
13 checks passed
@casey-tran casey-tran deleted the update-postman-metadata branch January 30, 2025 15:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

Successfully merging this pull request may close these issues.

3 participants