Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

IDSEQ-1178 - Stronger validation for input filenames #2538

Merged
merged 6 commits into from
Aug 21, 2019

Conversation

davidrissato
Copy link
Contributor

Description

Add stronger validation for input file names.

File names must be no longer than 120 characters and can only contain letters from English alphabet (A-Z, upper and lower case), numbers (0-9), periods (.), hyphens (-) and underscores (_). Spaces are not allowed.

I'm also replacing a few aws cli commands by their AWS ruby SDK counterparts.

Tests

  • Manual tests

@davidrissato davidrissato requested a review from a team August 21, 2019 03:11
Copy link
Contributor

@jshoe jshoe left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Small comments + a question about the "Undetermined" files

@@ -129,6 +129,12 @@ class LocalSampleFileUpload extends React.Component {
Paired files must be labeled with "_R1" or
"_R2" at the end of the basename.
</li>
<li>
File names must be no longer than 120 characters and can only
contain letters from English alphabet (A-Z, upper and lower
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

super nit: *the English alphabet ? (and below)

app/helpers/samples_helper.rb Outdated Show resolved Hide resolved
app/helpers/samples_helper.rb Show resolved Hide resolved
Copy link
Contributor

@jshoe jshoe left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Copy link
Contributor

@tfrcarvalho tfrcarvalho left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we do this verification on the cli too, before trying to upload?

@@ -129,6 +129,12 @@ class LocalSampleFileUpload extends React.Component {
Paired files must be labeled with &quot;_R1&quot; or
&quot;_R2&quot; at the end of the basename.
</li>
<li>
File names must be no longer than 120 characters and can only
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you make this text a constant for reuse?

Copy link
Contributor Author

@davidrissato davidrissato Aug 21, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You mean, the entire text or just this new sentence? Because it seems there are other lines that can be reused as well.
Do you think this is a blocker for this PR or is it a sanitization task that we should consider a specific PR dedicated to remove such redundancies?

Copy link
Contributor

@tfrcarvalho tfrcarvalho Aug 21, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would prefer to do it since it is a new sentence. It is not the problem of redundancy but keeping it in sync.
I see your point but I prefer to start doing it and set an example than keep following a bad pattern.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Considering this is not only about this sentence, but the entire block of instructions (the previous 2 <li> items are exactly the same), would it make sense to create a new react component with these instructions instead of using a constant?

Copy link
Contributor Author

@davidrissato davidrissato Aug 21, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@tfrcarvalho Do we have any pre-existing pattern that I should follow in this case?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, you got me convinced. Let's just make a low priority jira task to just consolidate this

app/helpers/samples_helper.rb Outdated Show resolved Hide resolved
case), numbers (0-9), periods (.), hyphens (-) and underscores
(_). Spaces are not allowed.
</li>
<li>File names must be no longer than 140 characters.</li>
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This line is not included above? Any particular reason?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, actually that line should have been removed, since the previous item already includes it. Thank you for pointing it out.

@davidrissato
Copy link
Contributor Author

davidrissato commented Aug 21, 2019

Tiago wrote:
Should we do this verification on the cli too, before trying to upload?

That's a good point. I created a new ticket for that: IDSEQ-1330

@czimergebot czimergebot merged commit ce3e632 into master Aug 21, 2019
@davidrissato davidrissato deleted the davidrc/IDSEQ-1178 branch August 21, 2019 20:35
czimergebot pushed a commit that referenced this pull request Oct 30, 2019
[IDSEQ-1643] Fix bug for cross-region S3 uploads### Description
- This is just to address the cross-region copying issue from last week. See JIRA for details.
- Using the S3 "global" non-regional specific endpoint means that AWS will resolve the region itself.

### Notes
- This was the most elegant solution I could find. My original proposal was to use `GetBucketLocation` but apparently it is only for the bucket owner (https://docs.aws.amazon.com/AmazonS3/latest/API/API_GetBucketLocation.html). I also found suggestions to use head bucket but got 301 redirects / couldn't figure out how to get the region info from that (aws/aws-sdk-ruby#796).
- This was the only use of S3_CLIENT that I found necessary to change, and it was the only one changed in the previous PR (#2538).
- Why not set this global endpoint all the time? AWS recommends against it "Please configure the proper region to avoid multiple unnecessary redirects and signing attempts". And for most other requests we're in our own buckets in our known region.

### Tests
- Testing with the reported user URL.

#### Before:
![Screen Shot 2019-10-29 at 2 59 37 PM](https://user-images.githubusercontent.com/5652739/67895204-4a3f6900-fb17-11e9-8a60-c4b9b05d36b6.png)

#### After:
![Screen Shot 2019-10-29 at 3 02 12 PM](https://user-images.githubusercontent.com/5652739/67895214-4f9cb380-fb17-11e9-81b9-698bc63ff13d.png)

#### Ran samples end-to-end locally and it works: 
<img width="1088" alt="Screen Shot 2019-10-30 at 12 53 59 PM" src="https://user-images.githubusercontent.com/5652739/67895228-54f9fe00-fb17-11e9-9d5b-69c19cbcf898.png">
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants