Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

8158 bug UI not using presign for uploading objects #8365

Merged
10 changes: 3 additions & 7 deletions webui/src/lib/api/index.js
Original file line number Diff line number Diff line change
Expand Up @@ -668,12 +668,8 @@ export const uploadWithProgress = (url, file, method = 'POST', onProgress = null
});
xhr.addEventListener('load', () => {
resolve({
status: xhr.status,
body: xhr.responseText,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These should not get deleted

contentType: xhr.getResponseHeader('Content-Type'),
Jonathan-Rosenberg marked this conversation as resolved.
Show resolved Hide resolved
etag: xhr.getResponseHeader('ETag'),
contentMD5: xhr.getResponseHeader('Content-MD5'),
Jonathan-Rosenberg marked this conversation as resolved.
Show resolved Hide resolved
})
rawHeaders: xhr.getAllResponseHeaders(), // add raw headers
});
});
xhr.addEventListener('error', () => reject(new Error('Upload Failed')));
xhr.addEventListener('abort', () => reject(new Error('Upload Aborted')));
Expand Down Expand Up @@ -1125,7 +1121,7 @@ class Statistics {

class Staging {
async get(repoId, branchId, path, presign = false) {
const query = qs({path, presign});
const query = qs({path, presign});
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

indentation looks off

const response = await apiRequest(`/repositories/${encodeURIComponent(repoId)}/branches/${encodeURIComponent(branchId)}/staging/backing?` + query, {
method: 'GET'
});
Expand Down
67 changes: 36 additions & 31 deletions webui/src/pages/repositories/repository/objects.jsx
Original file line number Diff line number Diff line change
Expand Up @@ -226,42 +226,47 @@ const ImportModal = ({config, repoId, referenceId, referenceType, path = '', onD
);
};

function extractChecksumFromResponse(response) {
if (response.contentMD5) {
// convert base64 to hex
const raw = atob(response.contentMD5)
let result = '';
for (let i = 0; i < raw.length; i++) {
const hex = raw.charCodeAt(i).toString(16);
result += (hex.length === 2 ? hex : '0' + hex);
}
return result;
function extractChecksumFromResponse(rawHeaders) {
const headersString = typeof rawHeaders === 'string' ? rawHeaders : rawHeaders.toString();
const cleanedHeadersString = headersString.trim();
const headerLines = cleanedHeadersString.split('\n');
const parsedHeaders = headerLines.reduce((acc, line) => {
let [key, ...value] = line.split(':'); // split into key and the rest of the value
key = key.trim();
value = value.join(':').trim();
if (key && value) {
acc[key.toLowerCase()] = value;
}
return acc;
}, {});
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please extract these lines to a function called parseRawHeaders (or something like that), and instead of those line run:

const parsedHeaders = parseRawHeaders(rawHeaders);
if (parsedHeaders['content-md5']) {
.....

I would actually pass the extractChecksumFromResponse function the parsed headers as an argument and let it fetch the checksum from it.

if (parsedHeaders['content-md5']) {
return parsedHeaders['content-md5'];
Jonathan-Rosenberg marked this conversation as resolved.
Show resolved Hide resolved
}

if (response.etag) {
// drop any quote and space
return response.etag.replace(/[" ]+/g, "");
// fallback to ETag
if (parsedHeaders['etag']) {
// drop any quote and space
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Indentation again 😆

return parsedHeaders['etag'].replace(/"/g, '');
}
return ""
return null;
}

const uploadFile = async (config, repo, reference, path, file, onProgress) => {
const fpath = destinationPath(path, file);
if (config.pre_sign_support_ui) {
let additionalHeaders;
if (config.blockstore_type === "azure") {
additionalHeaders = { "x-ms-blob-type": "BlockBlob" }
}
const getResp = await staging.get(repo.id, reference.id, fpath, config.pre_sign_support_ui);
const uploadResponse = await uploadWithProgress(getResp.presigned_url, file, 'PUT', onProgress, additionalHeaders)
if (uploadResponse.status >= 400) {
throw new Error(`Error uploading file: HTTP ${status}`)
const uploadFile = async (config, repo, reference, path, file, onProgress) => {
const fpath = destinationPath(path, file);
if (config.pre_sign_support_ui) {
let additionalHeaders;
if (config.blockstore_type === "azure") {
additionalHeaders = { "x-ms-blob-type": "BlockBlob" }
}
const getResp = await staging.get(repo.id, reference.id, fpath, config.pre_sign_support_ui);
const uploadResponse = await uploadWithProgress(getResp.presigned_url, file, 'PUT', onProgress, additionalHeaders);
if (uploadResponse.status >= 400) {
throw new Error(`Error uploading file: HTTP ${uploadResponse.status}`);
}
const checksum = extractChecksumFromResponse(uploadResponse.rawHeaders);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

indentation looks off

await staging.link(repo.id, reference.id, fpath, getResp, checksum, file.size, file.type);
} else {
await objects.upload(repo.id, reference.id, fpath, file, onProgress);
}
const checksum = extractChecksumFromResponse(uploadResponse)
await staging.link(repo.id, reference.id, fpath, getResp, checksum, file.size, file.type);
} else {
await objects.upload(repo.id, reference.id, fpath, file, onProgress);
}
};

const destinationPath = (path, file) => {
Expand Down
Loading