Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use correct VMDK sizes when building OVA #1831

Merged
merged 3 commits into from
Jun 22, 2018

Conversation

zjs
Copy link
Member

@zjs zjs commented Jun 19, 2018

The builder OVF used to construct the OVA has hard-coded disk sizes. Correct these to match the final size of the VMDKs to avoid warnings during the OVA deployment process.

VIC Appliance Checklist:

  • Up to date with master branch
  • Added tests
  • Considered impact to upgrade
  • Tests passing
  • Updated documentation
  • Impact assessment checklist

If this is a feature or change to existing functionality, consider areas of impact with the Impact
Assessment Checklist

Fixes #1166

@zjs zjs self-assigned this Jun 19, 2018
@zjs zjs requested review from hickeng and lcastellano June 19, 2018 03:59
@@ -135,7 +135,10 @@ function main {
log1 "packaging OVA..."
cp "${DIR}"/config/builder.ovf "${PACKAGE}/vic-${BUILD_OVA_REVISION}.ovf"
cd "${PACKAGE}"
sed -i -e s~--version--~${BUILD_OVA_REVISION}~ vic-${BUILD_OVA_REVISION}.ovf
sed -i -e s~--version--~${BUILD_OVA_REVISION}~ \
-e "/<File.*vic-disk1.vmdk.*/s|ovf:size=\"[^\"]*\"|ovf:size=\"$(stat --printf="%s" vic-disk1.vmdk)\"|" \
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please use the same separator characters as above and a space between the filter and the replacement:
-e "/<File.*vic-disk1.vmdk.*/ s~ovf:size=\"[^\"]*\"~ovf:size=\"$(stat --printf="%s" vic-disk1.vmdk)\"~"

Add an array of the expected disks and loop over it. Multiple sed passes are a fine cost to pay for increased clarity.

log2 "updating version number"
sed -i -e "s|--version--|${BUILD_OVA_REVISION}|" vic-${BUILD_OVA_REVISION}.ovf
log2 "updating disk sizes"
DISKS=("vic-disk1.vmdk" "vic-disk2.vmdk")
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't like listing the disk names in three places (the OVF, build-main.sh, and build-disks.sh), like my current change does.

However, one thing that I like about the current design is that build-disks.sh and build-main.sh seem to have fairly separate purposes: one creates the VMDKs, while the other bundles them into an OVA.

I'm not sure how important that distinction is in practice, but I worry that moving this logic into build-disks.sh blurs the line too much.

Perhaps build-main.sh should update the disk sizes, but not have a hardcoded value for DISKS. Instead, it could just base the array on the contents of the directory. This is a little fragile (what happens if there's something unexpected in the directory?), but may be better than explicitly listing the names in multiple places. (On the other hand, explicitly referring to them my name makes it easy for someone to grep for them by name and find this logic.)

Copy link

@ghost ghost Jun 21, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fair concerns about blurring the separation of logic between build-main and build-disks.

My original intent was to have all bull attributes (disk size, names, etc) parsed and read from ether a json file or the ovf template in build-main, and passed to the other builder scripts.

Something similar could be done here - just pass the newly-defined disk array you’ve added in build-main as an argument in calls to build-disk. That preserves the separation of logic while removing one repeated definition of the disks array.

Furthermore, it serves as a good stepping stone to eventually reading those values from the ovf config in build-main and passing them to any desired subsequent build step.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I did basically what you describe, except to avoid separating the disk names from the other information about them, I moved all three arrays into build-main. I think this is consistent with the idea of eventually reading the values from somewhere (JSON or OVF).

@zjs zjs force-pushed the topic/rewrite-vmdk-sizes branch from 22cc2ea to 514f8ae Compare June 21, 2018 19:08
zjs added 2 commits June 21, 2018 14:54
The builder OVF used to construct the OVA has hard-coded disk sizes.
Correct these to match the final size of the VMDKs to avoid warnings
during the OVA deployment process.
Refactor to pass image data from build-main to build-disks to avoid
duplication of image names.
@zjs zjs force-pushed the topic/rewrite-vmdk-sizes branch from 2464a5e to d680c8a Compare June 21, 2018 21:55

# check there were no extra args and the required ones are set
if [[ -n "$*" || -z "${PACKAGE}" || -z "${ACTION}" ]]; then
if [[ -n "$*" || -z "${PACKAGE}" || -z "${ACTION}" || ${#IMAGES[@]} -eq 0 || ${#IMAGESIZES[@]} -eq 0 || ${#IMAGEROOTS[@]} -eq 0 ]]; then
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

From a purely pedantic perspective, these should be tested for equality with each other as well as !0

sha256sum --tag "vic-${BUILD_OVA_REVISION}.ovf" *.vmdk | sed s/SHA256\ \(/SHA256\(/ > "vic-${BUILD_OVA_REVISION}.mf"
tar -cvf "${RESOURCE}/vic-${BUILD_OVA_REVISION}.ova" "vic-${BUILD_OVA_REVISION}.ovf" "vic-${BUILD_OVA_REVISION}.mf" *.vmdk
sha256sum --tag "vic-${BUILD_OVA_REVISION}.ovf" "${IMAGEFILES[@]}" | sed s/SHA256\ \(/SHA256\(/ > "vic-${BUILD_OVA_REVISION}.mf"
tar -cvf "${RESOURCE}/vic-${BUILD_OVA_REVISION}.ova" "vic-${BUILD_OVA_REVISION}.ovf" "vic-${BUILD_OVA_REVISION}.mf" "${IMAGEFILES[@]}"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This cannot be compressed?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Filed #1837 to track.

@zjs zjs merged commit 62bccb8 into vmware:master Jun 22, 2018
zjs added a commit to zjs/vic-product that referenced this pull request Jun 28, 2018
The builder OVF used to construct the OVA has hard-coded disk sizes.
Correct these to match the final size of the VMDKs to avoid warnings
during the OVA deployment process.

Refactor to pass image data from build-main to build-disks to avoid
duplication of image names.

(cherry picked from commit 62bccb8)
zjs added a commit to zjs/vic-product that referenced this pull request Jun 28, 2018
The builder OVF used to construct the OVA has hard-coded disk sizes.
Correct these to match the final size of the VMDKs to avoid warnings
during the OVA deployment process.

Refactor to pass image data from build-main to build-disks to avoid
duplication of image names.

(cherry picked from commit 62bccb8)
zjs added a commit to zjs/vic-product that referenced this pull request Jul 2, 2018
The builder OVF used to construct the OVA has hard-coded disk sizes.
Correct these to match the final size of the VMDKs to avoid warnings
during the OVA deployment process.

Refactor to pass image data from build-main to build-disks to avoid
duplication of image names.

(cherry picked from commit 62bccb8)
zjs added a commit to zjs/vic-product that referenced this pull request Jul 3, 2018
The builder OVF used to construct the OVA has hard-coded disk sizes.
Correct these to match the final size of the VMDKs to avoid warnings
during the OVA deployment process.

Refactor to pass image data from build-main to build-disks to avoid
duplication of image names.

(cherry picked from commit 62bccb8)
zjs added a commit to zjs/vic-product that referenced this pull request Jul 17, 2018
The builder OVF used to construct the OVA has hard-coded disk sizes.
Correct these to match the final size of the VMDKs to avoid warnings
during the OVA deployment process.

Refactor to pass image data from build-main to build-disks to avoid
duplication of image names.

(cherry picked from commit 62bccb8)
zjs added a commit that referenced this pull request Jul 17, 2018
The builder OVF used to construct the OVA has hard-coded disk sizes.
Correct these to match the final size of the VMDKs to avoid warnings
during the OVA deployment process.

Refactor to pass image data from build-main to build-disks to avoid
duplication of image names.

(cherry picked from commit 62bccb8)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants