Skip to content

Commit

Permalink
Renamed round-trip script to be more consistent with the other script…
Browse files Browse the repository at this point in the history
… names.

Added a validation step for the round-tripped file.
  • Loading branch information
david-waltermire committed Jun 6, 2019
1 parent c98a56a commit 36b591d
Show file tree
Hide file tree
Showing 3 changed files with 13 additions and 5 deletions.
7 changes: 3 additions & 4 deletions build/ci-cd/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,20 +9,19 @@ Using a CI/CD process ensures that all artifacts in [pull requests](https://gith

# Provided Scripts

The primary CircleCI supported CI/CD workflow, ```build```, is uses a series of bash scripts. The following scripts are used in this workflow:
The primary CircleCI supported CI/CD workflow, ```build```, is uses a series of bash scripts. The following scripts are used in this workflow for all PRs that are submitted:

1. [run-all.sh](run-all.sh): Runs all of the scripts in workflow order.
1. [validate-metaschema.sh](validate-metaschema.sh): Ensures that all [metaschema](https://github.com/usnistgov/OSCAL/tree/master/src/metaschema) are valid according to the [metaschema XML schema](https://github.com/usnistgov/OSCAL/blob/master/build/metaschema/lib/metaschema.xsd).
1. [generate-schema.sh](generate-schema.sh): Generates [XML schema](https://github.com/usnistgov/OSCAL/tree/master/xml/schema) and [JSON schema](https://github.com/usnistgov/OSCAL/tree/master/json/schema) files for each OSCAL model based on their [respective metaschemas](https://github.com/usnistgov/OSCAL/tree/master/src/metaschema).
1. [validate-content.sh](validate-content.sh): Validates OSCAL content in the repository's [src](https://github.com/usnistgov/OSCAL/tree/master/src) against the content's respective OSCAL model and format.
1. [generate-content-converters.sh](generate-content-converters.sh): Generates [JSON to XML](https://github.com/usnistgov/OSCAL/tree/master/xml/convert) and [XML to JSON](https://github.com/usnistgov/OSCAL/tree/master/json/convert) content conversion scripts for each OSCAL model based on their [respective metaschemas](https://github.com/usnistgov/OSCAL/tree/master/src/metaschema).
1. [copy-and-convert-content.sh](copy-and-convert-content.sh): Copies selective content from the [src/content](https://github.com/usnistgov/OSCAL/tree/master/src/content) directory, and generates instance of this content in alternate OSCAL formats using the appropriate content converters. Note __This script requires that the process have write access to the Git repository branch that the CI/CD is operating on to work completely.__
1. [roundTripXML.sh](roundTripXML.sh): Performs round trip validations of the content from XML->JSON->XML to confirm that the conversions are lossless and that converted files comply with their schemas. The same conversions are done in the inverse: JSON->XML->JSON.
1. [validate-round-trips.sh](validate-round-trips.sh): Performs round trip validations of the content from XML->JSON->XML to confirm that the conversions are lossless and that resulting files comply with their schemas.

Additional scripts are provided that generate content for the OSCAL project website, supported by the CircleCI ```build-and-deploy-site``` workflow.
Additional scripts are provided that generate content for the OSCAL project website, supported by the CircleCI ```build``` workflow on the ```master``` branch.
1. [generate-schema-documentation.sh](generate-schema-documentation.sh): This is used as part of the site generation pipeline to build HTML documentation for the OSCAL models.


# Running the Scripts Locally

The bash scripts used in the CI/CD workflow can be run locally in the correct environment. In addition to a bash shell, to run these scripts the following must be installed:
Expand Down
2 changes: 1 addition & 1 deletion build/ci-cd/run-all.sh
Original file line number Diff line number Diff line change
Expand Up @@ -14,4 +14,4 @@ bash $CIDIR/generate-schema.sh || ("Failed to generate schema files" && exit 2)
bash $CIDIR/validate-content.sh || ("Failed to validate content" && exit 3)
bash $CIDIR/generate-content-converters.sh || ("Failed to generate content converters" && exit 4)
bash $CIDIR/copy-and-convert-content.sh || ("Failed to convert content" && exit 5)
bash $CIDIR/roundTripXML.sh || ("Failed to test XML->JSON->XML round-trips" && exit 6)
bash $CIDIR/validate-round-trips.sh || ("Failed to validate XML->JSON->XML round-trips" && exit 6)
Original file line number Diff line number Diff line change
Expand Up @@ -95,6 +95,15 @@ while IFS="|" read path format model converttoformats || [[ -n "$path" ]]; do
echo "${P_OK}JSON converted to XML for '${P_END}${to_json}${P_OK}'.${P_END}"
fi

# Validate the resulting XML
schema="$working_dir/xml/schema/oscal_${model}_schema.xsd"
xmllint --noout --schema "$schema" "$output_path"
cmd_exitcode=$?
if [ $cmd_exitcode -ne 0 ]; then
echo "${P_ERROR}XML schema validation failed for '$output_path'.${P_END}"
exitcode=1
fi

# compare the XML files to see if there is data loss
echo "${P_INFO}Checking XML->JSON->XML conversion for '${P_END}${file}${P_ERROR}'.${P_END}\n"
python ${OSCALDIR}/build/ci-cd/python/xmlComparison.py "$file" "$back_to_xml"
Expand Down

0 comments on commit 36b591d

Please sign in to comment.