Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Unify README embedme Usage into a Wrapper Script #21859

Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
55 changes: 4 additions & 51 deletions common/perf-test-core/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,10 @@
<developerConnection>scm:git:https://github.com/Azure/azure-sdk-for-java.git</developerConnection>
</scm>

<properties>
<relative.path.to.eng.folder>../..</relative.path.to.eng.folder>
</properties>

<build>
<plugins>
<plugin>
Expand Down Expand Up @@ -62,57 +66,6 @@
</rules>
</configuration>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-javadoc-plugin</artifactId>
<version>3.1.1</version> <!-- {x-version-update;org.apache.maven.plugins:maven-javadoc-plugin;external_dependency} -->
<configuration>
<isOffline>true</isOffline>
<offlineLinks>
<!-- JDK APIs -->
<offlineLink>
<url>https://docs.oracle.com/javase/8/docs/api/</url>
<location>${project.basedir}/../../eng/javadoc-package-lists/java8/</location>
</offlineLink>

<!-- Reactor APIs -->
<offlineLink>
<url>https://projectreactor.io/docs/core/release/api/</url>
<location>${project.basedir}/../../eng/javadoc-package-lists/reactor-core/</location>
</offlineLink>
<offlineLink>
<url>https://projectreactor.io/docs/netty/release/api/</url>
<location>${project.basedir}/../../eng/javadoc-package-lists/reactor-netty/</location>
</offlineLink>

<!-- Jackson APIs -->
<offlineLink>
<url>https://fasterxml.github.io/jackson-annotations/javadoc/2.12/</url>
<location>${project.basedir}/../../eng/javadoc-package-lists/jackson-annotations/</location>
</offlineLink>
<offlineLink>
<url>https://fasterxml.github.io/jackson-core/javadoc/2.12/</url>
<location>${project.basedir}/../../eng/javadoc-package-lists/jackson-core/</location>
</offlineLink>
<offlineLink>
<url>https://fasterxml.github.io/jackson-databind/javadoc/2.12/</url>
<location>${project.basedir}/../../eng/javadoc-package-lists/jackson-databind/</location>
</offlineLink>

<!-- Netty APIs -->
<offlineLink>
<url>https://netty.io/4.1/api/</url>
<location>${project.basedir}/../../eng/javadoc-package-lists/netty/</location>
</offlineLink>

<!-- Avro APIs -->
<offlineLink>
<url>https://avro.apache.org/docs/current/api/java/</url>
<location>${project.basedir}/../../eng/javadoc-package-lists/apache-avro/</location>
</offlineLink>
</offlineLinks>
</configuration>
</plugin>
</plugins>
</build>

Expand Down
53 changes: 53 additions & 0 deletions eng/scripts/invoke_embedme.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,53 @@
# Copyright (c) Microsoft Corporation. All rights reserved.
# Licensed under the MIT License.

# Python version 3.4 or higher is required to run this script.

# Use case: Invokes embedme README codesnippet generation and validation regardless of OS.
#
# Flags
# --readme/-r: Path to the README.
# --verify/-v: Flag indicating to only perform a dry-run validation.
# --debug/-d: Flag indicating to perform debug level logging.
#
# For example: Generating README codesnippets for Azure Storage Blobs.
# python eng/scripts/invoke_embedme.py -r sdk/storage/azure-storage-blob/README.md
#
# For example: Valdate README codesnippets for Azure Core.
# python eng/scripts/invoke_embedme.py -r sdk/core/azure-core/README.md -v
#
# The script must be run at the root of azure-sdk-for-java.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit. You can easily check this on run-time and throw early instead of the undefined-behavior. But not such a big deal.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Or just switch to that directory based on the path of this script.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is just a standard comment in all the Python scripts used as tooling, I don't think any check the path used either.


import argparse
import os
import sys

# NPX command.
npx_command = 'npx embedme'
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should consider specifying a version at some point here because if there is every a breaking change in that tool then we will have to deal with it.


# Invoke embedme.
def invoke_embedme(readme: str, verify: bool, debug: bool):
command = npx_command

# If the passed README path was relative
command += ' ' + os.path.abspath(readme)

if verify:
command += ' --verify'

if debug:
print('Running embedme command: {}'.format(command))

sys.exit(os.system(command))

def main():
parser = argparse.ArgumentParser(description='Invokes embedme README codesnippet generation and validation regardless of OS.')
parser.add_argument('--readme', '-r', type=str, required=True, help='Path to the README')
parser.add_argument('--verify', '-v', action='store_true', help='Flag indicating to only perform a dry-run validation')
parser.add_argument('--debug', '-d', action='store_true', help='Flag indicating to perform debug level logging')
args = parser.parse_args()

invoke_embedme(args.readme, args.verify, args.debug)

if __name__ == '__main__':
main()
117 changes: 24 additions & 93 deletions sdk/parents/azure-client-sdk-parent/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -107,6 +107,7 @@
<jacoco.min.linecoverage>0.40</jacoco.min.linecoverage>
<jacoco.min.branchcoverage>0.30</jacoco.min.branchcoverage>
<jacoco.skip.coverage.check>false</jacoco.skip.coverage.check>
<relative.path.to.eng.folder>../../..</relative.path.to.eng.folder>
</properties>

<build>
Expand Down Expand Up @@ -441,9 +442,9 @@
</dependency>
</dependencies>
<configuration>
<configLocation>../../../eng/code-quality-reports/src/main/resources/checkstyle/checkstyle.xml</configLocation>
<suppressionsLocation>../../../eng/code-quality-reports/src/main/resources/checkstyle/checkstyle-suppressions.xml</suppressionsLocation>
<headerLocation>../../../eng/code-quality-reports/src/main/resources/checkstyle/java.header</headerLocation>
<configLocation>${project.basedir}/${relative.path.to.eng.folder}/eng/code-quality-reports/src/main/resources/checkstyle/checkstyle.xml</configLocation>
<suppressionsLocation>${project.basedir}/${relative.path.to.eng.folder}/eng/code-quality-reports/src/main/resources/checkstyle/checkstyle-suppressions.xml</suppressionsLocation>
<headerLocation>${project.basedir}/${relative.path.to.eng.folder}/eng/code-quality-reports/src/main/resources/checkstyle/java.header</headerLocation>
<propertyExpansion>samedir=</propertyExpansion>
<encoding>UTF-8</encoding>
<consoleOutput>true</consoleOutput>
Expand Down Expand Up @@ -548,43 +549,43 @@
<!-- JDK APIs -->
<offlineLink>
<url>https://docs.oracle.com/javase/8/docs/api/</url>
<location>${project.basedir}/../../../eng/javadoc-package-lists/java8/</location>
<location>${project.basedir}/${relative.path.to.eng.folder}/eng/javadoc-package-lists/java8/</location>
</offlineLink>

<!-- Reactor APIs -->
<offlineLink>
<url>https://projectreactor.io/docs/core/release/api/</url>
<location>${project.basedir}/../../../eng/javadoc-package-lists/reactor-core/</location>
<location>${project.basedir}/${relative.path.to.eng.folder}/eng/javadoc-package-lists/reactor-core/</location>
</offlineLink>
<offlineLink>
<url>https://projectreactor.io/docs/netty/release/api/</url>
<location>${project.basedir}/../../../eng/javadoc-package-lists/reactor-netty/</location>
<location>${project.basedir}/${relative.path.to.eng.folder}/eng/javadoc-package-lists/reactor-netty/</location>
</offlineLink>

<!-- Jackson APIs -->
<offlineLink>
<url>https://fasterxml.github.io/jackson-annotations/javadoc/2.12/</url>
<location>${project.basedir}/../../../eng/javadoc-package-lists/jackson-annotations/</location>
<location>${project.basedir}/${relative.path.to.eng.folder}/eng/javadoc-package-lists/jackson-annotations/</location>
</offlineLink>
<offlineLink>
<url>https://fasterxml.github.io/jackson-core/javadoc/2.12/</url>
<location>${project.basedir}/../../../eng/javadoc-package-lists/jackson-core/</location>
<location>${project.basedir}/${relative.path.to.eng.folder}/eng/javadoc-package-lists/jackson-core/</location>
</offlineLink>
<offlineLink>
<url>https://fasterxml.github.io/jackson-databind/javadoc/2.12/</url>
<location>${project.basedir}/../../../eng/javadoc-package-lists/jackson-databind/</location>
<location>${project.basedir}/${relative.path.to.eng.folder}/eng/javadoc-package-lists/jackson-databind/</location>
</offlineLink>

<!-- Netty APIs -->
<offlineLink>
<url>https://netty.io/4.1/api/</url>
<location>${project.basedir}/../../../eng/javadoc-package-lists/netty/</location>
<location>${project.basedir}/${relative.path.to.eng.folder}/eng/javadoc-package-lists/netty/</location>
</offlineLink>

<!-- Avro APIs -->
<offlineLink>
<url>https://avro.apache.org/docs/current/api/java/</url>
<location>${project.basedir}/../../../eng/javadoc-package-lists/apache-avro/</location>
<location>${project.basedir}/${relative.path.to.eng.folder}/eng/javadoc-package-lists/apache-avro/</location>
</offlineLink>
</offlineLinks>
<doclet>org.apidesign.javadoc.codesnippet.Doclet</doclet>
Expand Down Expand Up @@ -860,7 +861,7 @@
<name>generate-overview</name>
</property>
<file>
<exists>../../../eng/pipelines/scripts/generate_overview_from_readme.py</exists>
<exists>${project.basedir}/${relative.path.to.eng.folder}/eng/pipelines/scripts/generate_overview_from_readme.py</exists>
</file>
</activation>
<build>
Expand All @@ -880,7 +881,7 @@
<executable>python</executable>
<workingDirectory>${project.basedir}</workingDirectory>
<arguments>
<argument>../../../eng/pipelines/scripts/generate_overview_from_readme.py</argument>
<argument>${project.basedir}/${relative.path.to.eng.folder}/eng/pipelines/scripts/generate_overview_from_readme.py</argument>
<argument>--rf</argument>
<argument>${project.basedir}/README.md</argument>
<argument>--v</argument>
Expand Down Expand Up @@ -915,7 +916,7 @@
<name>inject-codesnippets</name>
</property>
<file>
<exists>../../../eng/pipelines/scripts/inject_codesnippets.py</exists>
<exists>${project.basedir}/${relative.path.to.eng.folder}/eng/pipelines/scripts/inject_codesnippets.py</exists>
</file>
</activation>
<build>
Expand All @@ -935,7 +936,7 @@
<executable>python</executable>
<workingDirectory>${project.basedir}</workingDirectory>
<arguments>
<argument>../../../eng/pipelines/scripts/inject_codesnippets.py</argument>
<argument>${project.basedir}/${relative.path.to.eng.folder}/eng/pipelines/scripts/inject_codesnippets.py</argument>
<argument>--project-dir</argument>
<argument>${project.basedir}</argument>
</arguments>
Expand All @@ -947,41 +948,6 @@
</build>
</profile>

<profile>
<!-- separate profile for windows as the executable on windows is named npx.cmd and npx on other os families -->
<id>readme-codesnippet-windows</id>
<activation>
<property>
<name>readme-codesnippet-windows</name>
</property>
</activation>
<build>
<plugins>
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>exec-maven-plugin</artifactId>
<version>1.2.1</version> <!-- {x-version-update;org.codehaus.mojo:exec-maven-plugin;external_dependency} -->
<executions>
<execution>
<id>code-snippet-for-readme-windows</id>
<phase>prepare-package</phase>
<goals>
<goal>exec</goal>
</goals>
<configuration>
<executable>npx.cmd</executable>
<arguments>
<argument>embedme</argument>
<argument>${project.basedir}/README.md</argument>
</arguments>
</configuration>
</execution>
</executions>
</plugin>
</plugins>
</build>
</profile>

<!-- Profiles for updating README file using embedme tool to inject code snippets from compilable Java source files.
Activate the profile below to update README files when corresponding Java files are updated. -->
<profile>
Expand All @@ -1005,45 +971,10 @@
<goal>exec</goal>
</goals>
<configuration>
<executable>npx</executable>
<arguments>
<argument>embedme</argument>
<argument>${project.basedir}/README.md</argument>
</arguments>
</configuration>
</execution>
</executions>
</plugin>
</plugins>
</build>
</profile>

<profile>
<!-- separate profile for windows as the executable on windows is named npx.cmd and npx on other os families -->
<id>verify-readme-windows</id>
<activation>
<property>
<name>verify-readme-windows</name>
</property>
</activation>
<build>
<plugins>
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>exec-maven-plugin</artifactId>
<version>1.2.1</version> <!-- {x-version-update;org.codehaus.mojo:exec-maven-plugin;external_dependency} -->
<executions>
<execution>
<id>verify-readme-codesnippet-windows</id>
<phase>prepare-package</phase>
<goals>
<goal>exec</goal>
</goals>
<configuration>
<executable>npx.cmd</executable>
<executable>python</executable>
<arguments>
<argument>embedme</argument>
<argument>--verify</argument>
<argument>${project.basedir}/${relative.path.to.eng.folder}/eng/scripts/invoke_embedme.py</argument>
<argument>--readme</argument>
<argument>${project.basedir}/README.md</argument>
</arguments>
</configuration>
Expand All @@ -1054,8 +985,7 @@
</build>
</profile>

<!-- Verify that there are no changes to readme file. Build fails if the there's a difference in README
after running the embedme tool. -->
<!-- Verify that there are no changes to readme file. Build fails if the there's a difference in README after running the embedme tool. -->
<profile>
<id>verify-readme</id>
<activation>
Expand All @@ -1077,11 +1007,12 @@
<goal>exec</goal>
</goals>
<configuration>
<executable>npx</executable>
<executable>python</executable>
<arguments>
<argument>embedme</argument>
<argument>--verify</argument>
<argument>${project.basedir}/${relative.path.to.eng.folder}/eng/scripts/invoke_embedme.py</argument>
<argument>--readme</argument>
<argument>${project.basedir}/README.md</argument>
<argument>--verify</argument>
</arguments>
</configuration>
</execution>
Expand Down