-
Notifications
You must be signed in to change notification settings - Fork 66
Conversation
docs/sdk-examples.md
Outdated
|
||
```python | ||
# define a custom script | ||
custom_script = aztk.spark.models.CustomScript( |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Since custom scripts are going away in favor of plugins, we should probably leave this out.
docs/sdk-examples.md
Outdated
status = client.get_application_status(cluster_config.cluster_id, app2.name) | ||
``` | ||
|
||
## stream logs of app, print to console as it runs |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
"stream" should be "Stream"
docs/sdk-examples.md
Outdated
``` | ||
|
||
|
||
## Run application against cluster |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I feel like "Run an application on the cluster" is better than "against".
docs/index.rst
Outdated
@@ -0,0 +1,40 @@ | |||
Welcome to aztk's documentation! | |||
================================ | |||
Azure Distributed Data Engineering Toolkit (AZTK) is a python CLI application for provisioning on-demand Spark on Docker clusters in Azure. It's a cheap and easy way to get up and running with a Spark cluster, and a great tool for Spark users who want to experiment and start testing at scale. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We should pick between aztk
and AZTK
and standardize in our docs.
In the SDK docs, I think the only package we should have is the edit: and the aztk.error module |
@@ -13,25 +13,25 @@ Creating a Job starts with defining the necessary properties in your `.aztk/job. | |||
Each Job has one or more applications given as a List in Job.yaml. Applications are defined using the following properties: | |||
```yaml | |||
applications: | |||
- name: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I thought we decided to leave in white spaces in yaml since it is required.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hhm i guess its the auto formatting that removed those
@@ -13,6 +13,14 @@ pylint==1.8.2 | |||
pytest==3.1.3 | |||
pytest-xdist==1.22.0 | |||
twine==1.9.1 | |||
docker==3.2.1 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
why is this here?
fix #496
http://aztk.readthedocs.io/en/feature-readthedocs/