Merge pull request #1 from spulec/master

Align

.coveragerc

@@ -3,6 +3,7 @@
 exclude_lines =
     if __name__ == .__main__.:
     raise NotImplemented.
+    return NotImplemented
     def __repr__
 
 [run]

.gitignore

@@ -22,3 +22,5 @@ tests/file.tmp
 .eggs/
 .mypy_cache/
 *.tmp
+.venv/
+htmlcov/

.travis.yml

@@ -1,4 +1,4 @@
-dist: bionic
+dist: focal
 language: python
 services:
 - docker
@@ -26,11 +26,13 @@ install:
     fi
     docker run --rm -t --name motoserver -e TEST_SERVER_MODE=true -e AWS_SECRET_ACCESS_KEY=server_secret -e AWS_ACCESS_KEY_ID=server_key -v `pwd`:/moto -p 5000:5000 -v /var/run/docker.sock:/var/run/docker.sock python:${PYTHON_DOCKER_TAG} /moto/travis_moto_server.sh &
   fi
+  travis_retry pip install -r requirements-dev.txt
+  travis_retry pip install docker>=2.5.1
   travis_retry pip install boto==2.45.0
   travis_retry pip install boto3
   travis_retry pip install dist/moto*.gz
   travis_retry pip install coveralls==1.1
-  travis_retry pip install -r requirements-dev.txt
+  travis_retry pip install coverage==4.5.4
 
   if [ "$TEST_SERVER_MODE" = "true" ]; then
     python wait_for.py

CHANGELOG.md

@@ -1,6 +1,224 @@
 Moto Changelog
 ===================
 
+Unreleased
+-----
+    * Reduced dependency overhead.
+      It is now possible to install dependencies for only specific services using:
+      pip install moto[service1,service1].
+      See the README for more information.
+
+
+
+1.3.16
+-----
+Full list of PRs merged in this release:
+https://github.com/spulec/moto/pulls?q=is%3Apr+is%3Aclosed+merged%3A2019-11-14..2020-09-07
+
+
+    General Changes:
+        * The scaffold.py-script has been fixed to make it easier to scaffold new services.
+          See the README for an introduction.
+
+    New Services:
+        * Application Autoscaling
+        * Code Commit
+        * Code Pipeline
+        * Elastic Beanstalk
+        * Kinesis Video
+        * Kinesis Video Archived Media
+        * Managed BlockChain
+        * Resource Access Manager (ram)
+        * Sagemaker
+
+    New Methods:
+        * Athena:
+            * create_named_query
+            * get_named_query
+            * get_work_group
+            * start_query_execution
+            * stop_query_execution
+        * API Gateway:
+            * create_authorizer
+            * create_domain_name
+            * create_model
+            * delete_authorizer
+            * get_authorizer
+            * get_authorizers
+            * get_domain_name
+            * get_domain_names
+            * get_model
+            * get_models
+            * update_authorizer
+        * Autoscaling:
+            * enter_standby
+            * exit_standby
+            * terminate_instance_in_auto_scaling_group
+        * CloudFormation:
+            * get_template_summary
+        * CloudWatch:
+            * describe_alarms_for_metric
+            * get_metric_data
+        * CloudWatch Logs:
+            * delete_subscription_filter
+            * describe_subscription_filters
+            * put_subscription_filter
+        * Cognito IDP:
+            * associate_software_token
+            * create_resource_server
+            * confirm_sign_up
+            * initiate_auth
+            * set_user_mfa_preference
+            * sign_up
+            * verify_software_token
+        * DynamoDB:
+            * describe_continuous_backups
+            * transact_get_items
+            * transact_write_items
+            * update_continuous_backups
+        * EC2:
+            * create_vpc_endpoint
+            * describe_vpc_classic_link
+            * describe_vpc_classic_link_dns_support
+            * describe_vpc_endpoint_services
+            * disable_vpc_classic_link
+            * disable_vpc_classic_link_dns_support
+            * enable_vpc_classic_link
+            * enable_vpc_classic_link_dns_support
+            * register_image
+        * ECS:
+            * create_task_set
+            * delete_task_set
+            * describe_task_set
+            * update_service_primary_task_set
+            * update_task_set
+        * Events:
+            * delete_event_bus
+            * create_event_bus
+            * list_event_buses
+            * list_tags_for_resource
+            * tag_resource
+            * untag_resource
+        * Glue:
+            * get_databases
+        * IAM:
+            * delete_group
+            * delete_instance_profile
+            * delete_ssh_public_key
+            * get_account_summary
+            * get_ssh_public_key
+            * list_user_tags
+            * list_ssh_public_keys
+            * update_ssh_public_key
+            * upload_ssh_public_key
+        * IOT:
+            * cancel_job
+            * cancel_job_execution
+            * create_policy_version
+            * delete_job
+            * delete_job_execution
+            * describe_endpoint
+            * describe_job_execution
+            * delete_policy_version
+            * get_policy_version
+            * get_job_document
+            * list_attached_policies
+            * list_job_executions_for_job
+            * list_job_executions_for_thing
+            * list_jobs
+            * list_policy_versions
+            * set_default_policy_version
+            * register_certificate_without_ca
+        * KMS:
+            * untag_resource
+        * Lambda:
+            * delete_function_concurrency
+            * get_function_concurrency
+            * put_function_concurrency
+        * Organisations:
+            * describe_create_account_status
+            * deregister_delegated_administrator
+            * disable_policy_type
+            * enable_policy_type
+            * list_delegated_administrators
+            * list_delegated_services_for_account
+            * list_tags_for_resource
+            * register_delegated_administrator
+            * tag_resource
+            * untag_resource
+            * update_organizational_unit
+        * S3:
+            * delete_bucket_encryption
+            * delete_public_access_block
+            * get_bucket_encryption
+            * get_public_access_block
+            * put_bucket_encryption
+            * put_public_access_block
+        * S3 Control:
+            * delete_public_access_block
+            * get_public_access_block
+            * put_public_access_block
+        * SecretsManager:
+            * get_resource_policy
+            * update_secret
+        * SES:
+            * create_configuration_set
+            * create_configuration_set_event_destination
+            * create_receipt_rule_set
+            * create_receipt_rule
+            * create_template
+            * get_template
+            * get_send_statistics
+            * list_templates
+        * STS:
+            * assume_role_with_saml
+        * SSM:
+            * create_documen
+            * delete_document
+            * describe_document
+            * get_document
+            * list_documents
+            * update_document
+            * update_document_default_version
+        * SWF:
+            * undeprecate_activity_type
+            * undeprecate_domain
+            * undeprecate_workflow_type
+
+    General Updates:
+        * API Gateway - create_rest_api now supports policy-parameter
+        * Autoscaling - describe_auto_scaling_instances now supports InstanceIds-parameter
+        * AutoScalingGroups - now support launch templates
+        * CF - Now supports DependsOn-configuration
+        * CF - Now supports FN::Transform AWS::Include mapping
+        * CF - Now supports update and deletion of Lambdas
+        * CF - Now supports creation, update and deletion of EventBus (Events)
+        * CF - Now supports update of Rules (Events)
+        * CF - Now supports creation, update and deletion of EventSourceMappings (AWS Lambda)
+        * CF - Now supports update and deletion of Kinesis Streams
+        * CF - Now supports creation of DynamoDB streams
+        * CF - Now supports deletion of  DynamoDB tables
+        * CF - list_stacks now supports the status_filter-parameter
+        * Cognito IDP - list_users now supports filter-parameter
+        * DynamoDB - GSI/LSI's now support ProjectionType=KEYS_ONLY
+        * EC2 - create_route now supports the NetworkInterfaceId-parameter
+        * EC2 - describe_instances now supports additional filters (owner-id)
+        * EC2 - describe_instance_status now supports additional filters (instance-state-name, instance-state-code)
+        * EC2 - describe_nat_gateways now supports additional filters (nat-gateway-id, vpc-id, subnet-id, state)
+        * EC2 - describe_vpn_gateways now supports additional filters (attachment.vpc_id, attachment.state, vpn-gateway-id, type)
+        * IAM - list_users now supports path_prefix-parameter
+        * IOT - list_thing_groups now supports parent_group, name_prefix_filter, recursive-parameters
+        * S3 - delete_objects now supports deletion of specific VersionIds
+        * SecretsManager - list_secrets now supports filters-parameter
+        * SFN - start_execution now receives and validates input
+        * SNS - Now supports sending a message directly to a phone number
+        * SQS - MessageAttributes now support labeled DataTypes
+
+1.3.15
+-----
+
+This release broke dependency management for a lot of services - please upgrade to 1.3.16.
+
 1.3.14
 -----
 

CONFIG_README.md

@@ -23,8 +23,8 @@ However, this will only work on resource types that have this enabled.
 
 ### Current enabled resource types:
 
-1. S3
-
+1. S3 (all)
+1. IAM (Role, Policy)
 
 ## Developer Guide
 
@@ -53,15 +53,14 @@ An example of the above is implemented for S3. You can see that by looking at:
 1. `moto/s3/config.py`
 1. `moto/config/models.py`
 
-As well as the corresponding unit tests in:
+### Testing
+For each resource type, you will need to test write tests for a few separate areas:
+
+- Test the backend queries to ensure discovered resources come back (ie for `IAM::Policy`, write `tests.tests_iam.test_policy_list_config_discovered_resources`). For writing these tests, you must not make use of `boto` to create resources. You will need to use the backend model methods to provision the resources. This is to make tests compatible with the moto server. You must make tests for the resource type to test listing and object fetching.
 
-1. `tests/s3/test_s3.py`
-1. `tests/config/test_config.py`
+- Test the config dict for all scenarios (ie for `IAM::Policy`, write `tests.tests_iam.test_policy_config_dict`). For writing this test, you'll need to create resources in the same way as the first test (without using `boto`), in every meaningful configuration that would produce a different config dict. Then, query the backend and ensure each of the dicts are as you expect.
 
-Note for unit testing, you will want to add a test to ensure that you can query all the resources effectively. For testing this feature,
-the unit tests for the `ConfigQueryModel` will not make use of `boto` to create resources, such as S3 buckets. You will need to use the 
-backend model methods to provision the resources. This is to make tests compatible with the moto server. You should absolutely make tests
-in the resource type to test listing and object fetching.
+- Test that everything works end to end with the `boto` clients. (ie for `IAM::Policy`, write `tests.tests_iam.test_policy_config_client`). The main two items to test will be the `boto.client('config').list_discovered_resources()`, `boto.client('config').list_aggregate_discovered_resources()`, `moto.client('config').batch_get_resource_config()`, and `moto.client('config').batch_aggregate_get_resource_config()`. This test doesn't have to be super thorough, but it basically tests that the front end and backend logic all works together and returns correct resources. Beware the aggregate methods all have capital first letters (ie `Limit`), while non-aggregate methods have lowercase first letters (ie `limit`)
 
 ### Listing
 S3 is currently the model implementation, but it also odd in that S3 is a global resource type with regional resource residency.
@@ -117,4 +116,4 @@ return for it.
 When implementing resource config fetching, you will need to return at a minimum `None` if the resource is not found, or a `dict` that looks
 like what AWS Config would return.
 
-It's recommended to read the comment for the `ConfigQueryModel` 's `get_config_resource` function in [base class here](moto/core/models.py).
+It's recommended to read the comment for the `ConfigQueryModel` 's `get_config_resource` function in [base class here](moto/core/models.py).