Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

*: update build process and ssh in bench.py #1204

Closed

Conversation

martingrambow
Copy link
Contributor

There were some small errors in bench.py (fixes #1202):

  • ssh_connect_with_retries() did not ask for the ssh key
  • I also added some logging in ssh_cmd()
  • bootstrap(addr) followed the old build process
  • run() called the async ssh command, however, sometimes the pkill killed the deamon immediately; so I switched to the usual ssh_cmd and to screen

@ploxiln
Copy link
Member

ploxiln commented Oct 25, 2019

Clearly we have not maintained this script in the last couple years. Thanks.

bench/bench.py Outdated Show resolved Hide resolved
@ploxiln
Copy link
Member

ploxiln commented Oct 25, 2019

while you're here, the defaults for the instance_type and ami options should probably be updated

@martingrambow
Copy link
Contributor Author

Thanks for the feedback, I think I fixed all the issues.

@@ -287,11 +289,11 @@ def decomm():
help='AWS account access key')
tornado.options.define('secret_key', type=str, default='',
help='AWS account secret key')
tornado.options.define('ami', type=str, default='ami-48fd2120',
tornado.options.define('ami', type=str, default='ami-02df9ea15c1778c9c',
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This ami does not seem to exist in the default region, us-east-1 (and I also checked a couple of other common regions and could not find it). Can you find an ami from the default region?

$ aws --region us-east-1 ec2 describe-images --image-ids ami-48fd2120
{
    "Images": [
        {
            "Architecture": "x86_64",
            "CreationDate": "2014-08-16T21:40:48.000Z",
            "ImageId": "ami-48fd2120",
            "ImageLocation": "ubuntu-us-east-1/images/hvm-instance/ubuntu-trusty-14.04-amd64-server-20140816.manifest.xml",
            "ImageType": "machine",
            "Public": true,
            "OwnerId": "099720109477",
            "State": "available",
            "BlockDeviceMappings": [],
            "Hypervisor": "xen",
            "Name": "ubuntu/images/hvm-instance/ubuntu-trusty-14.04-amd64-server-20140816",
            "RootDeviceType": "instance-store",
            "SriovNetSupport": "simple",
            "VirtualizationType": "hvm"
        }
    ]
}
$ aws --region us-east-1 ec2 describe-images --image-ids ami-02df9ea15c1778c9c

An error occurred (InvalidAMIID.NotFound) when calling the DescribeImages operation: The image id '[ami-02df9ea15c1778c9c]' does not exist

(I guess we should update to ubuntu-16.04 or ubuntu-18.04, but you probably did that, I just can't tell because I can't find the ami ...)

help='AMI ID')
tornado.options.define('ssh_key_name', type=str, default='default',
help='SSH key name')
tornado.options.define('instance_type', type=str, default='c3.2xlarge',
tornado.options.define('instance_type', type=str, default='t2.medium',
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think c4.xlarge or c5.xlarge would be more in the spirit of the previous default. A bit expensive, but they'll run for less than an hour, and t2 or t3 instances are probably not good for benchmarking because they are more oversubscribed and throttled.

cmd = bootstrap
if (cmd_name == 'run'):
cmd = run
if (cmd_name == 'decomm'):
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This part did not have to change. The restructure is OK with me, except that using parentheses like this is odd and unusual for python (they're not needed).

@mreiferson mreiferson changed the title update build process and ssh in bench.py *: update build process and ssh in bench.py Jun 14, 2020
@mreiferson mreiferson added the bug label Jun 14, 2020
@mreiferson
Copy link
Member

see #1264

@mreiferson mreiferson closed this Jun 14, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

nsqd: ec2 benchmark is broken
3 participants