azure troubles #4

psifertex · 2014-07-25T02:18:28Z

Was trying a manual setup of a streisand instance on azure, but unfortunately, it looks like obfsproxy port changes and needs to be added manually, but this hangs the setup process? Or maybe that step takes a long time.

Additionally, the correct external IP address (which is specified manually in my inventory) isn't used for some of the service configurations, but rather, the internal IP address of the azure node is, which isn't helpful since it won't be routable.

psifertex · 2014-07-25T02:20:08Z

I'm planning on trying to work through these if I get a chance, but figured I'd at least file the big here so I can either propose a patch or comment on my fixes. Only downside is I've been avoiding using the azure cli since it looked obnoxious but will probably have to if this is to get solved the right way.

jlund · 2014-07-25T06:36:15Z

Obfsproxy does choose random ports for both the obfs3 and ScrambleSuit pluggable transports. It shouldn't hang the setup process though, but I haven't tried running this on Azure yet.

Amazon AWS had the same issue with external/internal IP address discovery, and the server name on AWS was also not being set correctly. If you look in playbooks/roles/common/tasks/set-default-variables.yml you can see the logic that fixes this. All of the 'genesis' roles have tasks that set things appropriately before the common role gets hits and these conditionals are evaluated. The fix for Azure would likely be similar.

Azure and Google Compute Engine are next on my list of platforms to support. Ansible has modules for them, and I don't think it would take too long. I really appreciate you documenting what you have found so far. It is very helpful.

psifertex · 2014-07-28T20:08:13Z

For what it's worth, the install really does timeout and dies when the obfsproxy fails to start up properly (due to the aforementioned random port and azure):

failed: [hostname.cloudapp.net] => (item=obfs3) => {"elapsed": 300, "failed": true, "item": "obfs3"}
msg: Timeout when waiting for search string obfs3 in /var/lib/tor/state

I've just added a manual pause so I can fix it up by hand to see if that fixes it.

Also, I should note that so far, my changes involved:

root -> azureuser (as well as adding sudo: yes)
editing defaults to the set-default-variables as you described above. For now, I'm just putting in my azure hostname and IP address in by hand.

Running it again while trying to see why it wasn't connecting and I don't know what's not working. I've added all listening ports to the endpoint connection list on the azure interface, but it still times out. Interestingly, this time I get the same error message as above, but it continues on and tries REMOTE_MODULE wait_for path=/var/lib/tor/state search_regex=scramblesuit which I don't remember it trying before.

psifertex · 2014-07-29T15:43:40Z

Ok, I've worked through the azure command-line setup (and stumbled across an annoying bug in the process) and have the following setup notes:

Azure Command-Line Setup

npm install azure-cli --global
npm install azure --global

azure account download
azure account import path-to-credentials.publishsettings

Addendum 1: This bug: https://github.com/Azure/azure-sdk-tools-xplat/issues/1293 means the above scripts don’t work right now depending on what version of the command-line tools you have. D’oh! Not sure how to best work around that? Suggest people git clone the Azure repo directly, maybe? I dunno, I’ve just patched the fix in by hand myself. (The latest version fixes these related bugs)

Streisand Steps

//First generate a random hostname (azure instances get a hostname.cloudapp.net domain), username, and password.
//Maybe prompt the user for the zone they prefer? Or you can query to see if they have existing zones and use those? Not sure how it's handled in the EC2 playbook, probably easiest to replicate that.

// It's probably best to make a fresh ssh keypair for the newly created image and add the pubkey into the .ssh/config or just manually specify it. Again, I'd just go with whatever you do for EC2. Here's the conversion line though if needed (and once their bug below is fixed):
// openssl rsa -in ~/.ssh/id_rsa -outform pem > ~/.ssh/azure.pem

//The following doesn't work, waiting for https://github.com/Azure/azure-sdk-tools-xplat/issues/1284 to be fixed.
// azure vm create -l "East US" -z extrasmall --no-ssh-password --ssh-cert ~/.ssh/azure.pem -n $hostname $hostname b39f27a8b8c64d52b05eac6a62ebad85__Ubuntu-14_04-LTS-amd64-server-20140724-en-us-30GB $username

azure vm create -l "East US" -z extrasmall -n $hostname $hostname b39f27a8b8c64d52b05eac6a62ebad85__Ubuntu-12_04-LTS-amd64-server-20140724-en-us-30GB $username $password

//Note that extra small is a bit anemic and you will likely get some performance boost (and cost increase) out of using a small instance instead. 
//If you want to run 14.04 (which has bugs and won't work), just change the text of the image name in the above command.

//wait until "azure vm list" shows that machine with Status “ReadyRole”
// Sample output: 
// data:    Name     Status              Location  DNS Name              IP Address
// data:    -------  ------------------  --------  --------------------  -------------
// data:    streisn  RoleStateUnknown    East US   streisn.cloudapp.net  100.10.10.10
//
//Note that the IP address is not publicly routable, I’m querying the name servers (dig +short a $hostname.cloudapp.net @prd1.azuredns-cloud.net) to find it out, but there might be a way to do it directly

Firewall adjustment

azure vm endpoint create-multiple $hostname 636:636,8888:8888,443:443,8443:8443,16001:16001,1701:1701:UDP,$obfs3port:$obfs3port
//needs to be adjusted for the actual ports, can just steal from the ec2 setup, I'm sure.

Addendum 1: The VM commands sometimes fail, so it’d be good to have a retry count so that if they fail they are re-run a certain number of times.

psifertex · 2014-07-30T11:48:33Z

So this is still failing for me now that I've had more time to play with it. Same error as before:

failed: [191.238.11.40] => (item=obfs3) => {"elapsed": 300, "failed": true, "item": "obfs3"}
msg: Timeout when waiting for search string obfs3 in /var/lib/tor/state

and:

failed: [191.238.11.40] => (item=scramblesuit) => {"elapsed": 300, "failed": true, "item": "scramblesuit"}
msg: Timeout when waiting for search string scramblesuit in /var/lib/tor/state

FATAL: all hosts have already failed -- aborting

jlund · 2014-07-31T06:49:23Z

Thanks again for your help with this.

Are you using Ubuntu 14.04 instead of Debian 7 by any chance?

psifertex · 2014-07-31T07:52:30Z

Yup! Unfortunately, Azure doesn't have any Debian images.

jlund · 2014-07-31T14:35:57Z

Do they have any other Ubuntu images? There are known issues with Tor and 14.04 when pluggable transports are being used, which is why the port information never shows up in the state file.

On July 31, 2014 1:52:32 AM MDT, psifertex [email protected] wrote:

Yup! Unfortunately, Azure doesn't have any Debian images.

Reply to this email directly or view it on GitHub:
#4 (comment)

psifertex · 2014-07-31T14:39:16Z

Sure, easy enough to try a 14.10 alpha, or go older. Any suggestion as to which is preferable?

(Updated to add): I fixed the first bug by following the advice from https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=739279 which lets obfs3 run (actually, the first time around I just shut down apparmor entirely), but it still hung on scramblesuit, so I got past that by cheating and simply running: echo "scramblesuit" >> /var/log/tor/state as root. The port will be wrong, but it at least got me my first almost working streisand system and I'm not as concerned about not running scramblesuit initially.

Found one other minor bug which is that ansible complains about the ansible_system variable, not sure what might be causing that, this is something on my OS X host I assume, though it looks like it should be able to handle Darwin just fine as a result, this looks more like a version problem with ansible itself? Not sure, but I followed the install instructions pretty carefully for my native box.

fatal: [168.62.176.220] => error while evaluating conditional: hostvars['127.0.0.1']['ansible_system'] == "Linux"

jlund · 2014-07-31T14:57:09Z

I would try 12.04 if they have it. 'Precise' is still getting security updates and will be for another few years. It could be a good alternative base to have tested for providers that don't offer Debian 7.

That conditional is looking for the OS value from the system running the playbook, yeah. This is collected by Ansible during the "Gathering Facts" phase. What version of OS X are you on? I am considering removing those tasks. They happen after everything has been completely set up, and they seem to be error prone. All they do is open the HTML instructions at the end which is cool but might not be worth the error messages for some users.

On July 31, 2014 8:39:16 AM MDT, psifertex [email protected] wrote:

Sure, easy enough to try a 14.10 alpha, or go older. Any suggestion as
to which is preferable?

I got past the bug by cheating and simply running: echo "scramblesuit"

/var/log/tor/state as root. The port will be wrong, but it at least
got me my first almost working streisand system. ;-)

Found one other minor bug which is that ansible complains about the
ansible_system variable, not sure what might be causing that, this is
something on my OS X host I assume, though it looks like it should be
able to handle Darwin just fine as a result, this looks more like a
version problem with ansible itself? Not sure, but I followed the
install instructions pretty carefully for my native box.

fatal: [168.62.176.220] => error while evaluating conditional: hostvars['127.0.0.1']['ansible_system'] == "Linux"

Reply to this email directly or view it on GitHub:
#4 (comment)

psifertex · 2014-08-01T00:04:12Z

So I (sorta) got it working by disabling apparmor which got past the obfs3 error, but then had to manually echo into the state the scramblesuit plugin. Way, way too ugly.

Ok, so on to 12.04. I'll update the instructions in #4 (comment) with the filename of a 12.04 image, and update this comment in a few minutes when I've tested it out.

psifertex · 2014-08-01T00:23:12Z

Also, running the latest OS X Mavericks (10.9.4).

I don't really know much about ansible yet to be able to troubleshoot that system variable, looks like the bug is entirely within ansible or something it's doing as far as I can tell.

joaocc · 2016-06-05T00:57:41Z

Hi.
I started working on Azure support here (joaocc@95fba56).
It may take a few days, and all help is welcomed.
J

blotzu · 2017-04-11T15:45:10Z

Hi guys, I finished the Azure implementation here: (#598)

jlund · 2017-04-15T00:34:21Z

Azure support was just merged into the master branch. Thanks to @blotzu and @joaocc for making it happen!

# This is the 1st commit message: initial fr commit # This is the commit message StreisandEffect#2: readme completed # This is the commit message StreisandEffect#3: corrections # This is the commit message StreisandEffect#4: readme done # This is the commit message StreisandEffect#5: readme even more done # This is the commit message StreisandEffect#6: more corrections # This is the commit message StreisandEffect#7: initial commit for translated instructions # This is the commit message StreisandEffect#8: fixed typos # This is the commit message StreisandEffect#9: fixed markdown # This is the commit message StreisandEffect#10: fixed anchor # This is the commit message StreisandEffect#11: grammar and typos # This is the commit message StreisandEffect#12: fixed markdown # This is the commit message StreisandEffect#13: finished french instructions for windows l2tp/ipsec # This is the commit message StreisandEffect#14: androïde # This is the commit message StreisandEffect#15: finished osx instructions for l2tp/ipsec # This is the commit message StreisandEffect#16: finished linux l2tp/ipsec # This is the commit message StreisandEffect#17: finished ios l2tp/ipsec # This is the commit message StreisandEffect#18: common entries in l2tp translated # This is the commit message StreisandEffect#19: minor corrections # This is the commit message StreisandEffect#20: android l2tp/ipsec translations done # This is the commit message StreisandEffect#21: l2tp/ipsec french done # This is the commit message StreisandEffect#22: typos # This is the commit message StreisandEffect#23: initial commit for openconnect # This is the commit message StreisandEffect#24: partial windows instructions completed # This is the commit message StreisandEffect#25: removed idea files # This is the commit message StreisandEffect#26: renamed readme_fr to readme-fr # This is the commit message StreisandEffect#27: windows oc instructions done # This is the commit message StreisandEffect#28: finished osx instructions, updated brew url to https # This is the commit message StreisandEffect#29: oc linux instructions done, minor corrections # This is the commit message StreisandEffect#30: stuff # This is the commit message StreisandEffect#31: oc instructions french done # This is the commit message StreisandEffect#32: minor correction # This is the commit message StreisandEffect#33: initial commit for openvpn instructions fr # This is the commit message StreisandEffect#34: finished windows ovpn instructions # This is the commit message StreisandEffect#35: copypasta the common stuff # This is the commit message StreisandEffect#36: typos # This is the commit message StreisandEffect#37: removed azure from upcoming features fr # This is the commit message StreisandEffect#38: updated azure Readme-fr.md # This is the commit message StreisandEffect#39: Silence la censure. Automatiser l'effet # This is the commit message StreisandEffect#40: minor corrections # This is the commit message StreisandEffect#41: minor corrections # This is the commit message StreisandEffect#42: more translations # This is the commit message StreisandEffect#43: finished macos ovpn instructions # This is the commit message StreisandEffect#44: eeeeeeeeee # This is the commit message StreisandEffect#45: openvpn linux cli done # This is the commit message StreisandEffect#46: finish linux ovpn network manager # This is the commit message StreisandEffect#47: ovpn android done # This is the commit message StreisandEffect#48: finished ovpn ios instructions # This is the commit message StreisandEffect#49: minor correction # This is the commit message StreisandEffect#50: initial commit for shadowsocks-fr # This is the commit message StreisandEffect#51: windows shadowsocks done # This is the commit message StreisandEffect#52: shadowsocks macos done # This is the commit message StreisandEffect#53: really macos is done # This is the commit message StreisandEffect#54: commit before merge # This is the commit message StreisandEffect#55: finished ff shadowsocks instructions # This is the commit message StreisandEffect#56: shadowsocks done # This is the commit message StreisandEffect#57: initial commit for wg fr # This is the commit message StreisandEffect#58: initial commit for ssh fr # This is the commit message StreisandEffect#59: windows ssh instructions done # This is the commit message StreisandEffect#60: linux-osx ssh done # This is the commit message StreisandEffect#61: linux osx alternative ssh done # This is the commit message StreisandEffect#62: android almost done # This is the commit message StreisandEffect#63: finished instructions for shadowsocks # This is the commit message StreisandEffect#64: mirror index fr done # This is the commit message StreisandEffect#65: tunnel ssh # This is the commit message StreisandEffect#66: progress for streisang-gateway fr # This is the commit message StreisandEffect#67: finished windows gateway instructions # This is the commit message StreisandEffect#68: manual ssl verification done # This is the commit message StreisandEffect#69: one more line # This is the commit message StreisandEffect#70: finished osx android gateway fr # This is the commit message StreisandEffect#71: ios gateway done # This is the commit message StreisandEffect#72: finished all of gateway instructions fr # This is the commit message StreisandEffect#73: initial commit for tor-fr # This is the commit message StreisandEffect#74: tor done # This is the commit message StreisandEffect#75: initial commit for azure-fr.md # This is the commit message StreisandEffect#76: azure-fr done # This is the commit message StreisandEffect#77: preparing to link it all up # This is the commit message StreisandEffect#78: working on i18n generation # This is the commit message StreisandEffect#79: Streisand CI Testing # This is the commit message StreisandEffect#80: attempt i18n.. # This is the commit message StreisandEffect#81: removed commented stuff # This is the commit message StreisandEffect#82: fixed lang loop in md templates # This is the commit message StreisandEffect#83: spelling # This is the commit message StreisandEffect#84: regex_replace # This is the commit message StreisandEffect#85: regex_replace # This is the commit message StreisandEffect#86: langauge_name # This is the commit message StreisandEffect#87: item in loop # This is the commit message StreisandEffect#88: trying iteritems # This is the commit message StreisandEffect#89: ocserv i18n # This is the commit message StreisandEffect#90: openvpn i18n # This is the commit message StreisandEffect#91: fixed stunnel status # This is the commit message StreisandEffect#92: updated shadowsocks instructions # This is the commit message StreisandEffect#93: shadowsocks i18n # This is the commit message StreisandEffect#94: finished ssh i18n # This is the commit message StreisandEffect#95: finished gateway i18n # This is the commit message StreisandEffect#96: adding index and firewall fr # This is the commit message StreisandEffect#97: finished tor i18n, adding .html to templates # This is the commit message StreisandEffect#98: finished wg i18n # This is the commit message StreisandEffect#99: wg with_dict not with_items # This is the commit message StreisandEffect#100: fixed streisand mirror index task # This is the commit message StreisandEffect#101: fixed shadowsocks mirror index fr # This is the commit message #102: streisand_markdown_mirror_page removed # This is the commit message StreisandEffect#103: streisand_ci = no # This is the commit message StreisandEffect#104: fixed tor gateway instructions # This is the commit message StreisandEffect#105: hopefully fixes all page generation issues # This is the commit message StreisandEffect#106: updated index-fr.html # This is the commit message StreisandEffect#107: l2tp oconnect should generate correctly now # This is the commit message StreisandEffect#108: fixed mirror links for i18n # This is the commit message StreisandEffect#109: finished remaining translation, mirror link fixes # This is the commit message StreisandEffect#110: fixed stunnel instructions # This is the commit message StreisandEffect#111: hopefully the last of the corrections # This is the commit message StreisandEffect#112: typos and grammatics # This is the commit message StreisandEffect#113: more corrections # This is the commit message StreisandEffect#114: mirroring french localized tor # This is the commit message StreisandEffect#115: minor corrections # This is the commit message StreisandEffect#116: clarifications # This is the commit message StreisandEffect#117: Somme de contrôle # This is the commit message StreisandEffect#118: s/potatso/cross # This is the commit message StreisandEffect#119: changed google links for fr to hl=fr # This is the commit message StreisandEffect#120: copypasta quirk # This is the commit message StreisandEffect#121: cleanup, removed shadowsocks OTA # This is the commit message StreisandEffect#122: removed shadowsocks from unattended upgrades # This is the commit message StreisandEffect#123: removed unecessary stuff from unattended upgrades # This is the commit message StreisandEffect#124: Streisand CI Testing # This is the commit message StreisandEffect#125: CI updates based on feedback # This is the commit message StreisandEffect#126: permissions # This is the commit message StreisandEffect#127: updated readme-fr # This is the commit message StreisandEffect#128: Update readme-fr No need for brew update anymore # This is the commit message StreisandEffect#129: removed ci

psifertex mentioned this issue Sep 16, 2014

Support ubuntu 14.04 #73

Closed

joaocc mentioned this issue Jun 5, 2016

Native Azure Support #304

Closed

cpu added area/provisioning area/tor kind/feature labels Apr 8, 2017

jlund closed this as completed Apr 15, 2017

crysis30300 mentioned this issue Nov 10, 2017

OpenConnect windows client error -- DTLS handshake failed: Resource temporarily unavailable, try again. #1049

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

azure troubles #4

azure troubles #4

psifertex commented Jul 25, 2014

psifertex commented Jul 25, 2014

jlund commented Jul 25, 2014

psifertex commented Jul 28, 2014

psifertex commented Jul 29, 2014

psifertex commented Jul 30, 2014

jlund commented Jul 31, 2014

psifertex commented Jul 31, 2014

jlund commented Jul 31, 2014

psifertex commented Jul 31, 2014

jlund commented Jul 31, 2014

psifertex commented Aug 1, 2014

psifertex commented Aug 1, 2014

joaocc commented Jun 5, 2016

blotzu commented Apr 11, 2017

jlund commented Apr 15, 2017

azure troubles #4

azure troubles #4

Comments

psifertex commented Jul 25, 2014

psifertex commented Jul 25, 2014

jlund commented Jul 25, 2014

psifertex commented Jul 28, 2014

psifertex commented Jul 29, 2014

Azure Command-Line Setup

Streisand Steps

Firewall adjustment

psifertex commented Jul 30, 2014

jlund commented Jul 31, 2014

psifertex commented Jul 31, 2014

jlund commented Jul 31, 2014

psifertex commented Jul 31, 2014

jlund commented Jul 31, 2014

psifertex commented Aug 1, 2014

psifertex commented Aug 1, 2014

joaocc commented Jun 5, 2016

blotzu commented Apr 11, 2017

jlund commented Apr 15, 2017