Skip to content
This repository has been archived by the owner on Jun 6, 2024. It is now read-only.

cannot access webhdfs from openmind #1314

Closed
DongZhaoYu opened this issue Sep 10, 2018 · 4 comments
Closed

cannot access webhdfs from openmind #1314

DongZhaoYu opened this issue Sep 10, 2018 · 4 comments
Assignees

Comments

@DongZhaoYu
Copy link
Member

When putting data from openmind to webhdfs, the returned location header is :

http://xxx.xxx.xxx.xxx/a/xxx.xxx.xxx.xxx/50075/webhdfs/v1/user/code/code.py?op=CREATE&user.name=xxx&namenoderpcaddress=xxx.xxx.xxx.xxx:9000&createflag=&createparent=true&overwrite=false

webhdfs client cannot use this url to put data. the error is :
user.name=user: command not found

The cause of this problem:
the request is received by pylon and pylon cannot handle it.

We need to first confirm this address is correct. If it is correct, we should fix pylon to support this.

@DongZhaoYu DongZhaoYu self-assigned this Sep 10, 2018
@fanyangCS
Copy link
Contributor

@Gerhut , please have a check.

@Asakuri
Copy link
Contributor

Asakuri commented Sep 10, 2018

return 307 $scheme://$http_host/a/$1:$2/$3;
https://github.com/Microsoft/pai/blob/a59f05fe32934dd1164c26caafc0676f0763b692/pylon/src/nginx.conf.template#L108
and you should add quotes around the URL when you access it in shell using curl or something else

@Gerhut
Copy link
Member

Gerhut commented Sep 10, 2018

I'll open a pull request to fix this.

@TobeyQin
Copy link

Thank you for help resolve the issue. @Asakuri @DongZhaoYu @Gerhut

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

No branches or pull requests

5 participants