Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow to pass protoc command line arguments via a file or stdin #274

Closed
sergei-ivanov opened this issue Apr 8, 2015 · 7 comments
Closed

Comments

@sergei-ivanov
Copy link
Contributor

This has been a recurring source of pain on Windows for ages: in large projects with lots of .proto definitions or lots of .proto imports it is pretty easy to reach the command line length limit on Windows, which is ridiculously small. We've developed a number of workarounds for this problem in the Maven plugin, but there are still reports that some projects are struggling.
A simple solution that would have allowed us to solve that issue once and forever would be to pass a file with the actual command line arguments on the command line, or alternatively pipe them in.

@xfxyjwf
Copy link
Contributor

xfxyjwf commented Apr 8, 2015

If you have a lot of .proto files, you don't have to invoke protoc on all of them in a single pass. Instead, you can invoke protoc once for each of them.

@sergei-ivanov
Copy link
Contributor Author

I know, it is one of the workarounds that we either considered or already implemented. It may be less efficient, but it gets things done. I think the latest problem that one of the clients ran into was related to a large number of import paths (-I=xxxxxx), which busts the command line limit. Again, I think we can work around it in some creative way, but it would be much easier if we could run something like protoc @args.tmp where args.tmp would be a text file containing a newline-separated list of command line arguments.

@xfxyjwf
Copy link
Contributor

xfxyjwf commented Mar 7, 2017

We actually have this feature internally already.

@sergei-ivanov
Copy link
Contributor Author

This is amazingly good news. I shall add support to the Maven plugin as soon as it is publicly available in protoc.

zhangkun83 pushed a commit to google/protobuf-gradle-plugin that referenced this issue Nov 22, 2017
Spread proto generation over multiple protoc calls on systems that limit command line length, e.g., Windows. Once the command length exceeds the OS limit, it calls with the current arguments and resets the builder.

This is related to #167, allowing this issue to be avoided when many proto files are present, until protocolbuffers/protobuf#274 is completed.
@liujisi
Copy link
Contributor

liujisi commented Dec 11, 2017

I believe this is already implemented.

@sergei-ivanov
Copy link
Contributor Author

I've just tested it with protoc 3.5.0, and it appears to work beautifully. Is 3.5.0 the earliest version where this feature has become publicly available?
Many thanks for implementing this.

@xfxyjwf
Copy link
Contributor

xfxyjwf commented Dec 18, 2017

@sergei-ivanov Yes, 3.5.0 is the first version with this feature.

zhangkun83 pushed a commit to zhangkun83/protobuf-gradle-plugin-1 that referenced this issue Nov 7, 2018
Spread proto generation over multiple protoc calls on systems that limit command line length, e.g., Windows. Once the command length exceeds the OS limit, it calls with the current arguments and resets the builder.

This is related to google#167, allowing this issue to be avoided when many proto files are present, until protocolbuffers/protobuf#274 is completed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

4 participants