Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add an overload of PipeSource.FromCommand(...) that accepts a transform function #275

Merged
merged 2 commits into from
Feb 7, 2025

Conversation

Tyrrrz
Copy link
Owner

@Tyrrrz Tyrrrz commented Feb 7, 2025

This overload allows the caller to pipe the output of one command into the input of another, while applying a manual transform between the source and the destination data.

Currently, the transform is a rudimentary Func<Stream, Stream, CancellationToken, Task> low-level function which works with streams directly (output of the first command and the input of the second command, respectively). In the future, we can build upon this foundation to provide higher level transforms, but right now I'm not sure if there's much demand for it.

Example usage:

[Fact(Timeout = 15000)]
public async Task I_can_execute_a_command_and_pipe_the_stdin_from_another_command_with_a_transform()
{
    // Arrange
    var cmdInput = Cli.Wrap(Dummy.Program.FilePath)
        .WithArguments(["generate binary", "--length", "100000"]);

    var cmd = Cli.Wrap(Dummy.Program.FilePath)
        .WithArguments("length stdin")
        .WithStandardInputPipe(
            PipeSource.FromCommand(
                cmdInput,
                // Take only the first 5000 bytes
                async (source, destination, cancellationToken) =>
                {
                    using var buffer = MemoryPool<byte>.Shared.Rent(5000);

                    await source.ReadAtLeastAsync(
                        buffer.Memory,
                        5000,
                        false,
                        cancellationToken
                    );

                    await destination.WriteAsync(buffer.Memory[..5000], cancellationToken);
                }
            )
        );

    // Act
    var result = await cmd.ExecuteBufferedAsync();

    // Assert
    result.StandardOutput.Trim().Should().Be("5000");
}

The above example demonstrates how the command generate binary is piped into length stdin and the transform is responsible for cutting off all but the first 5000 bytes. Of course, you can perform much more complicated transformations here as well, and the fact that the operations are performed against streams (rather than materialized data) makes these operations memory-efficient.

Closes #206

@Tyrrrz Tyrrrz requested a review from Copilot February 7, 2025 20:47

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot reviewed 2 out of 2 changed files in this pull request and generated no comments.

Comments suppressed due to low confidence (1)

CliWrap/PipeSource.cs:137

  • [nitpick] The parameter name 'copyStreamAsync' is ambiguous. It should be renamed to 'transformStreamAsync' to better describe its purpose.
Func<Stream, Stream, CancellationToken, Task> copyStreamAsync
Copy link

codecov bot commented Feb 7, 2025

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 95.05%. Comparing base (a1aa848) to head (fc326a1).
Report is 1 commits behind head on master.

Additional details and impacted files
@@            Coverage Diff             @@
##           master     #275      +/-   ##
==========================================
+ Coverage   95.00%   95.05%   +0.05%     
==========================================
  Files          46       46              
  Lines        1100     1113      +13     
  Branches       85       85              
==========================================
+ Hits         1045     1058      +13     
  Misses         35       35              
  Partials       20       20              

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@Tyrrrz Tyrrrz merged commit a6ad926 into master Feb 7, 2025
7 checks passed
@Tyrrrz Tyrrrz deleted the intermediate-handlers branch February 7, 2025 20:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Support intermediate pipe handlers to transform output from one command into input for another
1 participant