Add memory pool configuration to datafusion-cli

[alamb]

Is your feature request related to a problem or challenge?

While trying to test #7400 with datafusion-cli I found I can't do it with datafuson-cli because datafusion-cli doesn't have a memory manager enabled.

Describe the solution you'd like

I would like to add two new new command line options to datafusion-cli

1. -m / --mem-limit that if set, would set the memory pool size limit. If unset no memory pool is used
2. --mem-pool-type=<greedy|fair>, defaults to greedy that specifies the pool type: GreedyMemoryPool or FairSpillPool respectively

Examples of usage

# memory is not limited
datafusion-cli -c 'select 1, 2 from foo';

# run query with greedy memory pool set to use 10G
datafusion-cli --memory-limit 10G -c 'select 1, 2 from foo'; 

# run query with greedy memory pool set to use 10G
datafusion-cli -m 10G -c 'select 1, 2 from foo'; 

# run query with fair memory pool set to use 10G
datafusion-cli --pool-type=fair -m 10G -c 'select 1, 2 from foo'; 

See https://docs.rs/datafusion/latest/datafusion/execution/memory_pool/struct.FairSpillPool.html for more details

Describe alternatives you've considered

I also thought about setting the pools via SET commands (like setting the target batch size). However, I don't think we should allow change memory limits via SQL because memory limits is likely not something a multi-tenant system would like to do . It should be setup before the session starts or by the runtime system, not the user in SQL

Additional context

Since this is well specified and is mostly an exercise in figuring out how datafusion-cli works, I think this would make a good first project

[Weijun-H]

I am glad to pick this ticket up this weekend.