Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: support specifying columns to include in the "Attach table" statement #17442

Merged

Conversation

dantengsky
Copy link
Member

@dantengsky dantengsky commented Feb 12, 2025

I hereby agree to the terms of the CLA available at: https://docs.databend.com/dev/policies/cla/

Summary

Allow optionally specifying which columns to include in the 'Attach table' statement.

e.g.

To create a table named test_tbl that attaches to the path 's3://databend-toronto/1/556/' and includes only the columns col1 and col2, you can use the following SQL statement:

ATTACH TABLE `test_tbl`  (col1, col2) 's3://databend-toronto/1/556/';

Suppose the table base is being attached:


create or replace table base(c1 string, c2 string, c3 string, c4 string) ...

  • Try Including columns which do not exist in the base table will fail

e.g. attach table attach_tbl (c2, c_not_exist) .... will fail

  • Dropping non-included columns from base does not matter

    e.g.

     -- attach to base table
    attach table attach_tbl (c2, c4) ....; 
    
    alter table base drop column c1;
    
    -- should still work
    select * from attach_tbl;
    
  • Renaming included columns of base does not matter

    e.g.

     -- attach to base table
    attach table attach_tbl (c2, c4) ....; 
    
    alter table base rename column c2 to c2_new;
    
    -- should still work
    select * from attach_tbl;
    select c2_new from attach_tbl;
    
    -- but the old column name is NO longer available
    select c2 from attach_tbl; -- this will fail
    
  • Column that have been dropped from base table will be inaccessible

    e.g.

     -- attach to base table
    attach table attach_tbl (c2_new, c4) ....; 
    
    alter table base drop column c4;
    
    -- this will fail
    select c4 from attach_tbl;
    
    -- but this should work
    select c2_new from attach_tbl;
    

    select * should still work, as long as there is at least one column left:

     -- should still work
    select * from attach_tbl;
    

    if all the attached columns have been dropped from the base table, attach_tbl can no longer be scanned:

    -- remove c4 from base
    alter table base drop column c4;
    
    -- this will fail
    select * from attach_tbl; 
    
  • fixes: Feature: Support specifying columns to include in the "Attach table" statement. #17439

Tests

  • Unit Test
  • Logic Test
  • Benchmark Test
  • No Test - Explain why

Type of change

  • Bug Fix (non-breaking change which fixes an issue)
  • New Feature (non-breaking change which adds functionality)
  • Breaking Change (fix or feature that could cause existing functionality not to work as expected)
  • Documentation Update
  • Refactoring
  • Performance Improvement
  • Other (please describe):

This change is Reviewable

@github-actions github-actions bot added the pr-feature this PR introduces a new feature to the codebase label Feb 12, 2025
@dantengsky dantengsky changed the title feat: support specifying columns to include in the "Attach table" sta… feat: support specifying columns to include in the "Attach table" statement Feb 13, 2025
@dantengsky dantengsky force-pushed the feat-specify-cols-in-attach-stmt branch from eaaf5c3 to 4d128da Compare February 14, 2025 11:55
@dantengsky dantengsky marked this pull request as ready for review February 14, 2025 13:48
@dantengsky dantengsky requested a review from wubx February 14, 2025 13:48
@wubx
Copy link
Member

wubx commented Feb 14, 2025

LGTM

@dantengsky dantengsky merged commit 67d4a16 into databendlabs:main Feb 15, 2025
70 checks passed
@wubx
Copy link
Member

wubx commented Feb 15, 2025

cc @soyeric128 need update docs

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
pr-feature this PR introduces a new feature to the codebase
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Feature: Support specifying columns to include in the "Attach table" statement.
2 participants