enhancements to transfer operations for add'l checking and path handling.

[michael-conway]

Also, I DataTransferOperations.getOperation behaves a bit oddly. If I ask it to get a collection, it will not let me name the resulting directory. Say I make cal call to copy /demo/home/dooley/somefolder to /Users/dooley/deleteme. If the local folder exists, some folder will be copied into it. If it does not, it will be created and some folder copied in. That’s the opposite of what I’d expect on a posix file system and forces me to manually check the local path, create the target directory, list the remote path, and copy each collection or data object individually to the new target directory. All of that after I have to manually check permissions manually so I can pass back valid error messages to my users.

Here is the behavior of scp:

dooley$ scp -r docker:deleteme .
dooley$ ls
deleteme
dooley$ scp -r docker:deleteme deleteme
dooley$ ls deleteme
deleteme
dooley$ scp -r docker:deleteme samefoldernewname
dooley$ ls
deleteme
samefoldernewname

example handler in pseudocode

If the target exists, 
    if can write,if source is directory 
             if target is file
                     throw io exception
               else
                 create source within target and copy contents recursively 
       else if target is directory
                  copy source into target, overwriting matching file if present
      else
           overwrite target
    else
           throw permission exception
else if you can stat the target’s parent folder, 
        if can write, 
             if source is directory then create the source under the parent and copy contents recursively.
           else
                copy source under parent
        else
                throw permission exception
else
        throw file not found 

[michael-conway]

on copy of dir to file now throws an exception..

testing with icp

target exists

icp -r source target
gives /target/source/blah

icp -r source/. target
gives /target/blah

target not exists

icp -r source target

gives /target/blah

icp -r source/. target
gives /target/blah

[michael-conway]

testing with cp

(newdir has file1.txt and file2.txt)

target exists

(newdir has file1.txt and file2.txt)

cp -r newdir newtarget
gives /newtarget/file1.txt,file2.txt

cp -r newdir/. newtarget
gives /newtarget/file1.txt,file2.txt

target not exists

.. same behavior

[michael-conway]

hao testing on ubuntu is consistent between cp and irods icp

[michael-conway]

see as Rion's reference

 @Override
    public void get(String remotedir, String localdir, RemoteTransferListener listener) 
    throws IOException, RemoteDataException
    {
        try
        {
            IRODSFile irodsFile = getFile(remotedir);

            if (listener == null) {
                listener = new RemoteTransferListener(null);
            }

            if (irodsFile.exists())
            {
                File localDir = new File(localdir);

                if (irodsFile.isDirectory())
                {
                    if (!localDir.exists()) 
                    {
                        if (!localDir.getParentFile().exists()) {
                            throw new java.io.FileNotFoundException("No such file or directory");
                        } else {
                            // create the target directory 
                            if (!localDir.mkdir()) {
                                throw new IOException("Failed to create local download directory");
                            }
                        }

                        // recursively copy files into the local folder since irods won't let you specify 
                        // the target folder name 
                        List<CollectionAndDataObjectListingEntry> entries = 
                                geCollectionAndDataObjectListAndSearchAO().listDataObjectsAndCollectionsUnderPath(resolvePath(remotedir));

                        for (CollectionAndDataObjectListingEntry entry : entries)
                        {
                            getDataTransferOperations().getOperation(irodsFile.getAbsolutePath() + "/" + entry.getNodeLabelDisplayValue(), 
                                    localDir.getAbsolutePath() + File.separator + entry.getNodeLabelDisplayValue(), resource, listener, null);

                            TransferStatus statusCallback = listener.getOverallStatusCallback();
                            if (statusCallback != null && statusCallback.getTransferException() != null) {
                                throw statusCallback.getTransferException();
                            }
                        }
                    } 
                    else 
                    {
                        getDataTransferOperations().getOperation(irodsFile, 
                                localDir, listener, null);

                        TransferStatus statusCallback = listener.getOverallStatusCallback();
                        if (statusCallback != null && statusCallback.getTransferException() != null) {
                            throw statusCallback.getTransferException();
                        }
                    }
                }
                else
                {
                    getDataTransferOperations().getOperation(irodsFile, 
                            localDir, listener, null);

                    TransferStatus statusCallback = listener.getOverallStatusCallback();
                    if (statusCallback != null && statusCallback.getTransferException() != null) {
                        throw statusCallback.getTransferException();
                    }   
                }
            }
            else 
            {
                throw new java.io.FileNotFoundException("No such file or directory");
            }
        } 
        catch (FileNotFoundException e) {
            throw new java.io.FileNotFoundException("No such file or directory");
        }
        catch (CatNoAccessException e) {
            throw new RemoteDataException("Failed to get " + remotedir + " due to insufficient privileges.", e);
        } 
        catch (DataNotFoundException e) {
            throw new java.io.FileNotFoundException("No such file or directory");
        } 
        catch (IOException e) {
            throw e;
        }
        catch (JargonException e) {
            throw new RemoteDataException("Failed to get file from irods.", e);
        } 
        catch (RemoteDataException e) {
            throw e;
        }
        catch (Exception e) {
            throw new RemoteDataException("Failed to copy file to irods.", e);
        }

    }

[michael-conway]

Useful integration of io utils for path normalization worth considering.

[michael-conway]

added dir create and folder permission checks to get and put code